{"title":"Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels","authors":"Yongquan Yang","doi":"10.1007/s10115-023-02047-6","DOIUrl":null,"url":null,"abstract":"<p>Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult to be precisely defined or even do not exist. To alleviate this situation, we propose logical assessment formula (LAF) and reveal its principles for evaluations with inaccurate ground-truth labels (IAGTLs) via logical reasoning under uncertainty. From the revealed principles of LAF, we summarize the practicability of LAF: (1) LAF can be applied for evaluations with IAGTLs on a more difficult task, able to act like usual strategies for evaluations with AGTLs reasonably; (2) LAF can be applied for evaluations with IAGTLs from the logical perspective on an easier task, unable to act like usual strategies for evaluations with AGTLs confidently.</p>","PeriodicalId":54749,"journal":{"name":"Knowledge and Information Systems","volume":"36 1","pages":""},"PeriodicalIF":2.5000,"publicationDate":"2024-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Knowledge and Information Systems","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s10115-023-02047-6","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult to be precisely defined or even do not exist. To alleviate this situation, we propose logical assessment formula (LAF) and reveal its principles for evaluations with inaccurate ground-truth labels (IAGTLs) via logical reasoning under uncertainty. From the revealed principles of LAF, we summarize the practicability of LAF: (1) LAF can be applied for evaluations with IAGTLs on a more difficult task, able to act like usual strategies for evaluations with AGTLs reasonably; (2) LAF can be applied for evaluations with IAGTLs from the logical perspective on an easier task, unable to act like usual strategies for evaluations with AGTLs confidently.
期刊介绍：
Knowledge and Information Systems (KAIS) provides an international forum for researchers and professionals to share their knowledge and report new advances on all topics related to knowledge systems and advanced information systems. This monthly peer-reviewed archival journal publishes state-of-the-art research reports on emerging topics in KAIS, reviews of important techniques in related areas, and application papers of interest to a general readership.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
