Marcello Chang, Joshua J. Reicher, Angad Kalra, Michael Muelly, Yousef Ahmad
{"title":"使用临床和病理学支持的诊断标签,分析机器学习分类器在 CT 上无明确或可能的通常间质性肺炎模式的间质性肺病病例中的验证性能","authors":"Marcello Chang, Joshua J. Reicher, Angad Kalra, Michael Muelly, Yousef Ahmad","doi":"10.1007/s10278-023-00914-w","DOIUrl":null,"url":null,"abstract":"<p>We previously validated Fibresolve, a machine learning classifier system that non-invasively predicts idiopathic pulmonary fibrosis (IPF) diagnosis. The system incorporates an automated deep learning algorithm that analyzes chest computed tomography (CT) imaging to assess for features associated with idiopathic pulmonary fibrosis. Here, we assess performance in assessment of patterns beyond those that are characteristic features of usual interstitial pneumonia (UIP) pattern. The machine learning classifier was previously developed and validated using standard training, validation, and test sets, with clinical plus pathologically determined ground truth. The multi-site 295-patient validation dataset was used for focused subgroup analysis in this investigation to evaluate the classifier’s performance range in cases with and without radiologic UIP and probable UIP designations. Radiologic assessment of specific features for UIP including the presence and distribution of reticulation, ground glass, bronchiectasis, and honeycombing was used for assignment of radiologic pattern. Output from the classifier was assessed within various UIP subgroups. The machine learning classifier was able to classify cases not meeting the criteria for UIP or probable UIP as IPF with estimated sensitivity of 56–65% and estimated specificity of 92–94%. Example cases demonstrated non-basilar-predominant as well as ground glass patterns that were indeterminate for UIP by subjective imaging criteria but for which the classifier system was able to correctly identify the case as IPF as confirmed by multidisciplinary discussion generally inclusive of histopathology. The machine learning classifier Fibresolve may be helpful in the diagnosis of IPF in cases without radiological UIP and probable UIP patterns.</p>","PeriodicalId":50214,"journal":{"name":"Journal of Digital Imaging","volume":"7 1","pages":""},"PeriodicalIF":2.9000,"publicationDate":"2024-01-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Analysis of Validation Performance of a Machine Learning Classifier in Interstitial Lung Disease Cases Without Definite or Probable Usual Interstitial Pneumonia Pattern on CT Using Clinical and Pathology-Supported Diagnostic Labels\",\"authors\":\"Marcello Chang, Joshua J. Reicher, Angad Kalra, Michael Muelly, Yousef Ahmad\",\"doi\":\"10.1007/s10278-023-00914-w\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>We previously validated Fibresolve, a machine learning classifier system that non-invasively predicts idiopathic pulmonary fibrosis (IPF) diagnosis. The system incorporates an automated deep learning algorithm that analyzes chest computed tomography (CT) imaging to assess for features associated with idiopathic pulmonary fibrosis. Here, we assess performance in assessment of patterns beyond those that are characteristic features of usual interstitial pneumonia (UIP) pattern. The machine learning classifier was previously developed and validated using standard training, validation, and test sets, with clinical plus pathologically determined ground truth. The multi-site 295-patient validation dataset was used for focused subgroup analysis in this investigation to evaluate the classifier’s performance range in cases with and without radiologic UIP and probable UIP designations. Radiologic assessment of specific features for UIP including the presence and distribution of reticulation, ground glass, bronchiectasis, and honeycombing was used for assignment of radiologic pattern. Output from the classifier was assessed within various UIP subgroups. The machine learning classifier was able to classify cases not meeting the criteria for UIP or probable UIP as IPF with estimated sensitivity of 56–65% and estimated specificity of 92–94%. Example cases demonstrated non-basilar-predominant as well as ground glass patterns that were indeterminate for UIP by subjective imaging criteria but for which the classifier system was able to correctly identify the case as IPF as confirmed by multidisciplinary discussion generally inclusive of histopathology. The machine learning classifier Fibresolve may be helpful in the diagnosis of IPF in cases without radiological UIP and probable UIP patterns.</p>\",\"PeriodicalId\":50214,\"journal\":{\"name\":\"Journal of Digital Imaging\",\"volume\":\"7 1\",\"pages\":\"\"},\"PeriodicalIF\":2.9000,\"publicationDate\":\"2024-01-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Digital Imaging\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://doi.org/10.1007/s10278-023-00914-w\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Digital Imaging","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1007/s10278-023-00914-w","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}
Analysis of Validation Performance of a Machine Learning Classifier in Interstitial Lung Disease Cases Without Definite or Probable Usual Interstitial Pneumonia Pattern on CT Using Clinical and Pathology-Supported Diagnostic Labels
We previously validated Fibresolve, a machine learning classifier system that non-invasively predicts idiopathic pulmonary fibrosis (IPF) diagnosis. The system incorporates an automated deep learning algorithm that analyzes chest computed tomography (CT) imaging to assess for features associated with idiopathic pulmonary fibrosis. Here, we assess performance in assessment of patterns beyond those that are characteristic features of usual interstitial pneumonia (UIP) pattern. The machine learning classifier was previously developed and validated using standard training, validation, and test sets, with clinical plus pathologically determined ground truth. The multi-site 295-patient validation dataset was used for focused subgroup analysis in this investigation to evaluate the classifier’s performance range in cases with and without radiologic UIP and probable UIP designations. Radiologic assessment of specific features for UIP including the presence and distribution of reticulation, ground glass, bronchiectasis, and honeycombing was used for assignment of radiologic pattern. Output from the classifier was assessed within various UIP subgroups. The machine learning classifier was able to classify cases not meeting the criteria for UIP or probable UIP as IPF with estimated sensitivity of 56–65% and estimated specificity of 92–94%. Example cases demonstrated non-basilar-predominant as well as ground glass patterns that were indeterminate for UIP by subjective imaging criteria but for which the classifier system was able to correctly identify the case as IPF as confirmed by multidisciplinary discussion generally inclusive of histopathology. The machine learning classifier Fibresolve may be helpful in the diagnosis of IPF in cases without radiological UIP and probable UIP patterns.
期刊介绍:
The Journal of Digital Imaging (JDI) is the official peer-reviewed journal of the Society for Imaging Informatics in Medicine (SIIM). JDI’s goal is to enhance the exchange of knowledge encompassed by the general topic of Imaging Informatics in Medicine such as research and practice in clinical, engineering, and information technologies and techniques in all medical imaging environments. JDI topics are of interest to researchers, developers, educators, physicians, and imaging informatics professionals.
Suggested Topics
PACS and component systems; imaging informatics for the enterprise; image-enabled electronic medical records; RIS and HIS; digital image acquisition; image processing; image data compression; 3D, visualization, and multimedia; speech recognition; computer-aided diagnosis; facilities design; imaging vocabularies and ontologies; Transforming the Radiological Interpretation Process (TRIP™); DICOM and other standards; workflow and process modeling and simulation; quality assurance; archive integrity and security; teleradiology; digital mammography; and radiological informatics education.