Harriet L. Lancaster , Beibei Jiang , Michael P.A. Davies , Jan Willem C. Gratama , Mario Silva , Jaeyoun Yi , Marjolein A. Heuvelmans , Geertruida H. de Bock , Anand Devaraj , John K. Field , Matthijs Oudkerk
{"title":"Histological proven AI performance in the UKLS CT lung cancer screening study: Potential for workload reduction","authors":"Harriet L. Lancaster , Beibei Jiang , Michael P.A. Davies , Jan Willem C. Gratama , Mario Silva , Jaeyoun Yi , Marjolein A. Heuvelmans , Geertruida H. de Bock , Anand Devaraj , John K. Field , Matthijs Oudkerk","doi":"10.1016/j.ejca.2025.115324","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><div>Artificial intelligence (AI) could reduce lung cancer screening computer tomography (CT)-reading workload if used as a first-reader, ruling-out negative CT-scans at baseline. Evidence is lacking to support AI performance when compared to gold-standard lung cancer outcomes. This study validated the performance of a commercially available AI software in the UK lung cancer screening (UKLS) trial dataset, with comparison to human reads and histological lung cancer outcomes, and estimated CT-reading workload reduction.</div></div><div><h3>Methods</h3><div>1252 UKLS-baseline-CT-scans were evaluated independently by AI and human readers. AI performance was evaluated on two-levels. Firstly, AI classification and individual reads were compared to a EU reference standard (based on NELSON2.0-European Position Statement) determined by a European expert panel blinded from individual results. A positive misclassification was defined as a nodule positive read ≥ 100mm<sup>3</sup> and no/<100mm<sup>3</sup> nodules in the expert read; A negative misclassification was defined as a nodule negative read, whereas an indeterminate or positive finding in the expert read. Secondly, AI nodule classification was compared to gold-standard histological lung cancer outcomes. CT-reading workload reduction was calculated from AI negative CT-scans when AI was used as first-reader.</div></div><div><h3>Results</h3><div>Expert panel reference standard reported 815 (65 %) negative and 437 (35 %) indeterminate/positive CT-scans in the dataset of 1252 UKLS-participants. Compared to the reference standard, AI resulted in less misclassification than human reads, NPV 92·0 %(90·2 %-95·3 %). On comparison to gold-standard, AI detected all 31 baseline-round lung cancers, but classified one as negative due to the 100mm<sup>3</sup> threshold, NPV 99·8 %(99·0 %-99·9 %). Estimated maximum CT-reading workload reduction was 79 %.</div></div><div><h3>Conclusion</h3><div>Implementing AI as first-reader to rule-out negative CT-scans, shows considerable potential to reduce CT-reading workload and does not lead to missed lung cancers.</div></div>","PeriodicalId":11980,"journal":{"name":"European Journal of Cancer","volume":"220 ","pages":"Article 115324"},"PeriodicalIF":7.6000,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Cancer","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0959804925001054","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose
Artificial intelligence (AI) could reduce lung cancer screening computer tomography (CT)-reading workload if used as a first-reader, ruling-out negative CT-scans at baseline. Evidence is lacking to support AI performance when compared to gold-standard lung cancer outcomes. This study validated the performance of a commercially available AI software in the UK lung cancer screening (UKLS) trial dataset, with comparison to human reads and histological lung cancer outcomes, and estimated CT-reading workload reduction.
Methods
1252 UKLS-baseline-CT-scans were evaluated independently by AI and human readers. AI performance was evaluated on two-levels. Firstly, AI classification and individual reads were compared to a EU reference standard (based on NELSON2.0-European Position Statement) determined by a European expert panel blinded from individual results. A positive misclassification was defined as a nodule positive read ≥ 100mm3 and no/<100mm3 nodules in the expert read; A negative misclassification was defined as a nodule negative read, whereas an indeterminate or positive finding in the expert read. Secondly, AI nodule classification was compared to gold-standard histological lung cancer outcomes. CT-reading workload reduction was calculated from AI negative CT-scans when AI was used as first-reader.
Results
Expert panel reference standard reported 815 (65 %) negative and 437 (35 %) indeterminate/positive CT-scans in the dataset of 1252 UKLS-participants. Compared to the reference standard, AI resulted in less misclassification than human reads, NPV 92·0 %(90·2 %-95·3 %). On comparison to gold-standard, AI detected all 31 baseline-round lung cancers, but classified one as negative due to the 100mm3 threshold, NPV 99·8 %(99·0 %-99·9 %). Estimated maximum CT-reading workload reduction was 79 %.
Conclusion
Implementing AI as first-reader to rule-out negative CT-scans, shows considerable potential to reduce CT-reading workload and does not lead to missed lung cancers.
期刊介绍:
The European Journal of Cancer (EJC) serves as a comprehensive platform integrating preclinical, digital, translational, and clinical research across the spectrum of cancer. From epidemiology, carcinogenesis, and biology to groundbreaking innovations in cancer treatment and patient care, the journal covers a wide array of topics. We publish original research, reviews, previews, editorial comments, and correspondence, fostering dialogue and advancement in the fight against cancer. Join us in our mission to drive progress and improve outcomes in cancer research and patient care.