Abstract SY01-03: The gold standard cancer diagnosis: Studies of physician variability, interpretive behavior, and the impact of AI

The current gold standard for cancer diagnoses is based on pathologists9 visual inspection of tissue sections. However, our research has found concerning levels of inter-observer and intra-observer variability among pathologists. Our prior work in melanoma shows that current diagnoses within the disease spectrum from benign nevi to melanoma in situ to invasive melanoma are neither reproducible nor accurate, yielding estimates that ~17% of all diagnoses for melanocytic lesions in the US are incorrect (Elmore et al. BMJ 2017). A study conducted by our team in breast pathology quantified the magnitude of diagnostic agreement among pathologists compared with a gold standard consensus reference: among DCIS cases, 16% of interpretations were discordant, while among atypia cases 52% of interpretations were discordant (Elmore et al. JAMA 2015). While computer systems, such as computer aided detection (CAD) tools, have been widely integrated into clinical practice to aid the interpretative and diagnostic process, our work has also found that the use of CAD can be associated with increases in potential harms, including higher recall and biopsy rates for screening mammography (Fenton et al. NEJM 2007). Given the critical need to improve the quality of our current diagnostic and prognostic capabilities, our multidisciplinary research team is conducting several studies that involve the development and integration of AI/machine learning and eye-tracking across clinical contexts. The challenges and implications associated with “gold standard” definitions for diagnoses, with data sharing infrastructure and with the eventual impact of AI on the human interface will be discussed. Citation Format: Joann Elmore. The gold standard cancer diagnosis: Studies of physician variability, interpretive behavior, and the impact of AI [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr SY01-03.