Joep MA Bogaerts, Miranda P Steenbeek, John-Melle Bokhorst, Majke HD van Bommel, Luca Abete, Francesca Addante, Mariel Brinkhuis, Alicja Chrzan, Fleur Cordier, Mojgan Devouassoux-Shisheboran, Juan Fernández-Pérez, Anna Fischer, C Blake Gilks, Angela Guerriero, Marta Jaconi, Tony G Kleijn, Loes Kooreman, Spencer Martin, Jakob Milla, Nadine Narducci, Chara Ntala, Vinita Parkash, Christophe de Pauw, Joseph T Rabban, Lucia Rijstenberg, Robert Rottscholl, Annette Staebler, Koen Van de Vijver, Gian Franco Zannoni, Monica van Zanten, AI-STIC Study Group, Joanne A de Hullu, Michiel Simons, Jeroen AWM van der Laak
{"title":"评估深度学习辅助对输卵管浆液性输卵管上皮内癌(STIC)组织病理学诊断的影响。","authors":"Joep MA Bogaerts, Miranda P Steenbeek, John-Melle Bokhorst, Majke HD van Bommel, Luca Abete, Francesca Addante, Mariel Brinkhuis, Alicja Chrzan, Fleur Cordier, Mojgan Devouassoux-Shisheboran, Juan Fernández-Pérez, Anna Fischer, C Blake Gilks, Angela Guerriero, Marta Jaconi, Tony G Kleijn, Loes Kooreman, Spencer Martin, Jakob Milla, Nadine Narducci, Chara Ntala, Vinita Parkash, Christophe de Pauw, Joseph T Rabban, Lucia Rijstenberg, Robert Rottscholl, Annette Staebler, Koen Van de Vijver, Gian Franco Zannoni, Monica van Zanten, AI-STIC Study Group, Joanne A de Hullu, Michiel Simons, Jeroen AWM van der Laak","doi":"10.1002/2056-4538.70006","DOIUrl":null,"url":null,"abstract":"<p>In recent years, it has become clear that artificial intelligence (AI) models can achieve high accuracy in specific pathology-related tasks. An example is our deep-learning model, designed to automatically detect serous tubal intraepithelial carcinoma (STIC), the precursor lesion to high-grade serous ovarian carcinoma, found in the fallopian tube. However, the standalone performance of a model is insufficient to determine its value in the diagnostic setting. To evaluate the impact of the use of this model on pathologists' performance, we set up a fully crossed multireader, multicase study, in which 26 participants, from 11 countries, reviewed 100 digitalized H&E-stained slides of fallopian tubes (30 cases/70 controls) with and without AI assistance, with a washout period between the sessions. We evaluated the effect of the deep-learning model on accuracy, slide review time and (subjectively perceived) diagnostic certainty, using mixed-models analysis. With AI assistance, we found a significant increase in accuracy (<i>p</i> < 0.01) whereby the average sensitivity increased from 82% to 93%. Further, there was a significant 44 s (32%) reduction in slide review time (<i>p</i> < 0.01). The level of certainty that the participants felt versus their own assessment also significantly increased, by 0.24 on a 10-point scale (<i>p</i> < 0.01). In conclusion, we found that, in a diverse group of pathologists and pathology residents, AI support resulted in a significant improvement in the accuracy of STIC diagnosis and was coupled with a substantial reduction in slide review time. This model has the potential to provide meaningful support to pathologists in the diagnosis of STIC, ultimately streamlining and optimizing the overall diagnostic process.</p>","PeriodicalId":48612,"journal":{"name":"Journal of Pathology Clinical Research","volume":null,"pages":null},"PeriodicalIF":3.4000,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11496567/pdf/","citationCount":"0","resultStr":"{\"title\":\"Assessing the impact of deep-learning assistance on the histopathological diagnosis of serous tubal intraepithelial carcinoma (STIC) in fallopian tubes\",\"authors\":\"Joep MA Bogaerts, Miranda P Steenbeek, John-Melle Bokhorst, Majke HD van Bommel, Luca Abete, Francesca Addante, Mariel Brinkhuis, Alicja Chrzan, Fleur Cordier, Mojgan Devouassoux-Shisheboran, Juan Fernández-Pérez, Anna Fischer, C Blake Gilks, Angela Guerriero, Marta Jaconi, Tony G Kleijn, Loes Kooreman, Spencer Martin, Jakob Milla, Nadine Narducci, Chara Ntala, Vinita Parkash, Christophe de Pauw, Joseph T Rabban, Lucia Rijstenberg, Robert Rottscholl, Annette Staebler, Koen Van de Vijver, Gian Franco Zannoni, Monica van Zanten, AI-STIC Study Group, Joanne A de Hullu, Michiel Simons, Jeroen AWM van der Laak\",\"doi\":\"10.1002/2056-4538.70006\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>In recent years, it has become clear that artificial intelligence (AI) models can achieve high accuracy in specific pathology-related tasks. An example is our deep-learning model, designed to automatically detect serous tubal intraepithelial carcinoma (STIC), the precursor lesion to high-grade serous ovarian carcinoma, found in the fallopian tube. However, the standalone performance of a model is insufficient to determine its value in the diagnostic setting. To evaluate the impact of the use of this model on pathologists' performance, we set up a fully crossed multireader, multicase study, in which 26 participants, from 11 countries, reviewed 100 digitalized H&E-stained slides of fallopian tubes (30 cases/70 controls) with and without AI assistance, with a washout period between the sessions. We evaluated the effect of the deep-learning model on accuracy, slide review time and (subjectively perceived) diagnostic certainty, using mixed-models analysis. With AI assistance, we found a significant increase in accuracy (<i>p</i> < 0.01) whereby the average sensitivity increased from 82% to 93%. Further, there was a significant 44 s (32%) reduction in slide review time (<i>p</i> < 0.01). The level of certainty that the participants felt versus their own assessment also significantly increased, by 0.24 on a 10-point scale (<i>p</i> < 0.01). In conclusion, we found that, in a diverse group of pathologists and pathology residents, AI support resulted in a significant improvement in the accuracy of STIC diagnosis and was coupled with a substantial reduction in slide review time. This model has the potential to provide meaningful support to pathologists in the diagnosis of STIC, ultimately streamlining and optimizing the overall diagnostic process.</p>\",\"PeriodicalId\":48612,\"journal\":{\"name\":\"Journal of Pathology Clinical Research\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.4000,\"publicationDate\":\"2024-10-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11496567/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Pathology Clinical Research\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/2056-4538.70006\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PATHOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Pathology Clinical Research","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/2056-4538.70006","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PATHOLOGY","Score":null,"Total":0}
Assessing the impact of deep-learning assistance on the histopathological diagnosis of serous tubal intraepithelial carcinoma (STIC) in fallopian tubes
In recent years, it has become clear that artificial intelligence (AI) models can achieve high accuracy in specific pathology-related tasks. An example is our deep-learning model, designed to automatically detect serous tubal intraepithelial carcinoma (STIC), the precursor lesion to high-grade serous ovarian carcinoma, found in the fallopian tube. However, the standalone performance of a model is insufficient to determine its value in the diagnostic setting. To evaluate the impact of the use of this model on pathologists' performance, we set up a fully crossed multireader, multicase study, in which 26 participants, from 11 countries, reviewed 100 digitalized H&E-stained slides of fallopian tubes (30 cases/70 controls) with and without AI assistance, with a washout period between the sessions. We evaluated the effect of the deep-learning model on accuracy, slide review time and (subjectively perceived) diagnostic certainty, using mixed-models analysis. With AI assistance, we found a significant increase in accuracy (p < 0.01) whereby the average sensitivity increased from 82% to 93%. Further, there was a significant 44 s (32%) reduction in slide review time (p < 0.01). The level of certainty that the participants felt versus their own assessment also significantly increased, by 0.24 on a 10-point scale (p < 0.01). In conclusion, we found that, in a diverse group of pathologists and pathology residents, AI support resulted in a significant improvement in the accuracy of STIC diagnosis and was coupled with a substantial reduction in slide review time. This model has the potential to provide meaningful support to pathologists in the diagnosis of STIC, ultimately streamlining and optimizing the overall diagnostic process.
期刊介绍:
The Journal of Pathology: Clinical Research and The Journal of Pathology serve as translational bridges between basic biomedical science and clinical medicine with particular emphasis on, but not restricted to, tissue based studies.
The focus of The Journal of Pathology: Clinical Research is the publication of studies that illuminate the clinical relevance of research in the broad area of the study of disease. Appropriately powered and validated studies with novel diagnostic, prognostic and predictive significance, and biomarker discover and validation, will be welcomed. Studies with a predominantly mechanistic basis will be more appropriate for the companion Journal of Pathology.