Ashkan Labaf, Linda Åhman-Persson, Leo Silvén Husu, J Gustav Smith, Annika Ingvarsson, Anna Werther Evaldsson
{"title":"Performance of a point-of-care ultrasound platform for artificial intelligence-enabled assessment of pulmonary B-lines.","authors":"Ashkan Labaf, Linda Åhman-Persson, Leo Silvén Husu, J Gustav Smith, Annika Ingvarsson, Anna Werther Evaldsson","doi":"10.1186/s12947-025-00338-2","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The incorporation of artificial intelligence (AI) into point-of-care ultrasound (POCUS) platforms has rapidly increased. The number of B-lines present on lung ultrasound (LUS) serve as a useful tool for the assessment of pulmonary congestion. Interpretation, however, requires experience and therefore AI automation has been pursued. This study aimed to test the agreement between the AI software embedded in a major vendor POCUS system and visual expert assessment.</p><p><strong>Methods: </strong>This single-center prospective study included 55 patients hospitalized for various respiratory symptoms, predominantly acutely decompensated heart failure. A 12-zone protocol was used. Two experts in LUS independently categorized B-lines into 0, 1-2, 3-4, and ≥ 5. The intraclass correlation coefficient (ICC) was used to determine agreement.</p><p><strong>Results: </strong>A total of 672 LUS zones were obtained, with 584 (87%) eligible for analysis. Compared with expert reviewers, the AI significantly overcounted number of B-lines per patient (23.5 vs. 2.8, p < 0.001). A greater proportion of zones with > 5 B-lines was found by the AI than by the reviewers (38% vs. 4%, p < 0.001). The ICC between the AI and reviewers was 0.28 for the total sum of B-lines and 0.37 for the zone-by-zone method. The interreviewer agreement was excellent, with ICCs of 0.92 and 0.91, respectively.</p><p><strong>Conclusion: </strong>This study demonstrated excellent interrater reliability of B-line counts from experts but poor agreement with the AI software embedded in a major vendor system, primarily due to overcounting. Our findings indicate that further development is needed to increase the accuracy of AI tools in LUS.</p>","PeriodicalId":9613,"journal":{"name":"Cardiovascular Ultrasound","volume":"23 1","pages":"3"},"PeriodicalIF":1.6000,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11874383/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cardiovascular Ultrasound","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12947-025-00338-2","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"CARDIAC & CARDIOVASCULAR SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Background: The incorporation of artificial intelligence (AI) into point-of-care ultrasound (POCUS) platforms has rapidly increased. The number of B-lines present on lung ultrasound (LUS) serve as a useful tool for the assessment of pulmonary congestion. Interpretation, however, requires experience and therefore AI automation has been pursued. This study aimed to test the agreement between the AI software embedded in a major vendor POCUS system and visual expert assessment.
Methods: This single-center prospective study included 55 patients hospitalized for various respiratory symptoms, predominantly acutely decompensated heart failure. A 12-zone protocol was used. Two experts in LUS independently categorized B-lines into 0, 1-2, 3-4, and ≥ 5. The intraclass correlation coefficient (ICC) was used to determine agreement.
Results: A total of 672 LUS zones were obtained, with 584 (87%) eligible for analysis. Compared with expert reviewers, the AI significantly overcounted number of B-lines per patient (23.5 vs. 2.8, p < 0.001). A greater proportion of zones with > 5 B-lines was found by the AI than by the reviewers (38% vs. 4%, p < 0.001). The ICC between the AI and reviewers was 0.28 for the total sum of B-lines and 0.37 for the zone-by-zone method. The interreviewer agreement was excellent, with ICCs of 0.92 and 0.91, respectively.
Conclusion: This study demonstrated excellent interrater reliability of B-line counts from experts but poor agreement with the AI software embedded in a major vendor system, primarily due to overcounting. Our findings indicate that further development is needed to increase the accuracy of AI tools in LUS.
背景：人工智能（AI）与点护理超声（POCUS）平台的结合迅速增加。肺超声（LUS）上出现的b线数量是评估肺充血的有用工具。然而，解释需要经验，因此人工智能自动化一直在追求。本研究旨在测试嵌入在主要供应商POCUS系统中的人工智能软件与视觉专家评估之间的一致性。方法：这项单中心前瞻性研究纳入了55例因各种呼吸道症状住院的患者，主要是急性失代偿性心力衰竭。使用了12个区域的协议。LUS的两位专家独立将b -line分为0、1-2、3-4和≥5。用类内相关系数（ICC）来确定一致性。结果：共获得672个LUS区，其中584个（87%）符合分析条件。与专家审稿人相比，人工智能明显高估了每位患者的b线数量（23.5 vs 2.8），人工智能发现的b线数量比审稿人发现的b线数量多（38% vs. 4%, p）。结论：本研究表明，专家提供的b线数量具有出色的相互可靠性，但与主要供应商系统中嵌入的人工智能软件的一致性较差，主要原因是高估。我们的研究结果表明，需要进一步开发以提高LUS中人工智能工具的准确性。
期刊介绍：
Cardiovascular Ultrasound is an online journal, publishing peer-reviewed: original research; authoritative reviews; case reports on challenging and/or unusual diagnostic aspects; and expert opinions on new techniques and technologies. We are particularly interested in articles that include relevant images or video files, which provide an additional dimension to published articles and enhance understanding.
As an open access journal, Cardiovascular Ultrasound ensures high visibility for authors in addition to providing an up-to-date and freely available resource for the community. The journal welcomes discussion, and provides a forum for publishing opinion and debate ranging from biology to engineering to clinical echocardiography, with both speed and versatility.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
