Methodology and development of a machine learning probability calculator: Data heterogeneity limits ability to predict recurrence after arthroscopic Bankart repair

IF 5.5 2区医学 Q1 ORTHOPEDICS Knee Surgery, Sports Traumatology, Arthroscopy Pub Date : 2024-09-26 DOI:10.1002/ksa.12443

Sanne H. van Spanning, Lukas P. E. Verweij, Laurent A. M. Hendrickx, Laurens J. H. Allaart, George S. Athwal, Thibault Lafosse, Laurent Lafosse, Job N. Doornberg, Jacobien H. F. Oosterhoff, Michel P. J. van den Bekerom, Geert Alexander Buijze, the Machine Learning Consortium

{"title":"Methodology and development of a machine learning probability calculator: Data heterogeneity limits ability to predict recurrence after arthroscopic Bankart repair","authors":"Sanne H. van Spanning, Lukas P. E. Verweij, Laurent A. M. Hendrickx, Laurens J. H. Allaart, George S. Athwal, Thibault Lafosse, Laurent Lafosse, Job N. Doornberg, Jacobien H. F. Oosterhoff, Michel P. J. van den Bekerom, Geert Alexander Buijze, the Machine Learning Consortium","doi":"10.1002/ksa.12443","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Purpose</h3>\n \n <p>The aim of this study was to develop and train a machine learning (ML) algorithm to create a clinical decision support tool (i.e., ML-driven probability calculator) to be used in clinical practice to estimate recurrence rates following an arthroscopic Bankart repair (ABR).</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>Data from 14 previously published studies were collected. Inclusion criteria were (1) patients treated with ABR without remplissage for traumatic anterior shoulder instability and (2) a minimum of 2 years follow-up. Risk factors associated with recurrence were identified using bivariate logistic regression analysis. Subsequently, four ML algorithms were developed and internally validated. The predictive performance was assessed using discrimination, calibration and the Brier score.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>In total, 5591 patients underwent ABR with a recurrence rate of 15.4% (<i>n</i> = 862). Age <35 years, participation in contact and collision sports, bony Bankart lesions and full-thickness rotator cuff tears increased the risk of recurrence (all <i>p</i> < 0.05). A single shoulder dislocation (compared to multiple dislocations) lowered the risk of recurrence (<i>p</i> < 0.05). Due to the unavailability of certain variables in some patients, a portion of the patient data had to be excluded before pooling the data set to create the algorithm. A total of 797 patients were included providing information on risk factors associated with recurrence. The discrimination (area under the receiver operating curve) ranged between 0.54 and 0.57 for prediction of recurrence.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>ML was not able to predict the recurrence following ABR with the current available predictors. Despite a global coordinated effort, the heterogeneity of clinical data limited the predictive capabilities of the algorithm, emphasizing the need for standardized data collection methods in future studies.</p>\n </section>\n \n <section>\n \n <h3> Level of Evidence</h3>\n \n <p>Level IV, retrospective cohort study.</p>\n </section>\n </div>","PeriodicalId":17880,"journal":{"name":"Knee Surgery, Sports Traumatology, Arthroscopy","volume":"33 4","pages":"1488-1499"},"PeriodicalIF":5.5000,"publicationDate":"2024-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ksa.12443","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Knee Surgery, Sports Traumatology, Arthroscopy","FirstCategoryId":"3","ListUrlMain":"https://esskajournals.onlinelibrary.wiley.com/doi/10.1002/ksa.12443","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ORTHOPEDICS","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose

The aim of this study was to develop and train a machine learning (ML) algorithm to create a clinical decision support tool (i.e., ML-driven probability calculator) to be used in clinical practice to estimate recurrence rates following an arthroscopic Bankart repair (ABR).

Methods

Data from 14 previously published studies were collected. Inclusion criteria were (1) patients treated with ABR without remplissage for traumatic anterior shoulder instability and (2) a minimum of 2 years follow-up. Risk factors associated with recurrence were identified using bivariate logistic regression analysis. Subsequently, four ML algorithms were developed and internally validated. The predictive performance was assessed using discrimination, calibration and the Brier score.

Results

In total, 5591 patients underwent ABR with a recurrence rate of 15.4% (n = 862). Age <35 years, participation in contact and collision sports, bony Bankart lesions and full-thickness rotator cuff tears increased the risk of recurrence (all p < 0.05). A single shoulder dislocation (compared to multiple dislocations) lowered the risk of recurrence (p < 0.05). Due to the unavailability of certain variables in some patients, a portion of the patient data had to be excluded before pooling the data set to create the algorithm. A total of 797 patients were included providing information on risk factors associated with recurrence. The discrimination (area under the receiver operating curve) ranged between 0.54 and 0.57 for prediction of recurrence.

Conclusion

ML was not able to predict the recurrence following ABR with the current available predictors. Despite a global coordinated effort, the heterogeneity of clinical data limited the predictive capabilities of the algorithm, emphasizing the need for standardized data collection methods in future studies.

Level of Evidence

Level IV, retrospective cohort study.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

机器学习概率计算器的方法与开发：数据异质性限制了预测关节镜下 Bankart 修复术后复发的能力。

目的：本研究旨在开发和训练一种机器学习（ML）算法，以创建一种临床决策支持工具（即ML驱动的概率计算器），用于临床实践，估计关节镜下Bankart修复术（ABR）后的复发率：方法：收集了 14 项以前发表的研究数据。纳入标准为：(1) 因外伤性肩关节前方不稳而接受 ABR 治疗但未行再植术的患者；(2) 至少随访 2 年。通过双变量逻辑回归分析确定了与复发相关的风险因素。随后，开发了四种 ML 算法并进行了内部验证。结果：共有 5591 名患者接受了 ABR，复发率为 15.4%（n = 862）。年龄结论：使用现有的预测指标，ML 无法预测 ABR 后的复发率。尽管在全球范围内进行了协调努力，但临床数据的异质性限制了该算法的预测能力，这强调了在未来研究中采用标准化数据收集方法的必要性：证据级别：IV级，回顾性队列研究。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Knee Surgery, Sports Traumatology, Arthroscopy 医学-外科

CiteScore

8.10

自引率

18.40%

发文量

418

审稿时长

2 months

期刊介绍： Few other areas of orthopedic surgery and traumatology have undergone such a dramatic evolution in the last 10 years as knee surgery, arthroscopy and sports traumatology. Ranked among the top 33% of journals in both Orthopedics and Sports Sciences, the goal of this European journal is to publish papers about innovative knee surgery, sports trauma surgery and arthroscopy. Each issue features a series of peer-reviewed articles that deal with diagnosis and management and with basic research. Each issue also contains at least one review article about an important clinical problem. Case presentations or short notes about technical innovations are also accepted for publication. The articles cover all aspects of knee surgery and all types of sports trauma; in addition, epidemiology, diagnosis, treatment and prevention, and all types of arthroscopy (not only the knee but also the shoulder, elbow, wrist, hip, ankle, etc.) are addressed. Articles on new diagnostic techniques such as MRI and ultrasound and high-quality articles about the biomechanics of joints, muscles and tendons are included. Although this is largely a clinical journal, it is also open to basic research with clinical relevance. Because the journal is supported by a distinguished European Editorial Board, assisted by an international Advisory Board, you can be assured that the journal maintains the highest standards. Official Clinical Journal of the European Society of Sports Traumatology, Knee Surgery and Arthroscopy (ESSKA).