Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement

IF 4.8 2区 医学 Q1 Dentistry Progress in Orthodontics Pub Date : 2024-09-16 DOI:10.1186/s40510-024-00535-1
Potjanee Kanchanapiboon, Pitipat Tunksook, Prinya Tunksook, Panrasee Ritthipravat, Supatchai Boonpratham, Yodhathai Satravaha, Chaiyapol Chaweewannakorn, Supakit Peanchitlertkajorn
{"title":"Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement","authors":"Potjanee Kanchanapiboon, Pitipat Tunksook, Prinya Tunksook, Panrasee Ritthipravat, Supatchai Boonpratham, Yodhathai Satravaha, Chaiyapol Chaweewannakorn, Supakit Peanchitlertkajorn","doi":"10.1186/s40510-024-00535-1","DOIUrl":null,"url":null,"abstract":"This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets. Three clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities. Features that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters. ML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality.","PeriodicalId":56071,"journal":{"name":"Progress in Orthodontics","volume":null,"pages":null},"PeriodicalIF":4.8000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Progress in Orthodontics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s40510-024-00535-1","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Dentistry","Score":null,"Total":0}
引用次数: 0

Abstract

This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets. Three clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities. Features that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters. ML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
利用机器学习模型对颈椎成熟阶段进行分类:充分利用观察者之间和观察者内部高度一致的数据集
本研究旨在评估采用特征选择技术的机器学习(ML)模型在对颈椎成熟阶段(CVMS)进行分类时的准确性。研究使用基于共识的数据集进行模型训练,并评估模型在未见数据集上的泛化能力。三位临床医生对 1380 张侧位头颅影像进行了独立的 CVMS 评级,最终创建了五个数据集:两个基于共识的数据集(完全一致和多数表决),以及三个基于单个评定者评估的数据集。此外,第二至第四颈椎的地标标注和患者信息也经过了特征选择过程。这些数据集被用来训练各种 ML 模型,并为每个数据集找出性能最好的模型。随后对这些模型的泛化能力进行了测试。在基于共识的数据集中被认为重要的特征与 CVMS 准则一致。支持向量机模型在完全一致数据集上的准确率最高(77.4%),其次是多层感知器模型在多数票数据集上的准确率(69.6%)。单个评分模型的准确率较低(60.4%-67.9%)。基于共识的训练模型还表现出较低的变异系数(CV),这表明与来自单个评分者的模型相比,基于共识的训练模型具有更强的泛化能力。在基于共识的数据集上训练的用于 CVMS 分类的 ML 模型表现出最高的准确性,其重要特征与最初的 CVMS 指南一致。这些模型还显示出强大的泛化能力,强调了数据集质量的重要性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Progress in Orthodontics
Progress in Orthodontics Dentistry-Orthodontics
CiteScore
7.30
自引率
4.20%
发文量
45
审稿时长
13 weeks
期刊介绍: Progress in Orthodontics is a fully open access, international journal owned by the Italian Society of Orthodontics and published under the brand SpringerOpen. The Society is currently covering all publication costs so there are no article processing charges for authors. It is a premier journal of international scope that fosters orthodontic research, including both basic research and development of innovative clinical techniques, with an emphasis on the following areas: • Mechanisms to improve orthodontics • Clinical studies and control animal studies • Orthodontics and genetics, genomics • Temporomandibular joint (TMJ) control clinical trials • Efficacy of orthodontic appliances and animal models • Systematic reviews and meta analyses • Mechanisms to speed orthodontic treatment Progress in Orthodontics will consider for publication only meritorious and original contributions. These may be: • Original articles reporting the findings of clinical trials, clinically relevant basic scientific investigations, or novel therapeutic or diagnostic systems • Review articles on current topics • Articles on novel techniques and clinical tools • Articles of contemporary interest
期刊最新文献
The role of orthodontists in the multidisciplinary management of obstructive sleep apnea. Polymerization kinetics of 3D-printed orthodontic aligners under different UV post-curing conditions. Comparison of AI-assisted cephalometric analysis and orthodontist-performed digital tracing analysis. Prevalence of psychosocial findings and their correlation with TMD symptoms in an adult population sample. Breaking a dogma: orthodontic tooth movement alters systemic immunity.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1