{"title":"Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement","authors":"Potjanee Kanchanapiboon, Pitipat Tunksook, Prinya Tunksook, Panrasee Ritthipravat, Supatchai Boonpratham, Yodhathai Satravaha, Chaiyapol Chaweewannakorn, Supakit Peanchitlertkajorn","doi":"10.1186/s40510-024-00535-1","DOIUrl":null,"url":null,"abstract":"This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets. Three clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities. Features that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters. ML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality.","PeriodicalId":56071,"journal":{"name":"Progress in Orthodontics","volume":"57 1","pages":""},"PeriodicalIF":4.8000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Progress in Orthodontics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s40510-024-00535-1","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Dentistry","Score":null,"Total":0}
引用次数: 0
Abstract
This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets. Three clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities. Features that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters. ML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality.
期刊介绍:
Progress in Orthodontics is a fully open access, international journal owned by the Italian Society of Orthodontics and published under the brand SpringerOpen. The Society is currently covering all publication costs so there are no article processing charges for authors.
It is a premier journal of international scope that fosters orthodontic research, including both basic research and development of innovative clinical techniques, with an emphasis on the following areas:
• Mechanisms to improve orthodontics
• Clinical studies and control animal studies
• Orthodontics and genetics, genomics
• Temporomandibular joint (TMJ) control clinical trials
• Efficacy of orthodontic appliances and animal models
• Systematic reviews and meta analyses
• Mechanisms to speed orthodontic treatment
Progress in Orthodontics will consider for publication only meritorious and original contributions. These may be:
• Original articles reporting the findings of clinical trials, clinically relevant basic scientific investigations, or novel therapeutic or diagnostic systems
• Review articles on current topics
• Articles on novel techniques and clinical tools
• Articles of contemporary interest