Wei Song, Wen Shang, Chunying Li, Xinyu Bian, Hong Lu, Jun Ma, Dahai Yu
{"title":"利用判别特征和类平衡技术提高深度学习模型在预测和分类伽马通过率方面的性能:一项回顾性队列研究。","authors":"Wei Song, Wen Shang, Chunying Li, Xinyu Bian, Hong Lu, Jun Ma, Dahai Yu","doi":"10.1186/s13014-024-02496-5","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The purpose of this study was to improve the deep learning (DL) model performance in predicting and classifying IMRT gamma passing rate (GPR) by using input features related to machine parameters and a class balancing technique.</p><p><strong>Methods: </strong>A total of 2348 fields from 204 IMRT plans for patients with nasopharyngeal carcinoma were retrospectively collected to form a dataset. Input feature maps, including fluence, leaf gap, leaf speed of both banks, and corresponding errors, were constructed from the dynamic log files. The SHAP framework was employed to compute the impact of each feature on the model output for recursive feature elimination. A series of UNet++ based models were trained on the obtained eight feature sets with three fine-tuning methods including the standard mean squared error (MSE) loss, a re-sampling technique, and a proposed weighted MSE loss (WMSE). Differences in mean absolute error, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were compared between the different models.</p><p><strong>Results: </strong>The models trained with feature sets including leaf speed and leaf gap features predicted GPR for failed fields more accurately than the other models (F(7, 147) = 5.378, p < 0.001). The WMSE loss had the highest accuracy in predicting GPR for failed fields among the three fine-tuning methods (F(2, 42) = 14.149, p < 0.001), while an opposite trend was observed in predicting GPR for passed fields (F(2, 730) = 9.907, p < 0.001). The WMSE_FS5 model achieved a superior AUC (0.92) and more balanced sensitivity (0.77) and specificity (0.89) compared to the other models.</p><p><strong>Conclusions: </strong>Machine parameters can provide discriminative input features for GPR prediction in DL. The novel weighted loss function demonstrates the ability to balance the prediction and classification accuracy between the passed and failed fields. The proposed approach is able to improve the DL model performance in predicting and classifying GPR, and can potentially be integrated into the plan optimization process to generate higher deliverability plans.</p><p><strong>Trial registration: </strong>This clinical trial was registered in the Chinese Clinical Trial Registry on March 26th, 2020 (registration number: ChiCTR2000031276). https://clinicaltrials.gov/ct2/show/ChiCTR2000031276.</p>","PeriodicalId":49639,"journal":{"name":"Radiation Oncology","volume":"19 1","pages":"98"},"PeriodicalIF":3.3000,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11293183/pdf/","citationCount":"0","resultStr":"{\"title\":\"Improving the performance of deep learning models in predicting and classifying gamma passing rates with discriminative features and a class balancing technique: a retrospective cohort study.\",\"authors\":\"Wei Song, Wen Shang, Chunying Li, Xinyu Bian, Hong Lu, Jun Ma, Dahai Yu\",\"doi\":\"10.1186/s13014-024-02496-5\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>The purpose of this study was to improve the deep learning (DL) model performance in predicting and classifying IMRT gamma passing rate (GPR) by using input features related to machine parameters and a class balancing technique.</p><p><strong>Methods: </strong>A total of 2348 fields from 204 IMRT plans for patients with nasopharyngeal carcinoma were retrospectively collected to form a dataset. Input feature maps, including fluence, leaf gap, leaf speed of both banks, and corresponding errors, were constructed from the dynamic log files. The SHAP framework was employed to compute the impact of each feature on the model output for recursive feature elimination. A series of UNet++ based models were trained on the obtained eight feature sets with three fine-tuning methods including the standard mean squared error (MSE) loss, a re-sampling technique, and a proposed weighted MSE loss (WMSE). Differences in mean absolute error, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were compared between the different models.</p><p><strong>Results: </strong>The models trained with feature sets including leaf speed and leaf gap features predicted GPR for failed fields more accurately than the other models (F(7, 147) = 5.378, p < 0.001). The WMSE loss had the highest accuracy in predicting GPR for failed fields among the three fine-tuning methods (F(2, 42) = 14.149, p < 0.001), while an opposite trend was observed in predicting GPR for passed fields (F(2, 730) = 9.907, p < 0.001). The WMSE_FS5 model achieved a superior AUC (0.92) and more balanced sensitivity (0.77) and specificity (0.89) compared to the other models.</p><p><strong>Conclusions: </strong>Machine parameters can provide discriminative input features for GPR prediction in DL. The novel weighted loss function demonstrates the ability to balance the prediction and classification accuracy between the passed and failed fields. The proposed approach is able to improve the DL model performance in predicting and classifying GPR, and can potentially be integrated into the plan optimization process to generate higher deliverability plans.</p><p><strong>Trial registration: </strong>This clinical trial was registered in the Chinese Clinical Trial Registry on March 26th, 2020 (registration number: ChiCTR2000031276). https://clinicaltrials.gov/ct2/show/ChiCTR2000031276.</p>\",\"PeriodicalId\":49639,\"journal\":{\"name\":\"Radiation Oncology\",\"volume\":\"19 1\",\"pages\":\"98\"},\"PeriodicalIF\":3.3000,\"publicationDate\":\"2024-07-31\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11293183/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Radiation Oncology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1186/s13014-024-02496-5\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ONCOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Radiation Oncology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s13014-024-02496-5","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ONCOLOGY","Score":null,"Total":0}
Improving the performance of deep learning models in predicting and classifying gamma passing rates with discriminative features and a class balancing technique: a retrospective cohort study.
Background: The purpose of this study was to improve the deep learning (DL) model performance in predicting and classifying IMRT gamma passing rate (GPR) by using input features related to machine parameters and a class balancing technique.
Methods: A total of 2348 fields from 204 IMRT plans for patients with nasopharyngeal carcinoma were retrospectively collected to form a dataset. Input feature maps, including fluence, leaf gap, leaf speed of both banks, and corresponding errors, were constructed from the dynamic log files. The SHAP framework was employed to compute the impact of each feature on the model output for recursive feature elimination. A series of UNet++ based models were trained on the obtained eight feature sets with three fine-tuning methods including the standard mean squared error (MSE) loss, a re-sampling technique, and a proposed weighted MSE loss (WMSE). Differences in mean absolute error, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were compared between the different models.
Results: The models trained with feature sets including leaf speed and leaf gap features predicted GPR for failed fields more accurately than the other models (F(7, 147) = 5.378, p < 0.001). The WMSE loss had the highest accuracy in predicting GPR for failed fields among the three fine-tuning methods (F(2, 42) = 14.149, p < 0.001), while an opposite trend was observed in predicting GPR for passed fields (F(2, 730) = 9.907, p < 0.001). The WMSE_FS5 model achieved a superior AUC (0.92) and more balanced sensitivity (0.77) and specificity (0.89) compared to the other models.
Conclusions: Machine parameters can provide discriminative input features for GPR prediction in DL. The novel weighted loss function demonstrates the ability to balance the prediction and classification accuracy between the passed and failed fields. The proposed approach is able to improve the DL model performance in predicting and classifying GPR, and can potentially be integrated into the plan optimization process to generate higher deliverability plans.
Trial registration: This clinical trial was registered in the Chinese Clinical Trial Registry on March 26th, 2020 (registration number: ChiCTR2000031276). https://clinicaltrials.gov/ct2/show/ChiCTR2000031276.
Radiation OncologyONCOLOGY-RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING
CiteScore
6.50
自引率
2.80%
发文量
181
审稿时长
3-6 weeks
期刊介绍:
Radiation Oncology encompasses all aspects of research that impacts on the treatment of cancer using radiation. It publishes findings in molecular and cellular radiation biology, radiation physics, radiation technology, and clinical oncology.