Tianqi Hong, Shitong Xie, Xinran Liu, Jing Wu, Gang Chen
{"title":"Do Machine Learning Approaches Perform Better than Regression Models in Mapping Studies? A Systematic Review.","authors":"Tianqi Hong, Shitong Xie, Xinran Liu, Jing Wu, Gang Chen","doi":"10.1016/j.jval.2024.12.010","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>To identify how machine learning (ML) approaches were implemented in mapping studies, and to what extent ML improved performance compared with regression models (RMs).</p><p><strong>Methods: </strong>A systematic literature search was conducted in 12 databases from inception to Dec-2023 to identify studies that applied ML to develop mapping algorithms. A data template was applied to extract dataset information, source and target measures, ML approaches and RMs, mapping types (direct vs. indirect), goodness-of-fit indicators (mean absolute error [MAE], mean squared error [MSE], root mean squared error [RMSE], R-squared, and intraclass-correlation coefficient [ICC]), and validation methods. Differences in goodness-of-fit indicators between ML and RMs were summarized. Potential advantages and challenges for ML were further discussed.</p><p><strong>Results: </strong>13 mapping studies were identified, in which both ML and RM were adopted. Bayesian networks was the most frequently used ML approach (n=6), followed by the least absolute shrinkage and selection operator (n=4). Ordinary least square model was the most used RM (n=8), followed by censored least absolute deviation model and multinomial logit model (n=5 each). The average improvement of goodness-of-fit of ML compared with RMs by indicators were 0.007 (MAE), 0.004 (MSE), 0.058 (R-squared), 0.016 (ICC), and -0.0004 (RMSE).</p><p><strong>Conclusion: </strong>There is an increasing number of studies using ML in developing mapping algorithms. Generally, a minor improvement of goodness-of-fit was observed compared with RMs when using mean-based comparisons. Issues such as how to interpret, apply and externally validate the ML-based outputs would affect their implementation. Future studies are warranted to verify advantages of ML approaches.</p>","PeriodicalId":23508,"journal":{"name":"Value in Health","volume":" ","pages":""},"PeriodicalIF":4.9000,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Value in Health","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.jval.2024.12.010","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ECONOMICS","Score":null,"Total":0}
引用次数: 0
Abstract
Objectives: To identify how machine learning (ML) approaches were implemented in mapping studies, and to what extent ML improved performance compared with regression models (RMs).
Methods: A systematic literature search was conducted in 12 databases from inception to Dec-2023 to identify studies that applied ML to develop mapping algorithms. A data template was applied to extract dataset information, source and target measures, ML approaches and RMs, mapping types (direct vs. indirect), goodness-of-fit indicators (mean absolute error [MAE], mean squared error [MSE], root mean squared error [RMSE], R-squared, and intraclass-correlation coefficient [ICC]), and validation methods. Differences in goodness-of-fit indicators between ML and RMs were summarized. Potential advantages and challenges for ML were further discussed.
Results: 13 mapping studies were identified, in which both ML and RM were adopted. Bayesian networks was the most frequently used ML approach (n=6), followed by the least absolute shrinkage and selection operator (n=4). Ordinary least square model was the most used RM (n=8), followed by censored least absolute deviation model and multinomial logit model (n=5 each). The average improvement of goodness-of-fit of ML compared with RMs by indicators were 0.007 (MAE), 0.004 (MSE), 0.058 (R-squared), 0.016 (ICC), and -0.0004 (RMSE).
Conclusion: There is an increasing number of studies using ML in developing mapping algorithms. Generally, a minor improvement of goodness-of-fit was observed compared with RMs when using mean-based comparisons. Issues such as how to interpret, apply and externally validate the ML-based outputs would affect their implementation. Future studies are warranted to verify advantages of ML approaches.
期刊介绍:
Value in Health contains original research articles for pharmacoeconomics, health economics, and outcomes research (clinical, economic, and patient-reported outcomes/preference-based research), as well as conceptual and health policy articles that provide valuable information for health care decision-makers as well as the research community. As the official journal of ISPOR, Value in Health provides a forum for researchers, as well as health care decision-makers to translate outcomes research into health care decisions.