Machine learning-based models for prediction of innovative medicine reimbursement decisions in Scotland.

Abstract

Objective: This study aimed to investigate the critical factors for reimbursement decisions of innovative medicines in Scotland and to explore the feasibility of machine learning models for predicting decisions.

Method: All appraisals for innovative medicines issued by the Scottish Medicines Consortium (SMC) from 2016 to 2020 were screened to extract decision outcomes and 24 explanatory factors. SelectKBest with chi-square test was used for factor selection. The factors with P-value <0.05 were considered to have statistically significant associations with decision outcomes and were selected. Six machine learning models including decision tree, random forest, support-vector machine, Xgboost and K-nearest neighbours and logistic regression were used to build models with selected factors. Indicators comprising accuracy, precision, recall, F1-score were used to evaluate the performance of models.

Result: A total of 111 appraisals were identified, among which, 47 medicines were recommended, 48 recommended with restricted use and 16 not recommended. Seven were identified to be significant and selected for the prediction models. The factors of request for restriction on indication by manufacture, uncertainty of economic evidence, validation of primary outcomes and acceptance of comparator were identified as the most important predictors for SMC decisions. Four models had good prediction performance with both accuracy and F1-score over 0.9 in the internal validation, and random forest had the best prediction performance.

Conclusion: Low uncertainty of economic evidence, validated primary outcomes and accepted comparators were significantly associated with positive SMC decisions. Machine learning models may be feasible for predicting reimbursement decisions in the future.