Methodological choices and clinical usefulness for machine learning predictions of outcome in Internet-based cognitive behavioural therapy

Abstract

While psychological treatments are effective, a substantial portion of patients do not benefit enough. Early identification of those may allow for adaptive treatment strategies and improved outcomes. We aimed to evaluate the clinical usefulness of machine-learning (ML) models predicting outcomes in Internet-based Cognitive Behavioural Therapy, to compare ML-related methodological choices, and guide future use of these. Eighty main models were compared. Baseline variables, weekly symptoms, and treatment activity were used to predict treatment outcomes in a dataset of 6695 patients from regular care. We show that the best models use handpicked predictors and impute missing data. No ML algorithm shows clear superiority. They have a mean balanced accuracy of 78.1% at treatment week four, closely matched by regression (77.8%). ML surpasses the benchmark for clinical usefulness (67%). Advanced and simple models perform equally, indicating a need for more data or smarter methodological designs to confirm advantages of ML. While there are many therapy treatments that are effective for mental health problems some patients don’t benefit enough. Predicting whom might need more help can guide therapists to adjust treatments for better results. Computer methods are increasingly used for predicting the outcome of treatment, but studies vary widely in accuracy and methodology. We examined a variety of models to test performance. Those examined were based on a several factors: what data is chosen, how the data is managed, as well as type of mathematical equations and function used for prediction. When used on ~6500 patients, none of the computer methods tested stood out as the best. Simple models were as accurate as more advanced. Accuracy of prediction of treatment outcome was good enough to inform clinicians’ decisions, suggesting they may still be useful tools in mental health care. Hentati Isacsson et al. investigate and compare several data preprocessing and machine learning approaches to predict treatment outcomes in internet-delivered cognitive behavioural therapy. Despite indications that no algorithm or method examined shows clear superiority, results still suggest promise for clinical implementations.