Comparison of Different Machine Learning Methodologies for Predicting the Non-Specific Treatment Response in Placebo Controlled Major Depressive Disorder Clinical Trials

Abstract

Placebo effect represents a serious confounder for the assessment of treatment effect to the extent that it has become increasingly difficult to develop antidepressant medications appropriate for outperforming placebo. Treatment effect in randomized, placebo-controlled trials, is usually estimated by the mean baseline adjusted difference of treatment response in active and placebo arms and is function of treatment-specific and non-specific effects. The non-specific treatment effect varies subject by subject conditional to the individual propensity to respond to placebo. This effect is not estimable at an individual level using the conventional parallel-group study design, since each subject enrolled in the trial is assigned to receive either active treatment or placebo, but not both. The objective of this study was to conduct a comparative analysis of the machine learning methodologies to estimate the individual probability of a non-specific treatment effect. The estimated probability is expected to support novel methodological approaches for better controlling effect of excessively high placebo response. At this purpose, six machine learning methodologies (gradient boosting machine, lasso regression, logistic regression, support vector machines, k-nearest neighbors, and random forests) were compared to the multilayer perceptrons artificial neural network (ANN) methodology for predicting the probability of individual non-specific treatment response. ANN achieved the highest overall accuracy among all methods tested. A fivefold cross-validation was used to assess performances and risks of overfitting of the ANN model. The analysis conducted without subjects with non-specific effect indicated a significant increase of signal detection with significant increase in effect size.