Stability-Based Multivariate Mapping Using SCoRS

Recently we proposed a feature selection method based on stability theory (SCoRS - Survival Count on Random Subspaces) and showed that the proposed approach was able to improve classification accuracy using different datasets. In the present work we propose: (i) an extension of SCoRS using reproducibility instead of model accuracy as the parameter optimization criterion and (ii) a procedure to estimate the rate of false positive selection associated with the set of features obtained. Our results using the proposed framework showed that, as expected, the optimal parameter was more stable across the cross-validation folds, the spatial map displaying the features selected was less noisy and there was no decrease in classification accuracy. In addition, our results suggest that the estimated false positive rate for the features selected by SCoRS is under 0.05 for both optimization approaches, nevertheless lower when optimizing reproducibility in comparison with the standard optimization approach.