Evidence quality estimation using selected machine learning approaches

Abstract

Evidence Based Medicine, is a practice, where medical actions/decisions are undertaken on the basis of best available evidence-based recommendations. In this context, we propose a system for automatic grading of evidence.Evidence grading is approached as a multi-label classification task. Here, classes represent grades, in a widely used Strength of Recommendation Taxonomy (SORT). Numerous ensemble methods are experimented with. It was found that the most successful one used Support Vector Classifiers, trained on multiple high level features, results of which are used to train a Random Forest Classifier. The best achieved accuracy score was 75.41%, which is a significant improvement over the baseline of 48%, achieved by classifying all instances as the majority class. It was also found that the most important predictor is the publication type of articles comprising the body of evidence. The designed system is tuned for use with medical publications and SORT. However, due to it’s generality, it can easily be used with other evidence grading systems.