Wrapper Methods to Correct Mislabelled Training Data

Abstract

Machine learning has obvious applications to the diagnosis of disease, and for many neurological conditions features extracted from brain images allow classifiers based on neuroimaging biomarkers to provide a useful complement to more traditional diagnostic methods based on symptoms and psychological testing. However the labels used in the training of such systems frequently depend on standard clinical diagnostic methods, meaning they are not completely reliable in many cases. This uncertainty makes the problems this causes hard to study, as it is difficult to measure both the extent of mislabelling and its effect on results. To avoid this problem, we perform classification of gender based on imaging, as this is definitely known for each subject. We then deliberately make known proportions of the training labels incorrect. This allows us to assess the effect of the level of label noise on classification accuracy, and evaluate methods that allow for the mislabelled data. The methods are wrappers using existing well known classifier algorithms. The results indicate that the methods can be significantly effective at realistic levels of noise in the training labels, but care must be taken in choosing which method to apply depending on the level of label noise.