Deep learning with noisy labels: Learning True Labels as Discrete Latent Variable

Abstract

In recent years, learning from data with noisy labels (Label Noise) has emerged as a critical issue for supervised learning. This issue has become even more concerning as a result of recent concerns about Deep Learning's generalization capabilities. Indeed, deep learning necessitates a large amount of data, which is typically gathered by search engines. However, these engines frequently return data with Noisy labels. In this study, the variational inference is used to investigate Label Noise in Deep Learning. (1) Using the Label Noise concept, observable labels are learned discriminatively while true labels are learned using reparameterization variational inference. (2) The noise transition matrix is learned during training without the use of any special methods, heuristics, or initial stages. The effectiveness of our approach is shown on several test datasets, including MNIST and CIFAR32, and theoretical results show how variational inference in any discriminating neural network can be used to learn the correct label distribution.