A novel deep learning by combining discriminative model with generative model

Abstract

Deep learning methods allow a classifier to learn features automatically through multiple layers of training. In a deep learning process, low-level features are abstracted into high-level features. In this paper, we propose a new probabilistic deep learning method that combines a discriminative model, namely, Support Vector Machine (SVM), with a generative model, namely, Gaussian Mixture Model (GMM). Combining the SVM with the GMM, we can represent a new input feature for deeper layer training of uncertain data in current layer construction. Bayesian rule is used to re-represent the output data of the previous layer of the SVM with GMM to serve as the input data for the next deep layer. As a result, deep features are reliably extracted without additional feature extraction efforts, using multiple layers of the SVM with GMM. Experimental results show that the proposed deep structure model allows for an easier classification of the uncertain data through multiple-layer training and it gives more accurate results.