Supervised kernel-based multi-modal Bhattacharya distance learning for imbalanced data classification

Abstract

Learned distance metrics measure the difference of the data according to the intrinsic properties of the data points and classes. Distance metric learning approaches are typically used to linearly distinguish the samples of different classes and do not perform well on real-world nonlinear data classes. A kernel-based nonlinear distance metric learning approach is proposed in this article which exploits the density of multimodal classes to properly differentiate the classes while reducing the within-class separation. Here, multimodality refers to the disjoint distribution of a class, resulting in each class having multiple density components. In the proposed kernel density-based distance metric learning approach, kernel trick is applied on the original data and maps the data to a higher-dimensional space. Then, given the possibility of multimodal classes, a mixture of multivariate Gaussian densities is considered for the distribution of each class. The number of components is calculated using a density-based clustering approach, and then the parameters of the Gaussian components are estimated using maximum a posteriori density estimation. Then, an iterative method is used to maximize the Bhattacharya distance among the classes' Gaussian mixtures. The distance among the external components is increased, while the distance among samples of each component is decreased to provide a wide between-class margin. The results of the experiments show that using the proposed approach significantly improves the efficiency of the simple K nearest neighbor algorithm on the imbalanced data set, but when the imbalance ratio is very high, the kernel function does not have a significant effect on the efficiency of the distance metric.