Action classification in polarimetric infrared imagery via diffusion maps

Abstract

This work explores the application of a nonlinear dimensionality reduction technique known as diffusion maps for performing action classification in polarimetric infrared video sequences. The diffusion maps algorithm has been used successfully in a variety of applications involving the extraction of low-dimensional embeddings from high-dimensional data. Our dataset is composed of eight subjects each performing three basic actions: walking, walking while carrying an object in one hand, and running. The actions were captured with a polarized microgrid sensor operating in the longwave portion of the electromagnetic (EM) spectrum with a temporal resolution of 24 Hz, yielding the Stokes traditional intensity (S0) and linearly polarized (S1, S2) components of data. Our work includes the use of diffusion maps as an unsupervised dimensionality reduction step prior to action classification with three conventional classifiers: the linear perceptron algorithm, the k nearest neighbors (KNN) algorithm, and the kernel-based support vector machine (SVM). We present classification results using both the low-dimensional principal components via PCA and the low-dimensional diffusion map embedding coordinates of the data for each class. Results indicate that the diffusion map lower-dimensional embeddings provide a salient feature space for action classification, yielding an increase of overall classification accuracy by ~40% compared to PCA. Additionally, we examine the utility that the polarimetric sensor may provide by concurrently performing these analyses in the polarimetric feature spaces.