PCA Approaches for Optimal Convolution Kernels in Convolutional Neural Networks

Abstract

Convolutional neural networks (CNNs) have become one of most powerful machine learning models; with enough data, their accuracy in tasks such as image-related classifications and natural language processing is unmatched. The drawback that many scientists have commented on is the fact that these networks, usually trained from randomly-initialized parameters, are black-boxes. This article introduces an innovative variant for CNNs, which incorporates principal components (PCs) derived from well-trained convolution kernels. The variant is called the principal component-incorporating CNN (PC-CNN), in which the PCs are employed either as a complete replacement for randomly-initialized convolution kernels or as an initialization for the convolution kernels to be re-trained. The objective is to help training processes converge to the global minimizer. The PC-CNN is applied for the MNIST handwritten digit dataset to prove its effectiveness.