A robust canonical correlation neural network

Abstract

We review a neural implementation of canonical correlation analysis and show, using ideas suggested by ridge regression, how to make the algorithm robust. The network is shown to operate on data sets which exhibit multicollinearity. We develop a second model which not only performs as well on multicollinear data but also on general data sets. This model allows us to vary a single parameter so that the network is capable of performing partial least squares regression (at one extreme) to canonical correlation analysis (at the other) and every intermediate operation between the two. On multicollinear data, the parameter setting is shown to be important but on more general data no particular parameter setting is required. Finally, the algorithm acts on such data as a smoother in that the resulting weight vectors are much smoother and more interpretable than the weights without the robustification term.