PRINCIPAL COMPONENTS AND THE PROBLEM OF MULTICOLLINEARITY

Abstract

Multicollinearity among independent variables within a regress ion model is one of the most frequently encountered problems faced by the applied researcher. In a recent article in this Journal (Willis, e1 a/.) , a catalog of"remedies" for multicollinearity was presented to assist in reducing its level and associated adverse consequences . One of these remediesprincipal componentswas suggested as a method oftransforming a set of collinear explanatory variables into new variables that are orthogonal to each other with the first few of these transformed va riables accounting for the majority of the variability in the origina l data set. In principal components regression , a transformed variable is determined to be important a nd included or unimportant and excluded in the regression model depending upon the size of the characteristic root (eigenvalue) associated with its corresponding characteristic vector (eigenvector) (Massy), the statistical significance of its regression coefficient (Mittelhammer and Baritelle) , or some combination of eigenvalue size and correlation with the dependent variable (Johnson, et a/.) . Unfortunately, this technique is widely abused and misunderstood by the applied researcher. Confusion exists with respect to (I) its relationship to other orthogonalization techniques; (2) the meaning of the orthogonalized components and the necessity of transforming the principal component estimators back to the original parameter space; (3) the implications of deleting components and the correspondence of this technique to a particular type of restricted least squares estimator; (4) the proper way to delete components and evaluate these implied restrictions; and (5) actual implementation of this estimation procedure via available computer routines . The purpose of this note, therefore, is to place the technique of principal components in perspective and to suggest a methodology for implementing this technique correctly.