The Uses of Entropy in the Multivariate Analysis of Categorical Variables

Abstract

Entropy analysis is suggested as a useful technique for the examination of multivariate relationships among categorical variables. It makes no assumptions about the numerical properties of variables nor about the nature of the interrelationships among sets of variables. The technique is able to describe completely the explanatory power of all the constraints within a set of variables and to partition them into the following categories: distributional constraints, internal constraints, external constraints and conditional entropy (unconstrained). After discussing the entropy concept and outlining the statistics, we perform an entropy analysis of Converse's "Belief Systems of Mass Publics" data, controlling for social groups. The results show that the responses of the mass sample to the several political issue items are as constrained as those of the candidate sample.