Information-theoretic and Set-theoretic Similarity

Abstract

We introduce a definition of similarity based on Tversky's set-theoretic linear contrast model and on information-theoretic principles. The similarity measures the residual entropy with respect to a random object. This residual entropy similarity strongly captures context, which we conjecture is important for similarity-based statistical learning. Properties of the similarity definition are established and examples illustrate its characteristics. We show that a previously-defined information-theoretic similarity is also set-theoretic, and compare it to the residual entropy similarity. The similarity between random objects is also treated