Assessing suitability of similarity coefficients in measuring human mental models

Abstract

Understanding human mental models is important for designers to build their interactive systems or training programs. Human mental models can be realized and quantified through the card sorting technique and the employment of similarity coefficients. However, an apparent lack of justification for the choice of employed similarity coefficients in previous studies shows the necessity of research on this subject. In this study, we applied the card sorting technique, questionnaires and statistical analysis to assess eight different similarity coefficients. Results showed that five similarity coefficients, Anderberg, Dice, Jaccard, Ochiai, and Ochiai II, were relatively suitable in measuring human mental models compared to the other three coefficients, Rogers-Tanimoto, Russell-Rao, and Simple Matching. The results may be due to the major difference of these similarity coefficients in including or excluding the count of the number of categories in which a concerned pair of cards was not present. Potential gender effect on similarity judgment was found and discussed. The findings of this study can provide useful guidelines for the use of similarity coefficients in measuring human mental models.