Assessment of panel performance in CATA and RATA experiment

Abstract

CATATIS is a recently developed method for the analysis of multiple binary datasets such as Check-All-That-Apply (CATA) data. CATATIS yields a homogeneity index reflecting the overall agreement among the panelists and weights associated with the panelists that highlight the extent to which they agree with the general point of view. These weights are used to compute an average group configuration, which is submitted to Correspondence Analysis to derive a perceptual product map. Here, CATATIS is extended to situations where the data are not binary. In particular, we focus on the case of data from a CATA experiment with repetitions and data from a Rate-All-That-Apply (RATA) experiment. Furthermore, a hypothesis-testing framework based on permutations is set up to assess the significance of the weights associated with the panelists. Finally, the general strategy of analysis is illustrated using data from real case studies pertaining to CATA and RATA experiments.

Practical Applications

Evaluating panel performance is often important in sensory studies, especially in industrial contexts. While many professionals are familiar with evaluating the outputs of conventional descriptive analysis, the increasing adoption of rapid sensory methods calls for methods and tools tailored to different sensory tasks. Our paper introduces an approach (CATATIS) to enhance the interpretation of the outputs of CATA and RATA tasks, focusing on attribute consistency, overall panel agreement, and individual weights associated with panelists. The approach is demonstrated through concrete case studies. CATATIS, together with its extensions, are available in XLSTAT software and in the R package ClustBlock.