Evaluating gesture user interfaces: Quantitative measures, qualitative scales, and method

Abstract

Although many methods currently exist to evaluate a user interface, they have been mainly developed and applied for graphical and vocal user interfaces, leaving aside other modalities such as gesture interaction. As a consequence, the evaluation of the global quality of a gesture user interface most often resorts to these methods, which take little or no explicit and specific account of the gesture modality or which adapt existing methods in a way that jeopardizes their validity. To remedy this situation, this paper introduces, defines, and justifies a method for evaluating explicitly and specifically a gesture user interface based on a conceptual model that consists of: (1) six quantitative measures covering user and system aspects, such as gesture thinking time and recognition rate; (2) three new qualitative scales, i.e., discoverability, learnability, and social acceptability, combined with seven scales in a gesture evaluation scheme, which is formally based on four measures, i.e., subscale score, subscale importance, scale mean score and scale mean importance; (3) a debriefing interview to cross-analyze the results of the quantitative measures and qualitative scales. This method also includes the use of multiple sessions to take into account short and long-term memory, as well as discoverability, and integrates tests specific to each gesture, as well as concrete use cases. For validation and illustration purposes, this method is applied to a gesture user interface serving as a case study. The results are then analyzed, starting with classic methods and gradually adding the particularities of our method to highlight its added value. Based on this experiment, we suggest some implications for evaluating gesture interfaces and for incorporating these scales into UEQ+, a modular user interface evaluation method.