Exploring how expert and novice evaluators perceive G4H to consolidate heuristic evaluation

Abstract

Game for Heuristic Evaluation (G4H) is a gamification technique designed to increase the evaluators' motivation and engagement during the Heuristic Evaluation (HE) consolidation. The preliminary study conducted to validate G4H has shown promising results for improving evaluators' motivation and satisfaction, however, G4H needed further evaluation considering other subjects. In this paper, we present an exploratory study carried out to investigate the G4H effectiveness for expert and novice evaluators and identify the perceptions that they have on G4H use in HE consolidation. The study was subdivided into two parts: experts are subjects in Study A and novices are subjects in Study B. In each study we conducting a user study to investigate the G4H use by the participants, and then performing a focus group to collect the participant's perceptions regarding G4H use. Our findings indicate that G4H fulfill the intention to improve motivation, satisfaction, and collaborative work, but had a different impact on experts and novices. Experts used G4H to systematize the HE consolidation tasks, and novices perceive G4H as a ludic approach to consolidate HE.