[POSTER] Affording Visual Feedback for Natural Hand Interaction in AR to Assess Upper Extremity Motor Dysfunction

Abstract

For the clinical community, there is great need for objective, quantitative and valid measures of the factors contributing to motor dysfunction. Currently, there are no standard protocols to assess motor dysfunction in various patient groups, where each medical discipline uses subjectively scored clinical tests, qualitative video analysis, or marker-based motion capturing. We investigate the potential of Augmented Reality (AR) combined with serious gaming and marker-less tracking of the hand to facilitate efficient, cost-effective and patient-friendly methods for evaluation of upper extremity motor dysfunction in different patient groups. First, the design process of the game and the system architecture of the AR framework are described. To provide unhindered assessment of motor dysfunction, patients should operate with the system in a natural way and be able to understand their actions in the virtual AR world. To test this in our system, we conducted a usability study with five healthy people (aged between 57-63) on three different modalities of visual feedback for natural hand interaction with AR objects. These modalities are: no augmented hand, partial augmented hand (tip of index finger and tip of thumb) and a full augmented hand model. The results of the study show that a virtual representation of the fingertips or hand improves the usability of natural hand interaction.