Designing 3D Gesture Guidance: Visual Feedback and Feedforward Design Options

Abstract

Dynamic symbolic in-air hand gestures are an increasingly popular means of interaction with smart environments. However, novices need to know what commands are available and which gesture to execute in order to trigger these commands. We propose to adapt OctoPocus, a 2D gesture guiding system, to the case of 3D. The OctoPocus3D guidance system displays a set of 3D gestures as 3D pipes and allows users to understand how the system processes gesture input. Several feedback and feedforward visual alternatives are proposed in the literature. However, their impact on guidance remains to be evaluated. We report the results of two user experiments that aim at designing OctoPocus3D by exploring these alternatives. The results show that a concurrent feedback, which visually simplifies the 3D scene during the execution of the gesture, increases the recognition rate, but only during the first two repetitions. After the first two repetitions, users achieve the same recognition rate with a terminal feedback (after the execution of the gesture), a concurrent feedback, both or neither. With respect to feedforward, the overall stability of the 3D scene explored through the origin of the pipes during the execution of the gestures does not influence the recognition rate or the execution time. Finally, the results also show that displaying upcoming portions of the gestures allows 8% faster completion times than displaying the complete remaining portions. This indicates that preventing visual clutter of the 3D scene prevails over gesture anticipation.