Natural multimodal interaction in immersive flow visualization

Abstract

In the immersive flow visualization based on virtual reality, how to meet the needs of complex professional flow visualization analysis by natural human–computer interaction is a pressing problem. In order to achieve the natural and efficient human–computer interaction, we analyze the interaction requirements of flow visualization and study the characteristics of four human–computer interaction channels: hand, head, eye and voice. We give out some multimodal interaction design suggestions and then propose three multimodal interaction methods: head & hand, head & hand & eye and head & hand & eye & voice. The freedom of gestures, the stability of the head, the convenience of eyes and the rapid retrieval of voices are used to improve the accuracy and efficiency of interaction. The interaction load is balanced by multimodal interaction to reduce fatigue. The evaluation shows that our multimodal interaction has higher accuracy, faster time efficiency and much lower fatigue than the traditional joystick interaction.