Ergonomic assessment of mid-air interaction and device-assisted interactions under vibration environments based on task performance, muscle activity and user perceptions

Mid-air interaction has been increasingly introduced for human-computer interaction (HCI) tasks in vibration environments, but it has seldom been assessed from ergonomic aspects, especially in comparison with device-assisted interactions. This study aimed to provide a comprehensive ergonomic assessment of mid-air interaction and device-assisted interactions under vibration environments based on task performance, muscle activity in the upper limb and shoulder, and user perceptions. A within-subjects design was implemented in this study, where participants were required to perform basic pointing and dragging tasks with four interaction modes (i.e., one mid-air interaction and three device-assisted interactions) under static, low and high vibration environments, respectively. Both small and large target sizes were examined. Muscle activity was recorded with surface electromyography for five muscles from participants’ dominant arm. Results showed that mid-air interaction yielded longer task completion time, more errors, higher perceived workload, lower usability ratings, and larger muscle activities in the forearm, upper arm and shoulder compared with device-assisted interactions. There were significant interaction effects between vibration and interaction mode. Specifically, compared with device-assisted interactions, mid-air interaction was associated with greater susceptibility to the detrimental effects of vibration (poorer task performance and larger muscle activities). Target size significantly affected task performance, but the effects varied by tasks. Overall, our results suggest that mid-air interaction presents a higher ergonomic risk compared with device-assisted interactions, especially in vibration environments. These findings provide implications for better use, configuration and ergonomic assessment of interaction tools in vibration environments, and are useful in developing evidence-based interventions to control ergonomic risk in HCI tasks in vibration environments.