Using gesture and speech communication modalities for safe human-drone interaction in construction

Abstract

Drones are increasingly being used in the construction industry for numerous applications. However, their presence poses safety risks to construction workers who work around them but have limited control and information about these drones. To ensure safety, general construction workers who are not part of the pilot teams should also be able to communicate their concerns with drones effectively and naturally. Despite its importance, research on human-drone communication within construction for non-operator workers is scarce. This study developed and evaluated communication protocols using gesture and speech modalities to ensure safe human-drone interactions for non-operator workers in construction environments. An immersive VR environment replicating construction site dynamics was developed, enabling workers to utilize gesture or speech communication protocols while working with drones. A total of 100 participants were recruited for the user-centered study analysis on an immersive VR construction site, and the safety implications and cognitive loads of both protocols were assessed both quantitatively and qualitatively. The findings suggest that gesture-based communication is more effective than speech-based communication in mitigating risks and alleviating the negative impacts of drones without imposing additional cognitive strain on users on construction sites.