Human-robot and robot-robot sound interaction using a 3-Dimensional Acoustic Ranging (3DAR) in audible and inaudible frequency

Abstract

Highly reliable sound-based interaction in noisy and dynamic environments is a known challenge with simultaneous Human-Robot Interaction (HRI) and Robot-Robot Interaction (RRI). Here, we introduce 3-dimensional acoustic ranging (3DAR) using the meta-structured single microphone rotation, a compact system with a three-dimensional meta-structure with a phase-cancellation mechanism for enhanced beamforming across audible and inaudible frequencies. Inspired by dolphin communication, the 3DAR employs frequency modulation and separation of sound channels for seamless HRI and RRI. The system achieved over 90 % accuracy in multiple source localization for HRI and 99 % for RRI, even in challenging noise conditions, along with 94 % accuracy in tracking multiple sound sources. Furthermore, real-world tests in a factory demonstrated 95.6 % accuracy in multi-HRI localization and up to 93.8 % accuracy when human speech direction originated from an angle of 120° relative to the system. Real-world application of a collaborative rescue robot confirmed effectiveness of the 3DAR in various applications, highlighting its potential for robust, sound-based collaboration among humans and robots.