Proximal auditory AR rehabilitation: system integration and wellness applications—from hearing support up toward vestibular rehabilitation

We present a new rehabilitation system based on novel principles, which consists of an auditory augmented reality (AR) headset we originated. The auditory AR headset, which does not cover both ears, allows users to hear both Real and Virtual environmental sounds at the same time. It can also be used in combination with Hearing Aids. We have studied a system to support hearing-impaired people and conducted a test evaluation. The system was able to provide convenience akin to “reading glasses for sound” to those who had mild hearing disabilities. Furthermore, by combining the system with surrounding speakers, a completely novel virtual auditory illusion was created in which the sound image jumps into the ear and runs away. We name this “proximal auditory AR (PAAR)” system. This system directly affects the unconscious level of reflexes for maintaining a standing position and can generate very subtle body motion disturbance. Using this system, we can modulate the standing posture and observe the autonomic nerve system's ability to subliminally compensate for the disturbance, using a stabilometer that measures body sways by center of pressure (COP). We observed a significant difference in the declination of COP only when using the PAAR, which is combined with array speakers and the auditory AR headphone, compared using a conventional closed-type and a bone-conduction headphone. By analyzing such big data of physical movement through machine learning, we expect to realize new systems for diagnosis, rehabilitation, function maintenance, and fall prevention.