On the development of a dynamic virtual reality system using audio and visual scenes

Abstract

Virtual reality systems have been widely used in many popular and diverse applications including education and gaming. However, development of a dynamic virtual reality system which combines both audio and visual scenes has hitherto not been investigated. In this work a dynamic virtual reality system which synchronizes both audio and visual information is developed. Realtime audio and visual information is obtained from a spherical audio visual camera with 64 microphones and 5 cameras. Subsequently, a head mounted display application is designed to render spherical video. A three dimensional sound rendering algorithm using head related transfer functions is developed. Finally, a virtual reality system that combines both spherical audio and video is realized. The head position of the user is also integrated into this system adaptively to make the system dynamic. Both subjective and objective evaluations of the proposed virtual reality system indicate its significance.