Authoring Interactive and Immersive Experiences Using Programming by Demonstration

Abstract

Immersive technologies, like virtual and augmented reality, allow engaging the general public in interactive experiences, which are particularly promising in educational and cultural activities (e.g. museums, exhibits). For now, the design of such interactive experiences requires extensive knowledge of expert programming tools, and thus they are not accessible to non-developers. A promising method to ease the prototyping of interactive scenes for non-expert users stands on the concept of programming-by-demonstration. With such an approach, novice users can simply demonstrate an interaction or the expected behavior of a virtual object to build their prototype, without knowing the underlying coding mechanisms. In this paper, we propose an immersive authoring system that bases on that approach. We prototyped two use cases based on this system, as well as a user study. Using our observations and the results of the study, we discuss challenges associated with the design of such systems and provide guidelines for the development of future immersive programming-by-demonstration tools.