Vocal Programming for People with Upper-Body Motor Impairments

Programming heavily relies on entering text using traditional QWERTY keyboards, which poses challenges for people with limited upper-body movement. Developing tools using a publicly available speech recognition API could provide a basis for keyboard free programming. In this paper, we describe our efforts in design, development, and evaluation of a voice-based IDE to support people with limited dexterity. We report on a formative Wizard of Oz (WOz) based design process to gain an understanding of how people would use and what they expect from a speech-based programming environment. Informed by the findings from the WOz, we developed VocalIDE, a prototype speech-based IDE with features such as Context Color Editing that facilitates vocal programming. Finally, we evaluate the utility of VocalIDE with 8 participants who have upper limb motor impairments. The study showed that VocalIDE significantly improves the participants' ability to make navigational edits and select text while programming.