Voice-Enabled Blockly: Usability Impressions of a Speech-driven Block-based Programming System

Abstract

Block-based programming environments pose a challenge for people with upper-limb motor impairments. This is because they are highly dependent on the physical manipulation of a mouse or keyboard to drag and drop elements on the screen. Our research aims to make the block-based programming environment Blockly, accessible to users with upper limb motor impairments by adding voice as an alternative input modality. This voice-enabled version of Blockly will reduce the need for the use of a pointing device, thus increasing access for people with limited dexterity. The Voice-enabled Blockly system consists of the Blockly application, a speech recognition API, predefined voice commands, and a custom function. A usability study was conducted using a prototype of Voice-enabled Blockly. The results revealed that people with upper-limb motor impairments can use the system. However, it also exposed some shortcomings of the tool and gave some suggestions on how to fix them. Based on the findings, changes will be made to the system, and then, it will be evaluated in another user study in the near future.