Robust object-identification from inaccurate recognition-based inputs

Abstract

Eyesight and speech are two channels that humans naturally use to communicate with each other. However both the eye tracking and the speech recognition technique existing are still far from perfect. This work explored how to integrate two (or more) error-prone sources of information on users' selection of objects in a visual interface. The implemented system integrated a commercial speech recognition system with gaze tracking in order to improve recognition results. In addition, we employed a new measure of the rate of mutual disambiguation for the multimodal system and conducted an experimental evaluation.