Challenges of creating multimodal interfaces on mobile devices

Abstract

Smart mobile devices support web technologies, audio and video playback, integrated extra features and 3rd party development beyond telephony. Multimodal user interfaces are favorable in mobile environments, when the traditional standard modality (keyboard and stylus) is supported by other modalities, primarily by speech input and output. Multimodality increases the usability of applications and makes them accessible for impaired persons, although creating multimodal interfaces on mobile devices is a challenging task. This paper investigates the main problems, briefly examines the upcoming issues of the graphical user interface and introduces the most important challenges of creating speech user interfaces on mobile devices. A new approach of creating multimodal, scaleable user interfaces for mobile devices is shortly discussed and some application scenarios are given, as well.