IMAGE: a deployment framework for creating multimodal experiences of web graphics

Abstract

Existing screen reader software can convey graphical content to blind and low vision web users through text information, but does not offer richer multimedia representations. Standalone research projects have attempted to fill this gap, but have not achieved lasting, widespread deployment, thus motivating the creation of a common platform for implementing and deploying multimodal experiences. We are creating the IMAGE system to be an open-source "playground" for prototyping, exploring, and deploying novel solutions to accomplish this. IMAGE does not replace a screen reader or alt-tags, but rather works with them to provide a more complete understanding of web graphics. In this communication, we describe how the IMAGE browser extension and server components form a modular, extensible system that can accelerate the development of new haptic and audio renderings. We explain how various parties, whether as developers or designers, can benefit from this architecture, building on our pipeline for their own purposes.