Extending Mobile Interaction Through Near-Field Visible Light Sensing

Abstract

Mobile devices are shrinking their form factors for portability, but user-mobile interaction is becoming increasingly challenging. In this paper, we propose a novel system called Okuli to meet this challenge. Okuli is a compact, low-cost system that can augment a mobile device and extend its interaction workspace to any nearby surface area. Okuli piggybacks on visible light communication modules, and uses a low-power LED and two light sensors to locate user's finger within the workspace. It is built on a light propagation/reflection model that achieves around one-centimeter location precision, with zero run-time training overhead. We have prototyped Okuli as an Android peripheral, with a 3D-printed shroud to host the LED and light sensors. Our experiments demonstrate Okuli's accuracy, stability, energy efficiency, as well as its potential in serving virtual keyboard and trackpad applications.