Interacting with our Environment through Sentient Mobile Phones

Abstract

The latest mobile phones are offering more multimedia features, better communication capabilities (Bluetooth, GPRS, 3G) and are far more easily programmable (extensible) than ever before. So far, the “killer apps” to exploit these new capabilities have been presented in the form of MMS (Multimedia Messaging), video conferencing and multimedia-on-demand services. We deem that a new promising application domain for the latest Smart Phones is their use as intermediaries between us and our surrounding environment. Thus, our mobiles will behave as personal butlers who assist us in our daily tasks, taking advantage of the computational services provided at our working or living environments. For this to happen, a key element is to add senses to our mobiles: capability to see (camera), hear (michrophone), notice (Bluetooth) the objects and devices offering computational services. In this paper, we present a solution to this issue, the MobileSense system. We illustrate its use in two scenarios: (1) making mobiles more accessible to people with disabilities and (2) enabling the mobiles as guiding devices within a museum.