Inferring smartphone keypress via smartwatch inertial sensing

Abstract

Due to numerous benefits, sensor-rich smartwatches and wrist-worn wearable devices are quickly gaining popularity. The popularity of these devices also raises privacy concerns. In this paper we explore one such privacy concern: the possibility of extracting the location of a user's touch-event on a smartphone, using the inertial sensor data of a smartwatch worn by the user on the same arm. This is a major concern not only because it might be possible for an attacker to extract private and sensitive information from the inputs provided but also because the attack mode utilises a device (smartwatch) that is distinct from the device being attacked (smartphone). Through a user study we find that such attacks are possible. Specifically, we can infer the user's entry pattern on a qwerty keyboard, with an error bound of ±2 neighboring keys, with 73.85% accuracy. As a possible preventive mechanism, we also show that adding a little white noise to inertial sensor data can reduce the inference accuracy by almost 30%, without affecting the accuracy of macro-gesture recognition.