Poster: fast, automatic iPhone shoulder surfing

Touchscreen devices increase the risk of shoulder surfing to such an extent that attackers could steal sensitive information by simply following the victim and observe his or her portable device. We underline this concern by proposing an automatic shoulder surfing attack against modern touchscreen keyboards that display magnified keys in predictable positions. We demonstrate this attack against the Apple iPhone - although it can work with other layouts and different devices - and show that it recognizes up to 97.07% (91.03% on average) of the keystrokes, with only 1.15% of errors, at 37 to 51 keystrokes per minute: About eight times faster than a human analyzing a recorded video. Our attack, described thoroughly in [2], accurately recovers the sequence of keystrokes input by the user. The attack described in [1], which targeted desktop scenarios and thus worked with very restrictive settings, is similar in spirit to ours. However, as it assumes that camera and target keyboard are both in fixed, perpendicular position, it cannot suite mobile settings, characterized by moving target and skewed, rotated viewpoints. Our attack, instead, requires no particular settings and even allows for natural movements of both target device and shoulder surfer's camera. In addition, our attack yields accurate output without any grammar or syntax checks, so that it can detect large context-free text or non-dictionary words. In summary: - We are the first studying the practical risks brought forth by mainstream touchscreen keyboards. - We design a practical attack that detects keystrokes on modern touchscreen keyboards: The attacker requires not to stand exactly behind the victim nor to observe the screen perpendicularly. Our attack is robust to occlusions (eg, typing fingers), thanks to our efficient filtering technique that validates detected keys and reconstructs keystroke sequences accurately.