Clean Vibes: Hand Washing Monitoring Using Structural Vibration Sensing

We present a passive and non-intrusive sensing system for monitoring hand washing activity using structural vibration sensing. Proper hand washing is one of the most effective ways to limit the spread and transmission of disease, and has been especially critical during the COVID-19 pandemic. Prior approaches include direct observation and sensing-based approaches, but are limited in non-clinical settings due to operational restrictions and privacy concerns in sensitive areas such as restrooms. Our work introduces a new sensing modality for hand washing monitoring, which measures hand washing activity-induced vibration responses of sink structures, and uses those responses to monitor the presence and duration of hand washing. Primary research challenges are that vibration responses are similar for different activities, occur on different surfaces/structures, and tend to overlap/coincide. We overcome these challenges by extracting information about signal periodicity for similar activities through cepstrum-based features, leveraging hierarchical learning to differentiate activities on different surfaces, and denoting “primary/secondary” activities based on their relative frequency and importance. We evaluate our approach using real-world hand washing data across four different sink structures/locations, and achieve an average F1-score for hand washing activities of 0.95, which represents an 8.8X and 10.2X reduction in error over two different baseline approaches.