Feature extractors for integration of cameras and sensors during end-user programming of assistive monitoring systems

Abstract

Assistive monitoring systems increasingly include cameras along with sensors. End-users require the capability to program such systems to monitor user-specified events and provide customized notifications in response. We introduce feature extractors, which provide a means for integrating camera video with sensor data. A feature extractor takes a video stream as input, and outputs a stream of integer values corresponding to the amount of a particular sensor phenomenon such as motion, sound, or light, or of more advanced phenomena such as human motion, screams, or falls. Feature extractors provide an elegant means for end-users to integrate cameras into their monitoring programs. We introduce feature extractors, provide examples illustrating their effectiveness for various common assistive monitoring scenarios, and summarize usability trials with 51 lay users demonstrating 56%-96% correct utilization of feature extractors.