Accurate and Efficient Algorithms that Adapt to Privacy-Enhanced Video for Improved Assistive Monitoring

Automated monitoring algorithms operating on live video streamed from a home can effectively aid in several assistive monitoring goals, such as detecting falls or estimating daily energy expenditure. Use of video raises obvious privacy concerns. Several privacy enhancements have been proposed such as modifying a person in video by introducing blur, silhouette, or bounding-box. Person extraction is fundamental in video-based assistive monitoring and degraded in the presence of privacy enhancements; however, privacy enhancements have characteristics that can opportunistically be adapted to. We propose two adaptive algorithms for improving assistive monitoring goal performance with privacy-enhanced video: specific-color hunter and edge-void filler. A nonadaptive algorithm, foregrounding, is used as the default algorithm for the adaptive algorithms. We compare nonadaptive and adaptive algorithms with 5 common privacy enhancements on the effectiveness of 8 automated monitoring goals. The nonadaptive algorithm performance on privacy-enhanced video is degraded from raw video. However, adaptive algorithms can compensate for the degradation. Energy estimation accuracy in our tests degraded from 90.9% to 83.9%, but the adaptive algorithms significantly compensated by bringing the accuracy up to 87.1%. Similarly, fall detection accuracy degraded from 1.0 sensitivity to 0.86 and from 1.0 specificity to 0.79, but the adaptive algorithms compensated accuracy back to 0.92 sensitivity and 0.90 specificity. Additionally, the adaptive algorithms were computationally more efficient than the nonadaptive algorithm, averaging 1.7% more frames processed per second.