Portable Real-Time System for Multi-Subject Localization and Vital Sign Estimation

Abstract

A real-time non-contact vital sign detection system is developed by utilizing neural network-based detection, multi-object tracking, and direction of arrival (DoA) techniques. The DoA produces a spatial-based image, which is fed into the detector. The detector is a convolutional neural network (CNN), which produces a list potential subject locations. These locations are propagated and associated via a tracking method called BYTE. All of these methods allow the system to accurately localize and track subjects as well as improve the robustness of vital sign estimation for stationary, multi-subject scenarios. We demonstrate that this real-time system produces low error rates of less than 1 and 3 BPM for breathing and heart rate estimations respectively in both single and multi-subject scenarios. All this is done while maintaining an average of 14 FPS on a portable Jetson Xavier NX.