Real-time collection and analysis of 3-Kinect v2 skeleton data in a single application

Abstract

It was not possible to do reliable 3D skeletal tracking with the currently publicly available inexpensive consumer grade hardware/software tools, such as depth cameras and their SDKs using multiple of such sensors in a single application (e.g., a game, motion recording for animation, or 3D scanning). We successfully attached 3 Kinect v2 sensors to a single application to track skeletal data without using Microsoft's Kinect 2 SDK. We created a new toolkit -- MultiCamTk++ for 3 or more Kinects v2 with skeleton support in C++. It is a successor of our previous version, MultiCamTk, done in Processing/Java that had no skeletal tracking. We achieve high resiliency and good frame rate even if 1--2 Kinects are disconnected at runtime. We are able to receive the skeleton data from the multiple sources to correlate the coordinates for spatial 3D user tracking.