AI-driven Event Recognition with a Real-Time 3D 60-GHz Radar System

A vertically integrated antennas-to-AI system is presented. 60-GHz 16-element phased array transmitter and receiver modules, previously developed for Gb/s NLOS communications, are used to implement a 3D radar system that extracts volumetric information from a scene at a high frame rate. The system employs an FMCW signal with 1-GHz bandwidth and can process 1250 radar readouts per second. An efficient timing control scheme between the radar electronics and the phased array module control enables obtaining each of the radar readouts from a separate beam direction. The system can scan a frame of 5×5 directions 50 times per second. All the radar system components including signal generation and ADC are assembled in a single portable chassis. A camera is also included in the system to enable the simultaneous capture of radar and video streams. A DNN was developed to extract temporal and volumetric features from the 3D radar information stream and enable the automatic recognition of fast evolving events. As an application example, the DNN was trained to perform automatic hand gesture recognition. The overall radar system and the associated DNN achieved a recognition accuracy of 93% on a set of 9 different gestures involving two hands.