SmallTAL: Real-Time Egocentric Online Temporal Action Localization for the Data-Impoverished

Abstract

We propose a real-time, online temporal action localization system that requires a small amount of annotated data. The main challenges we address are high intra-class variability and a large and diverse background class. We address these using a flexible frame descriptor, dynamic time warping, and a novel approach to database construction. Our solution receives egocentric RGB-D streams as input and makes predictions at regular temporal intervals. We validate our approach by localizing actions in a digital twin of an electrical substation, in which certain objects have been replaced by functional virtual replicas.