UniTS: Short-Time Fourier Inspired Neural Networks for Sensory Time Series Classification

Abstract

Discovering patterns in time series data is essential to many key tasks in intelligent sensing systems, such as human activity recognition and event detection. These tasks involve the classification of sensory information from physical measurements such as inertial or temperature change measurements. Due to differences in the underlying physics, existing methods for classification use handcrafted features combined with traditional learning algorithms, or employ distinct deep neural models to directly learn from raw data. We propose here a unified neural architecture, UniTS, for sensory time series classification in various tasks, which obviates the need for domain-specific feature, model customization or polished hyper-parameter tuning. This is possible as we believe that discriminative patterns in sensory measurements would manifest when we combine information from both the time and frequency domains. In particular, to reveal the commonality of sensory signals, we integrate Short-Time Fourier Transform (STFT) into neural networks by initializing convolutional filter weights as the Fourier coefficients. Instead of treating STFT as a static linear transform with fixed coefficients, we make these weights optimizable during network training, which essentially learns to weigh each frequency channel. Recognizing that time-domain signals might represent intuitive physics such as temperature and acceleration, we combine linearly transformed time-domain hidden features with the frequency components within each time chunk. We further extend our model to multiple branches with different time-frequency resolutions to avoid the need of hyper-parameter search. We conducted experiments on four public datasets containing time-series data from various IoT systems, including motion, WiFi, EEG, and air quality, and compared UniTS with numerous recent models. Results demonstrate that our proposed method achieves an average F1 score of 91.85% with a 2.3-point improvement over the state of the art. We also verified the efficacy of STFT-inspired structures through numerous quantitative studies.