Tiny Time-Series Transformers: Realtime Multi-Target Sensor Inference At The Edge

Abstract

Large-scale wireless sensor networks have become an invaluable tool for dense spatiotemporal modeling of urban air pollution. When coupled with complex nonlinear regression schemes, they become an unparalleled tool capable of dynamic, autonomous sensor calibration as well as completely latent parametric inference. In this work we present T3: The Tiny Time-Series Transformer, a hard-shared multi-target deep neural network based on the Transformer Encoder architecture and designed for multivariate realtime inference at the edge of large-scale environmental sensor networks. We demonstrate our approach by deploying T3 to an active pollution monitoring network, where it is tasked with the multi-target output of calibrated particulate matter and temperature, as well as the latent inference of tropospheric ozone, using fused time-series measurements from the onboard sensors as input. We show that T3 greatly outperforms classical linear regression techniques while matching accuracy of current state-of-the-art nonlinear regression architectures at a fraction of the footprint size.