A hierarchical transformer-based network for multivariate time series classification

Abstract

In recent years, Transformer has demonstrated considerable potential in multivariate time series classification due to its exceptional strength in capturing global dependencies. However, as a generalized approach, it still faces challenges in processing time series data, such as insufficient temporal sensitivity and inadequate ability to capture local features. In this paper, a hierarchical Transformer-based network (Hformer) is innovatively proposed to address these problems. Hformer handles time series progressively at various stages to aggregate multi-scale representations. At the start of each stage, Hformer segments the input sequence and extracts features independently on each temporal slice. Furthermore, to better accommodate multivariate time series data, a more efficient absolute position encoding as well as relative position encoding are employed by Hformer. Experimental results on 30 multivariate time series datasets of the UEA archive demonstrate that the proposed method outperforms most state-of-the-art methods.