A Transformer Based Method with Wide Attention Range for Enhanced Short-term Load Forecasting

Abstract

Short-term Load Forecasting (STLF) is important for the operational security and economics of power system. However, the existing Short-term Load Forecasting Models (SLFMs) generally consider the temporal dependency as static. Meanwhile, the load characteristics of periodic soft alignment and planed are ignored. Those neglect limits the STLF accuracy. In this paper, a Transformer based Short-term Load Forecasting Model (TSLFM) considering dynamic temporal dependency, periodic soft alignment and future information was proposed. Based on the encoder-decoder structure, TSLFM can be easily modified to satisfy different forecast ranges. Besides, the attention mechanism is employed in Transformer, TSLFM can capture the dynamic temporal dependency and realize periodic soft alignment. Additionally, TSLFM expands the attention range to combine historical and future information to infer the planed load. The results from two empirical studies in Switzerland and China suggest that: 1) TSLFM has good forecast performance (the maximum improvement of MAPE is 15.78% and 14.07%, and the minimum improvement is 8.49%, 8.99%, respectively) and can satisfy the high requirements for STLF, and 2) the attention maps further verify that TSLFM can consider dynamic temporal dependency, periodic soft alignment and future information.