GMTPM: A General Multitask Pretrained Model for Electricity Data in Various Scenarios

Abstract

The smart grid has become increasingly complex due to the integration of diverse energy sources and loads. Recognizing the limitations of traditional data sharing for future applications, the need to share model between different systems becomes unavoidable. An universal pretrained model for electricity data representation meets two primary challenges: The diversity of electricity data types and the range of issues in electricity analysis. To address these issues, this study presents a comprehensive univariate time-series representation learning framework called GMTPM for multiple tasks, for the first time in electricity analysis. This method extracts comprehensive feature extraction from univariate electricity data and shows improved performance across various tasks and data scenarios. Our approach incorporates a hybrid contextual attention mechanism to learn both local and global contextual relationships within the data. To adapt to various tasks, we optimize the model through three objectives: Reconstruction-based regression, context-consistent comparative learning, and time-consistent comparative learning. At last, we conducted extensive experiments on multiple univariate time-series datasets and achieved competitive performance in time-series analysis tasks such as imputation, forecasting, classification, anomaly detection, zero-shot forecasting, and few-shot classification.