Adding attention to the neural ordinary differential equation for spatio-temporal prediction

AbstractExplainable spatio-temporal prediction gains attraction in the development of geospatial artificial intelligence. The neural ordinal differential equation (NODE) emerges as a new solution for explainable spatio-temporal prediction. However, challenges still need to be solved in most existing NODE-based prediction models, such as difficulty modeling spatial data and mining long-term temporal dependencies in data. In this study, we propose a spatio-temporal attentional NODE (STA-ODE) to address the two challenges above. First, we define a spatio-temporal ordinary differential equation to predict a value at each time iteratively by a novel spatio-temporal derivative network. Second, we develop an attention mechanism to fuse multiple prediction values for capturing long-term temporal dependencies in data. To train the STA-ODE model, we design a loss function that aligns the prediction results in spatial dimension with prediction results in temporal dimension to calibrate the parameters of the model. The proposed model was validated with three real-world spatio-temporal datasets (traffic flow dataset, PM2.5 monitoring dataset, and temperature monitoring dataset). Experimental results showed that STA-ODE outperformed seven existing baselines regarding prediction accuracy. In addition, we used visualization to demonstrate the sound interpretability and prediction accuracy of the STA-ODE model.Keywords: Geospatial artificial intelligencespatio-temporal predictionspatio-temporal attentionneural ordinary differential equation AcknowledgementsThe numerical calculations in this paper have been done on the supercomputing system in the Supercomputing Center of Wuhan University.Disclosure statementNo potential conflict of interest was reported by the author(s).Data and codes availability statementThe data and codes that support the findings of this study are available in ‘figshare.com’ with the identifier https://doi.org/10.6084/m9.figshare.22678153.Additional informationFundingThis project was supported by National Key Research and Development Program of China [Grant No. 2021YFB3900803], National Postdoctoral Innovation Talents Support Program [Grant No. BX20230360], Open funds of the Wuhan University-Huawei Geoinformatics Innovation Laboratory [Grant No. TC20210901025-2023-04], National Natural Science Foundation of China [Grant Nos. 42101423 and 42371470], Special Research Assistant Program of Chinese Academy of Sciences, Innovation Project of LREIS [Grant No. 08R8A092YA].Notes on contributorsPeixiao WangPeixiao Wang is a Postdoctoral Fellow from State Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Nature Resources Research, Chinese Academy of Sciences. He received Ph.D. degree under from State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, and received the M.S. degree from The Academy of Digital China, Fuzhou University. His research topics include spatiotemporal data mining, and spatiotemporal prediction, especially focus on spatiotemporal prediction of transportation systems.Tong ZhangTong Zhang is a Professor with the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University. He received the M.Eng. degree in cartography and geographic information system (GIS) from Wuhan University, Wuhan, China, in 2003, and the Ph.D. degree in geography from San Diego State University, and the University of California at Santa Barbara in 2007. His research topics include urban computing and machine learning.Hengcai ZhangHengcai Zhang is an Associate Professor of State Key Laboratory of Resources and Environmental Information Systems, Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences. He received his Ph.D. degree from Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences. He is the member of the Theory and Methodology Committee of the Chinese Association of Geographic Information System, and member of Chinese Branch of ACM SIGSPATIAL. His interests focus on spatial-temporal data mining and 3D-Computing.Shifen ChengShifen Cheng is an Associate Professor of State Key Laboratory of Resources and Environmental Information Systems, Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences. He received his Ph.D. degree from Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences. His research interests include spatiotemporal data mining, urban computing and intelligent transportation.Wangshu WangWangshu Wang is a postdoctoral fellow at the Research Unit Cartography at the Vienna University of Technology. She received her Ph.D. degree from the Vienna University of Technology in 2023. Her research focuses on spatiotemporal data mining and indoor pedestrian navigation.