Adaptive Multistep Prediction With Sequence-to-Sequence (Seq2Seq) Models

This brief demonstrates for the first time that the sequence-to-sequence (Seq2Seq) model is an adaptive multistep predictor. The Seq2Seq model is fixed-weight adaptive, which means that the model can adapt to time-varying behaviors without having to update its weights and biases. Instead, the learning algorithm is embedded into the recurrent neural network (RNN) decoder. This brief examines the Seq2Seq model’s ability to adapt to time-varying behaviors using both simulated and experimental data, and it also identifies a mechanism within the model that enables the adaptation.