On a computational paradigm for a class of fractional order direct and inverse problems in terms of physics-informed neural networks with the attention mechanism

Abstract

Physics-Informed Neural Networks (PINNs) have recently gained significant attention for their ability to solve both forward and inverse problems associated with linear and nonlinear fractional partial differential equations (PDEs). However, PINNs, relying on feedforward neural networks (FNNs), overlook the crucial temporal dependencies inherent in practical physics systems. As a result, they fail to globally propagate the initial condition constraints and accurately capture the true solutions under various scenarios. In contrast, the attention mechanism offers flexible means to implicitly exploit patterns within inputs and, moreover, establish relationships between arbitrary query locations and inputs. Thus, we present an attention-based framework for PINNs, which we term PINNs-Transformer (Zhao et al., 2023). The framework was constructed using self-attention and a set of point-wise multilayer perceptrons (MLPs). The novelty is in applying the framework to the various fractional differential equations with stiff dynamics as well as their inverse formulations. We have also validated the PINNs-Transformer on two examples: one involving a fractional diffusion differential equation over time, and the other focused on identifying a space-dependent parameter associated with the direct problem described in the first example. We reinforce this finding by conducting a numerical comparison with variant of PINN methods based on criteria such as relative error, complexity, memory needs and execution time.