GPINN with Neural Tangent Kernel Technique for Nonlinear Two Point Boundary Value Problems

Abstract

Neural networks as differential equation solvers are a good choice of numerical technique because of their fast solutions and their nature in tackling some classical problems which traditional numerical solvers faced. In this article, we look at the famous gradient descent optimization technique, which trains the network by updating parameters which minimizes the loss function. We look at the theoretical part of gradient descent to understand why the network works great for some terms of the loss function and not so much for other terms. The loss function considered here is built in such a way that it incorporates the differential equation as well as the derivative of the differential equation. The fully connected feed-forward network is designed in such a way that, without training at boundary points, it automatically satisfies the boundary conditions. The neural tangent kernel for gradient enhanced physics informed neural networks is examined in this work, and we demonstrate how it may be used to generate a closed-form expression for the kernel function. We also provide numerical experiments demonstrating the effectiveness of the new approach for several two point boundary value problems. Our results suggest that the neural tangent kernel based approach can significantly improve the computational accuracy of the gradient enhanced physics informed neural network while reducing the computational cost of training these models.