Overlapping Schwarz preconditioners for randomized neural networks with domain decomposition

Abstract

Randomized neural networks (RaNNs), characterized by fixed hidden layers after random initialization, offer a computationally efficient alternative to fully parameterized neural networks trained using stochastic gradient descent-type algorithms. In this paper, we integrate RaNNs with overlapping Schwarz domain decomposition in two primary ways: firstly, to formulate the least-squares problem with localized basis functions, and secondly, to construct effective overlapping Schwarz preconditioners for solving the resulting linear systems. Specifically, neural networks are randomly initialized in each subdomain following a uniform distribution, and these localized solutions are combined through a partition of unity, providing a global approximation to the solution of the partial differential equation. Boundary conditions are imposed via a constraining operator, eliminating the necessity for penalty methods. Furthermore, we apply principal component analysis (PCA) within each subdomain to reduce the number of basis functions, thereby significantly improving the conditioning of the resulting linear system. By constructing additive Schwarz (AS) and restricted AS preconditioners, we efficiently solve the least-squares problems using iterative solvers such as the Conjugate Gradient (CG) and generalized minimal residual methods. Numerical experiments clearly demonstrate that the proposed methodology substantially reduces computational time, particularly for multi-scale and time-dependent PDE problems. Additionally, we present a three-dimensional numerical example illustrating the superior efficiency of employing the CG method combined with an AS preconditioner over direct methods like QR decomposition for solving the associated least-squares system.