A Descent Algorithm for the Optimal Control of ReLU Neural Network Informed PDEs Based on Approximate Directional Derivatives

SIAM Journal on Optimization, Volume 34, Issue 3, Page 2314-2349, September 2024.
Abstract. We propose and analyze a numerical algorithm for solving a class of optimal control problems for learning-informed semilinear partial differential equations (PDEs). Such PDEs contain constituents that are in principle unknown and are approximated by nonsmooth ReLU neural networks. We first show that direct smoothing of the ReLU network with the aim of using classical numerical solvers can have disadvantages, such as potentially introducing multiple solutions for the corresponding PDE. This motivates us to devise a numerical algorithm that treats directly the nonsmooth optimal control problem, by employing a descent algorithm inspired by a bundle-free method. Several numerical examples are provided and the efficiency of the algorithm is shown.