{"title":"An analysis and solution of ill-conditioning in physics-informed neural networks","authors":"Wenbo Cao , Weiwei Zhang","doi":"10.1016/j.jcp.2024.113494","DOIUrl":null,"url":null,"abstract":"<div><div>Physics-informed neural networks (PINNs) have recently emerged as a novel and popular approach for solving forward and inverse problems involving partial differential equations (PDEs). However, ensuring stable training and obtaining accurate results remain challenging in many scenarios, often attributed to the ill-conditioning of PINNs. Despite this, a deeper analysis is still lacking, which hampers progress and application of PINNs in complex engineering problems. Drawing inspiration from the ill-conditioning analysis in traditional numerical methods, we establish a strong connection between the ill-conditioning of PINNs and the Jacobian matrix of the PDE system. Specifically, for any given PDE system, we construct a controlled system that allows for the adjustment of the Jacobian matrix's condition number while retaining the same solution as the original system. Our numerical experiments show that as the condition number of the Jacobian matrix decreases, PINNs exhibit faster convergence and higher accuracy. Building upon this principle and the extension of controlled systems, we propose a general approach to mitigate the ill-conditioning in PINNs, leading to successful simulations of three-dimensional flow around the M6 wing at a Reynolds number of 5,000. To the best of our knowledge, this is the first time that PINNs have successfully simulated such complex systems, offering a promising new technique for addressing industrial complexity problems. Our findings also provide valuable insights to guide the future development of PINNs.</div></div>","PeriodicalId":352,"journal":{"name":"Journal of Computational Physics","volume":"520 ","pages":"Article 113494"},"PeriodicalIF":3.8000,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computational Physics","FirstCategoryId":"101","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0021999124007423","RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Physics-informed neural networks (PINNs) have recently emerged as a novel and popular approach for solving forward and inverse problems involving partial differential equations (PDEs). However, ensuring stable training and obtaining accurate results remain challenging in many scenarios, often attributed to the ill-conditioning of PINNs. Despite this, a deeper analysis is still lacking, which hampers progress and application of PINNs in complex engineering problems. Drawing inspiration from the ill-conditioning analysis in traditional numerical methods, we establish a strong connection between the ill-conditioning of PINNs and the Jacobian matrix of the PDE system. Specifically, for any given PDE system, we construct a controlled system that allows for the adjustment of the Jacobian matrix's condition number while retaining the same solution as the original system. Our numerical experiments show that as the condition number of the Jacobian matrix decreases, PINNs exhibit faster convergence and higher accuracy. Building upon this principle and the extension of controlled systems, we propose a general approach to mitigate the ill-conditioning in PINNs, leading to successful simulations of three-dimensional flow around the M6 wing at a Reynolds number of 5,000. To the best of our knowledge, this is the first time that PINNs have successfully simulated such complex systems, offering a promising new technique for addressing industrial complexity problems. Our findings also provide valuable insights to guide the future development of PINNs.
期刊介绍:
Journal of Computational Physics thoroughly treats the computational aspects of physical problems, presenting techniques for the numerical solution of mathematical equations arising in all areas of physics. The journal seeks to emphasize methods that cross disciplinary boundaries.
The Journal of Computational Physics also publishes short notes of 4 pages or less (including figures, tables, and references but excluding title pages). Letters to the Editor commenting on articles already published in this Journal will also be considered. Neither notes nor letters should have an abstract.