Surrogate construction via weight parameterization of residual neural networks

IF 6.9 1区 工程技术 Q1 ENGINEERING, MULTIDISCIPLINARY Computer Methods in Applied Mechanics and Engineering Pub Date : 2024-10-30 DOI:10.1016/j.cma.2024.117468
{"title":"Surrogate construction via weight parameterization of residual neural networks","authors":"","doi":"10.1016/j.cma.2024.117468","DOIUrl":null,"url":null,"abstract":"<div><div>Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. In this work we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.</div></div>","PeriodicalId":55222,"journal":{"name":"Computer Methods in Applied Mechanics and Engineering","volume":null,"pages":null},"PeriodicalIF":6.9000,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Methods in Applied Mechanics and Engineering","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0045782524007230","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. In this work we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
通过残差神经网络的权重参数化构建代用系统
对于复杂计算模型的不确定性量化或其他样本密集型任务来说,代用模型的开发是一个关键步骤。在这项工作中,我们利用一类采用捷径连接的神经网络(NN),即残差神经网络(ResNets),开发了一种多输出代用形式。众所周知,残差神经网络可以规范代用学习问题,并提高代用结果的效率和准确性。受连续神经 ODE 类比的启发,我们采用与 ResNet 深度相关的权重参数化策略来增强 ResNets。权重参数化 ResNets 可规范化 NN 代理学习问题,并在大幅减少可学习参数数量的情况下实现更好的泛化。我们证明,权重参数化 ResNets 比传统的前馈多层感知器网络更准确、更高效。我们还比较了权重参数化作为 ResNet 深度函数的各种选项。我们在合成示例和感兴趣的大型地球系统模型上演示了结果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
12.70
自引率
15.30%
发文量
719
审稿时长
44 days
期刊介绍: Computer Methods in Applied Mechanics and Engineering stands as a cornerstone in the realm of computational science and engineering. With a history spanning over five decades, the journal has been a key platform for disseminating papers on advanced mathematical modeling and numerical solutions. Interdisciplinary in nature, these contributions encompass mechanics, mathematics, computer science, and various scientific disciplines. The journal welcomes a broad range of computational methods addressing the simulation, analysis, and design of complex physical problems, making it a vital resource for researchers in the field.
期刊最新文献
Peridynamic modelling of time-dependent behaviour and creep damage in hyper-viscoelastic solids with pre-cracks Modeling pulmonary perfusion and gas exchange in alveolar microstructures Data-driven projection pursuit adaptation of polynomial chaos expansions for dependent high-dimensional parameters A novel global prediction framework for multi-response models in reliability engineering using adaptive sampling and active subspace methods Modeling via peridynamics for damage and failure of hyperelastic composites
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1