连续时间随机梯度下降的收敛性及其在线性深度神经网络中的应用

arXiv - STAT - Machine Learning Pub Date : 2024-09-11 DOI:arxiv-2409.07401

Gabor Lugosi, Eulalia Nualart

{"title":"连续时间随机梯度下降的收敛性及其在线性深度神经网络中的应用","authors":"Gabor Lugosi, Eulalia Nualart","doi":"arxiv-2409.07401","DOIUrl":null,"url":null,"abstract":"We study a continuous-time approximation of the stochastic gradient descent\nprocess for minimizing the expected loss in learning problems. The main results\nestablish general sufficient conditions for the convergence, extending the\nresults of Chatterjee (2022) established for (nonstochastic) gradient descent.\nWe show how the main result can be applied to the case of overparametrized\nlinear neural network training.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"16 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Convergence of continuous-time stochastic gradient descent with applications to linear deep neural networks\",\"authors\":\"Gabor Lugosi, Eulalia Nualart\",\"doi\":\"arxiv-2409.07401\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We study a continuous-time approximation of the stochastic gradient descent\\nprocess for minimizing the expected loss in learning problems. The main results\\nestablish general sufficient conditions for the convergence, extending the\\nresults of Chatterjee (2022) established for (nonstochastic) gradient descent.\\nWe show how the main result can be applied to the case of overparametrized\\nlinear neural network training.\",\"PeriodicalId\":501340,\"journal\":{\"name\":\"arXiv - STAT - Machine Learning\",\"volume\":\"16 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - STAT - Machine Learning\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.07401\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - STAT - Machine Learning","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.07401","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

摘要

我们研究了随机梯度下降过程的连续时间近似值，用于最小化学习问题中的预期损失。主要结果建立了收敛的一般充分条件，扩展了 Chatterjee (2022) 为（非随机）梯度下降建立的结果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

Convergence of continuous-time stochastic gradient descent with applications to linear deep neural networks

We study a continuous-time approximation of the stochastic gradient descent process for minimizing the expected loss in learning problems. The main results establish general sufficient conditions for the convergence, extending the results of Chatterjee (2022) established for (nonstochastic) gradient descent. We show how the main result can be applied to the case of overparametrized linear neural network training.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

arXiv - STAT - Machine Learning

自引率

0.00%

发文量

期刊最新文献

Fitting Multilevel Factor Models Cartan moving frames and the data manifolds Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks Recurrent Interpolants for Probabilistic Time Series Prediction PieClam: A Universal Graph Autoencoder Based on Overlapping Inclusive and Exclusive Communities