Pub Date : 2025-03-27DOI: 10.1016/j.acha.2025.101765
Len Spek , Tjeerd Jan Heeringa , Felix Schwenninger , Christoph Brune
Reproducing Kernel Hilbert spaces (RKHS) have been a very successful tool in various areas of machine learning. Recently, Barron spaces have been used to prove bounds on the generalisation error for neural networks. Unfortunately, Barron spaces cannot be understood in terms of RKHS due to the strong nonlinear coupling of the weights. This can be solved by using the more general Reproducing Kernel Banach spaces (RKBS). We show that these Barron spaces belong to a class of integral RKBS. This class can also be understood as an infinite union of RKHS spaces. Furthermore, we show that the dual space of such RKBSs, is again an RKBS where the roles of the data and parameters are interchanged, forming an adjoint pair of RKBSs including a reproducing kernel. This allows us to construct the saddle point problem for neural networks, which can be used in the whole field of primal-dual optimisation.
{"title":"Duality for neural networks through Reproducing Kernel Banach Spaces","authors":"Len Spek , Tjeerd Jan Heeringa , Felix Schwenninger , Christoph Brune","doi":"10.1016/j.acha.2025.101765","DOIUrl":"10.1016/j.acha.2025.101765","url":null,"abstract":"<div><div>Reproducing Kernel Hilbert spaces (RKHS) have been a very successful tool in various areas of machine learning. Recently, Barron spaces have been used to prove bounds on the generalisation error for neural networks. Unfortunately, Barron spaces cannot be understood in terms of RKHS due to the strong nonlinear coupling of the weights. This can be solved by using the more general Reproducing Kernel Banach spaces (RKBS). We show that these Barron spaces belong to a class of integral RKBS. This class can also be understood as an infinite union of RKHS spaces. Furthermore, we show that the dual space of such RKBSs, is again an RKBS where the roles of the data and parameters are interchanged, forming an adjoint pair of RKBSs including a reproducing kernel. This allows us to construct the saddle point problem for neural networks, which can be used in the whole field of primal-dual optimisation.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"78 ","pages":"Article 101765"},"PeriodicalIF":2.6,"publicationDate":"2025-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143869805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-25DOI: 10.1016/j.acha.2025.101764
Michael Unser, Alexis Goujon, Stanislas Ducotterd
We present a general variational framework for the training of freeform nonlinearities in layered computational architectures subject to some slope constraints. The regularization that we add to the traditional training loss penalizes the second-order total variation of each trainable activation. The slope constraints allow us to impose properties such as 1-Lipschitz stability, firm non-expansiveness, and monotonicity/invertibility. These properties are crucial to ensure the proper functioning of certain classes of signal-processing algorithms (e.g., plug-and-play schemes, unrolled proximal gradient, invertible flows). We prove that the global optimum of the stated constrained-optimization problem is achieved with nonlinearities that are adaptive nonuniform linear splines. We then show how to solve the resulting function-optimization problem numerically by representing the nonlinearities in a suitable (nonuniform) B-spline basis. Finally, we illustrate the use of our framework with the data-driven design of (weakly) convex regularizers for the denoising of images and the resolution of inverse problems.
{"title":"Controlled learning of pointwise nonlinearities in neural-network-like architectures","authors":"Michael Unser, Alexis Goujon, Stanislas Ducotterd","doi":"10.1016/j.acha.2025.101764","DOIUrl":"10.1016/j.acha.2025.101764","url":null,"abstract":"<div><div>We present a general variational framework for the training of freeform nonlinearities in layered computational architectures subject to some slope constraints. The regularization that we add to the traditional training loss penalizes the second-order total variation of each trainable activation. The slope constraints allow us to impose properties such as 1-Lipschitz stability, firm non-expansiveness, and monotonicity/invertibility. These properties are crucial to ensure the proper functioning of certain classes of signal-processing algorithms (e.g., plug-and-play schemes, unrolled proximal gradient, invertible flows). We prove that the global optimum of the stated constrained-optimization problem is achieved with nonlinearities that are adaptive nonuniform linear splines. We then show how to solve the resulting function-optimization problem numerically by representing the nonlinearities in a suitable (nonuniform) B-spline basis. Finally, we illustrate the use of our framework with the data-driven design of (weakly) convex regularizers for the denoising of images and the resolution of inverse problems.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101764"},"PeriodicalIF":2.6,"publicationDate":"2025-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143738438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-24DOI: 10.1016/j.acha.2025.101763
Holger Boche , Adalbert Fono , Gitta Kutyniok
Deep learning still has drawbacks regarding trustworthiness, which describes a comprehensible, fair, safe, and reliable method. To mitigate the potential risk of AI, clear obligations associated with trustworthiness have been proposed via regulatory guidelines, e.g., in the European AI Act. Therefore, a central question is to what extent trustworthy deep learning can be realized. Establishing the described properties constituting trustworthiness requires that the factors influencing an algorithmic computation can be retraced, i.e., the algorithmic implementation is transparent. Motivated by the observation that the current evolution of deep learning models necessitates a change in computing technology, we derive a mathematical framework that enables us to analyze whether a transparent implementation in a computing model is feasible. The core idea is to formalize and subsequently relate the properties of a transparent algorithmic implementation to the mathematical model of the computing platform, thereby establishing verifiable criteria.
We exemplarily apply our trustworthiness framework to analyze deep learning approaches for inverse problems in digital and analog computing models represented by Turing and Blum-Shub-Smale machines, respectively. Based on previous results, we find that Blum-Shub-Smale machines have the potential to establish trustworthy solvers for inverse problems under fairly general conditions, whereas Turing machines cannot guarantee trustworthiness to the same degree.
{"title":"Mathematical algorithm design for deep learning under societal and judicial constraints: The algorithmic transparency requirement","authors":"Holger Boche , Adalbert Fono , Gitta Kutyniok","doi":"10.1016/j.acha.2025.101763","DOIUrl":"10.1016/j.acha.2025.101763","url":null,"abstract":"<div><div>Deep learning still has drawbacks regarding trustworthiness, which describes a comprehensible, fair, safe, and reliable method. To mitigate the potential risk of AI, clear obligations associated with trustworthiness have been proposed via regulatory guidelines, e.g., in the European AI Act. Therefore, a central question is to what extent trustworthy deep learning can be realized. Establishing the described properties constituting trustworthiness requires that the factors influencing an algorithmic computation can be retraced, i.e., the algorithmic implementation is transparent. Motivated by the observation that the current evolution of deep learning models necessitates a change in computing technology, we derive a mathematical framework that enables us to analyze whether a transparent implementation in a computing model is feasible. The core idea is to formalize and subsequently relate the properties of a transparent algorithmic implementation to the mathematical model of the computing platform, thereby establishing verifiable criteria.</div><div>We exemplarily apply our trustworthiness framework to analyze deep learning approaches for inverse problems in digital and analog computing models represented by Turing and Blum-Shub-Smale machines, respectively. Based on previous results, we find that Blum-Shub-Smale machines have the potential to establish trustworthy solvers for inverse problems under fairly general conditions, whereas Turing machines cannot guarantee trustworthiness to the same degree.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101763"},"PeriodicalIF":2.6,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143696911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-21DOI: 10.1016/j.acha.2025.101762
Thomas Allard, Helmut Bölcskei
We derive a precise general relation between the entropy of a compact operator and its eigenvalues. It is then shown how this result along with the underlying philosophy can be applied to improve substantially on the best known characterizations of the entropy of the Landau-Pollak-Slepian operator and the metric entropy of unit balls in Sobolev spaces.
{"title":"Entropy of compact operators with applications to Landau-Pollak-Slepian theory and Sobolev spaces","authors":"Thomas Allard, Helmut Bölcskei","doi":"10.1016/j.acha.2025.101762","DOIUrl":"10.1016/j.acha.2025.101762","url":null,"abstract":"<div><div>We derive a precise general relation between the entropy of a compact operator and its eigenvalues. It is then shown how this result along with the underlying philosophy can be applied to improve substantially on the best known characterizations of the entropy of the Landau-Pollak-Slepian operator and the metric entropy of unit balls in Sobolev spaces.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101762"},"PeriodicalIF":2.6,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143746775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-20DOI: 10.1016/j.acha.2025.101761
Lutz Kämmerer
For an arbitrary given span of high dimensional multivariate Chebyshev polynomials, an approach to construct spatial discretizations is presented, i.e., the construction of a sampling set that allows for the unique reconstruction of each polynomial of this span.
The approach presented here combines three different types of efficiency. First, the construction of a spatial discretization should be computationally efficient with respect to the dimension of the span of the Chebyshev polynomials. Second, the constructed discretization should be sample efficient, i.e., the number of sampling nodes within the constructed discretization should be reasonably low. Third, there should be an efficient algorithm for the unique reconstruction of a polynomial from given sampling values at the sampling nodes of the discretization.
The first two mentioned types of efficiency are also present in constructions based on random sampling nodes, but the lack of structure here causes the inefficiency of the reconstruction method. Our approach uses a combination of cosine transformed rank-1 lattices whose structure allows for applications of univariate fast Fourier transforms for the reconstruction algorithm and is thus a priori efficiently realizable.
Besides the theoretical estimates of numbers of sampling nodes and failure probabilities due to a random draw of the used lattices, we present several improvements of the basic design approach that significantly increases its practical applicability. Numerical tests, which discretize spans of multivariate Chebyshev polynomials depending on up to more than 50 spatial variables, corroborate the theoretical results and the significance of the improvements.
{"title":"An efficient spatial discretization of spans of multivariate Chebyshev polynomials","authors":"Lutz Kämmerer","doi":"10.1016/j.acha.2025.101761","DOIUrl":"10.1016/j.acha.2025.101761","url":null,"abstract":"<div><div>For an arbitrary given span of high dimensional multivariate Chebyshev polynomials, an approach to construct spatial discretizations is presented, i.e., the construction of a sampling set that allows for the unique reconstruction of each polynomial of this span.</div><div>The approach presented here combines three different types of efficiency. First, the construction of a spatial discretization should be computationally efficient with respect to the dimension of the span of the Chebyshev polynomials. Second, the constructed discretization should be sample efficient, i.e., the number of sampling nodes within the constructed discretization should be reasonably low. Third, there should be an efficient algorithm for the unique reconstruction of a polynomial from given sampling values at the sampling nodes of the discretization.</div><div>The first two mentioned types of efficiency are also present in constructions based on random sampling nodes, but the lack of structure here causes the inefficiency of the reconstruction method. Our approach uses a combination of cosine transformed rank-1 lattices whose structure allows for applications of univariate fast Fourier transforms for the reconstruction algorithm and is thus a priori efficiently realizable.</div><div>Besides the theoretical estimates of numbers of sampling nodes and failure probabilities due to a random draw of the used lattices, we present several improvements of the basic design approach that significantly increases its practical applicability. Numerical tests, which discretize spans of multivariate Chebyshev polynomials depending on up to more than 50 spatial variables, corroborate the theoretical results and the significance of the improvements.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101761"},"PeriodicalIF":2.6,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143697625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-20DOI: 10.1016/j.acha.2025.101760
Yu Ping Wang , Yan-Hsiou Cheng
In this paper, we study direct and inverse problems for Dirac systems with complex-valued potentials on p-star-shaped graphs. More precisely, we firstly obtain sharp 2-term asymptotics of the corresponding eigenvalues. We then formulate and address a Horváth-type theorem, specifically, if the potentials on edges of the p-star-shaped graph are predetermined, we demonstrate that the remaining potential on can be uniquely determined by part of its eigenvalues and the given remaining potential on , , under certain conditions.
{"title":"An inverse problem for Dirac systems on p-star-shaped graphs","authors":"Yu Ping Wang , Yan-Hsiou Cheng","doi":"10.1016/j.acha.2025.101760","DOIUrl":"10.1016/j.acha.2025.101760","url":null,"abstract":"<div><div>In this paper, we study direct and inverse problems for Dirac systems with complex-valued potentials on <em>p</em>-star-shaped graphs. More precisely, we firstly obtain sharp 2-term asymptotics of the corresponding eigenvalues. We then formulate and address a Horváth-type theorem, specifically, if the potentials on <span><math><mi>p</mi><mo>−</mo><mn>1</mn></math></span> edges of the <em>p</em>-star-shaped graph are predetermined, we demonstrate that the remaining potential on <span><math><mo>[</mo><mn>0</mn><mo>,</mo><mi>π</mi><mo>]</mo></math></span> can be uniquely determined by part of its eigenvalues and the given remaining potential on <span><math><mo>[</mo><mi>a</mi><mo>,</mo><mi>π</mi><mo>]</mo></math></span>, <span><math><mn>0</mn><mo><</mo><mi>a</mi><mo>≤</mo><mi>π</mi></math></span>, under certain conditions.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101760"},"PeriodicalIF":2.6,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143679153","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-14DOI: 10.1016/j.acha.2025.101759
Jiuyang Liang , Zhenli Xu , Qi Zhou
This paper provides an error estimate for the u-series method of the Coulomb interaction in molecular dynamics simulations. We show that the number of truncated Gaussians M in the u-series and the base of interpolation nodes b in the bilateral serial approximation are two key parameters for the algorithm accuracy, and that the errors converge as for the energy and for the force. Error bounds due to numerical quadrature and cutoff in both the electrostatic energy and forces are obtained. Closed-form formulae are also provided, which are useful in the parameter setup for simulations under a given accuracy. The results are verified by analyzing the errors of two practical systems.
{"title":"Error estimate of the u-series method for molecular dynamics simulations","authors":"Jiuyang Liang , Zhenli Xu , Qi Zhou","doi":"10.1016/j.acha.2025.101759","DOIUrl":"10.1016/j.acha.2025.101759","url":null,"abstract":"<div><div>This paper provides an error estimate for the u-series method of the Coulomb interaction in molecular dynamics simulations. We show that the number of truncated Gaussians <em>M</em> in the u-series and the base of interpolation nodes <em>b</em> in the bilateral serial approximation are two key parameters for the algorithm accuracy, and that the errors converge as <span><math><mi>O</mi><mo>(</mo><msup><mrow><mi>b</mi></mrow><mrow><mo>−</mo><mi>M</mi></mrow></msup><mo>)</mo></math></span> for the energy and <span><math><mi>O</mi><mo>(</mo><msup><mrow><mi>b</mi></mrow><mrow><mo>−</mo><mn>3</mn><mi>M</mi></mrow></msup><mo>)</mo></math></span> for the force. Error bounds due to numerical quadrature and cutoff in both the electrostatic energy and forces are obtained. Closed-form formulae are also provided, which are useful in the parameter setup for simulations under a given accuracy. The results are verified by analyzing the errors of two practical systems.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101759"},"PeriodicalIF":2.6,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143675537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-02-21DOI: 10.1016/j.acha.2025.101756
Mahya Ghandehari , Georgi S. Medvedev
The W-random graphs provide a flexible framework for modeling large random networks. Using the Large Deviation Principle (LDP) for W-random graphs from [19], we prove the LDP for the corresponding class of random symmetric Hilbert-Schmidt integral operators. Our main result describes how the eigenvalues and the eigenspaces of the integral operator are affected by large deviations in the underlying random graphon. To prove the LDP, we demonstrate continuous dependence of the spectral measures associated with integral operators on the corresponding graphons and use the Contraction Principle. To illustrate our results, we obtain leading order asymptotics of the eigenvalues of small-world and bipartite random graphs conditioned on atypical edge counts. These examples suggest several representative scenarios of how the eigenvalues and the eigenspaces are affected by large deviations. We discuss the implications of these observations for bifurcation analysis of Dynamical Systems and Graph Signal Processing.
W 随机图为大型随机网络建模提供了一个灵活的框架。利用文献[19]中针对 W-随机图的大偏差原理（LDP），我们证明了相应类别的随机对称希尔伯特-施密特积分算子的大偏差原理。我们的主要结果描述了积分算子的特征值和特征空间如何受到底层随机图元大偏差的影响。为了证明 LDP，我们证明了与积分算子相关的谱度量对相应图元的连续依赖性，并使用了收缩原理。为了说明我们的结果，我们获得了以非典型边数为条件的小世界和双方形随机图特征值的前阶渐近性。这些例子说明了特征值和特征空间如何受到大偏差的影响。我们将讨论这些观察结果对动态系统分岔分析和图信号处理的影响。
{"title":"The Large Deviation Principle for W-random spectral measures","authors":"Mahya Ghandehari , Georgi S. Medvedev","doi":"10.1016/j.acha.2025.101756","DOIUrl":"10.1016/j.acha.2025.101756","url":null,"abstract":"<div><div>The <em>W</em>-random graphs provide a flexible framework for modeling large random networks. Using the Large Deviation Principle (LDP) for <em>W</em>-random graphs from <span><span>[19]</span></span>, we prove the LDP for the corresponding class of random symmetric Hilbert-Schmidt integral operators. Our main result describes how the eigenvalues and the eigenspaces of the integral operator are affected by large deviations in the underlying random graphon. To prove the LDP, we demonstrate continuous dependence of the spectral measures associated with integral operators on the corresponding graphons and use the Contraction Principle. To illustrate our results, we obtain leading order asymptotics of the eigenvalues of small-world and bipartite random graphs conditioned on atypical edge counts. These examples suggest several representative scenarios of how the eigenvalues and the eigenspaces are affected by large deviations. We discuss the implications of these observations for bifurcation analysis of Dynamical Systems and Graph Signal Processing.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101756"},"PeriodicalIF":2.6,"publicationDate":"2025-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143479228","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-02-17DOI: 10.1016/j.acha.2025.101749
Massimo Fornasier , Timo Klock , Marco Mondelli , Michael Rauchensteiner
The identification of the parameters of a neural network from finite samples of input-output pairs is often referred to as the teacher-student model, and this model has represented a popular framework for understanding training and generalization. Even if the problem is NP-complete in the worst case, a rapidly growing literature – after adding suitable distributional assumptions – has established finite sample identification of two-layer networks with a number of neurons , D being the input dimension. For the range the problem becomes harder, and truly little is known for networks parametrized by biases as well. This paper fills the gap by providing efficient algorithms and rigorous theoretical guarantees of finite sample identification for such wider shallow networks with biases. Our approach is based on a two-step pipeline: first, we recover the direction of the weights, by exploiting second order information; next, we identify the signs by suitable algebraic evaluations, and we recover the biases by empirical risk minimization via gradient descent. Numerical results demonstrate the effectiveness of our approach.
{"title":"Efficient identification of wide shallow neural networks with biases","authors":"Massimo Fornasier , Timo Klock , Marco Mondelli , Michael Rauchensteiner","doi":"10.1016/j.acha.2025.101749","DOIUrl":"10.1016/j.acha.2025.101749","url":null,"abstract":"<div><div>The identification of the parameters of a neural network from finite samples of input-output pairs is often referred to as the <em>teacher-student model</em>, and this model has represented a popular framework for understanding training and generalization. Even if the problem is NP-complete in the worst case, a rapidly growing literature – after adding suitable distributional assumptions – has established finite sample identification of two-layer networks with a number of neurons <span><math><mi>m</mi><mo>=</mo><mi>O</mi><mo>(</mo><mi>D</mi><mo>)</mo></math></span>, <em>D</em> being the input dimension. For the range <span><math><mi>D</mi><mo><</mo><mi>m</mi><mo><</mo><msup><mrow><mi>D</mi></mrow><mrow><mn>2</mn></mrow></msup></math></span> the problem becomes harder, and truly little is known for networks parametrized by biases as well. This paper fills the gap by providing efficient algorithms and rigorous theoretical guarantees of finite sample identification for such wider shallow networks with biases. Our approach is based on a two-step pipeline: first, we recover the direction of the weights, by exploiting second order information; next, we identify the signs by suitable algebraic evaluations, and we recover the biases by empirical risk minimization via gradient descent. Numerical results demonstrate the effectiveness of our approach.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101749"},"PeriodicalIF":2.6,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143429952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-10DOI: 10.1016/j.acha.2025.101748
Ilya Krishtal, Brendan Miller
We extend the classical Kadec theorem for systems of exponential functions on an interval to frames and atomic decompositions formed by sampling an orbit of a vector under an isometric group representation.
{"title":"Kadec-type theorems for sampled group orbits","authors":"Ilya Krishtal, Brendan Miller","doi":"10.1016/j.acha.2025.101748","DOIUrl":"10.1016/j.acha.2025.101748","url":null,"abstract":"<div><div>We extend the classical Kadec <span><math><mfrac><mrow><mn>1</mn></mrow><mrow><mn>4</mn></mrow></mfrac></math></span> theorem for systems of exponential functions on an interval to frames and atomic decompositions formed by sampling an orbit of a vector under an isometric group representation.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"76 ","pages":"Article 101748"},"PeriodicalIF":2.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142990415","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号