Pub Date : 2023-11-08DOI: 10.1016/j.acha.2023.101604
Hartmut Führ , Irina Shafkulovska
We study the mapping properties of metaplectic operators on modulation spaces of the type . Our main result is a full characterization of the pairs for which the operator is (i) well-defined, (ii) bounded. It turns out that these two properties are equivalent, and they entail that is a Banach space automorphism. For polynomially bounded weight functions, we provide a simple sufficient criterion to determine whether the well-definedness (boundedness) of transfers to .
{"title":"The metaplectic action on modulation spaces","authors":"Hartmut Führ , Irina Shafkulovska","doi":"10.1016/j.acha.2023.101604","DOIUrl":"10.1016/j.acha.2023.101604","url":null,"abstract":"<div><p>We study the mapping properties of metaplectic operators <span><math><mover><mrow><mi>S</mi></mrow><mrow><mo>ˆ</mo></mrow></mover><mo>∈</mo><mrow><mi>Mp</mi></mrow><mo>(</mo><mn>2</mn><mi>d</mi><mo>,</mo><mi>R</mi><mo>)</mo></math></span> on modulation spaces of the type <span><math><msubsup><mrow><mi>M</mi></mrow><mrow><mi>m</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msubsup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span>. Our main result is a full characterization of the pairs <span><math><mo>(</mo><mover><mrow><mi>S</mi></mrow><mrow><mo>ˆ</mo></mrow></mover><mo>,</mo><msup><mrow><mi>M</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo><mo>)</mo></math></span> for which the operator <span><math><mover><mrow><mi>S</mi></mrow><mrow><mo>ˆ</mo></mrow></mover><mo>:</mo><msup><mrow><mi>M</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo><mo>→</mo><msup><mrow><mi>M</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span> is <em>(i)</em> well-defined, <em>(ii)</em> bounded. It turns out that these two properties are equivalent, and they entail that <span><math><mover><mrow><mi>S</mi></mrow><mrow><mo>ˆ</mo></mrow></mover></math></span> is a Banach space automorphism. For polynomially bounded weight functions, we provide a simple sufficient criterion to determine whether the well-definedness (boundedness) of <span><math><mover><mrow><mi>S</mi></mrow><mrow><mo>ˆ</mo></mrow></mover><mo>:</mo><msup><mrow><mi>M</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo><mo>→</mo><msup><mrow><mi>M</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span> transfers to <span><math><mover><mrow><mi>S</mi></mrow><mrow><mo>ˆ</mo></mrow></mover><mo>:</mo><msubsup><mrow><mi>M</mi></mrow><mrow><mi>m</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msubsup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo><mo>→</mo><msubsup><mrow><mi>M</mi></mrow><mrow><mi>m</mi></mrow><mrow><mi>p</mi><mo>,</mo><mi>q</mi></mrow></msubsup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span>.</p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101604"},"PeriodicalIF":2.5,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S106352032300091X/pdfft?md5=0769848d44f7ddda38eab0321ccdd78e&pid=1-s2.0-S106352032300091X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72364681","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-29DOI: 10.1016/j.acha.2023.101607
Götz Pfander , Shauna Revay , David Walnut
For a partition of into intervals we prove the existence of a partition of into such that the complex exponential functions with frequencies in form a Riesz basis for , and furthermore, that for any , the exponential functions with frequencies in form a Riesz basis for for any interval I with length . The construction extends to infinite partitions of , but with size limitations on the subsets ; it combines the ergodic properties of subsequences of known as Beatty-Fraenkel sequences with a theorem of Avdonin on exponential Riesz bases.
{"title":"Exponential bases for partitions of intervals","authors":"Götz Pfander , Shauna Revay , David Walnut","doi":"10.1016/j.acha.2023.101607","DOIUrl":"10.1016/j.acha.2023.101607","url":null,"abstract":"<div><p>For a partition of <span><math><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></math></span> into intervals <span><math><msub><mrow><mi>I</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><mo>…</mo><mo>,</mo><msub><mrow><mi>I</mi></mrow><mrow><mi>n</mi></mrow></msub></math></span> we prove the existence of a partition of <span><math><mi>Z</mi></math></span> into <span><math><msub><mrow><mi>Λ</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><mo>…</mo><mo>,</mo><msub><mrow><mi>Λ</mi></mrow><mrow><mi>n</mi></mrow></msub></math></span> such that the complex exponential functions with frequencies in <span><math><msub><mrow><mi>Λ</mi></mrow><mrow><mi>k</mi></mrow></msub></math></span> form a Riesz basis for <span><math><msup><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msup><mo>(</mo><msub><mrow><mi>I</mi></mrow><mrow><mi>k</mi></mrow></msub><mo>)</mo></math></span>, and furthermore, that for any <span><math><mi>J</mi><mo>⊆</mo><mo>{</mo><mn>1</mn><mo>,</mo><mspace></mspace><mn>2</mn><mo>,</mo><mspace></mspace><mo>…</mo><mo>,</mo><mspace></mspace><mi>n</mi><mo>}</mo></math></span>, the exponential functions with frequencies in <span><math><msub><mrow><mo>⋃</mo></mrow><mrow><mi>j</mi><mo>∈</mo><mi>J</mi></mrow></msub><msub><mrow><mi>Λ</mi></mrow><mrow><mi>j</mi></mrow></msub></math></span> form a Riesz basis for <span><math><msup><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msup><mo>(</mo><mi>I</mi><mo>)</mo></math></span> for any interval <em>I</em> with length <span><math><mo>|</mo><mi>I</mi><mo>|</mo><mo>=</mo><msub><mrow><mo>∑</mo></mrow><mrow><mi>j</mi><mo>∈</mo><mi>J</mi></mrow></msub><mo>|</mo><msub><mrow><mi>I</mi></mrow><mrow><mi>j</mi></mrow></msub><mo>|</mo></math></span>. The construction extends to infinite partitions of <span><math><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></math></span>, but with size limitations on the subsets <span><math><mi>J</mi><mo>⊆</mo><mi>Z</mi></math></span>; it combines the ergodic properties of subsequences of <span><math><mi>Z</mi></math></span> known as Beatty-Fraenkel sequences with a theorem of Avdonin on exponential Riesz bases.</p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101607"},"PeriodicalIF":2.5,"publicationDate":"2023-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71514551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-20DOI: 10.1016/j.acha.2023.101605
Cornelia Schneider , Jan Vybíral
We consider the trigonometric-like system of piecewise linear functions introduced recently by Daubechies, DeVore, Foucart, Hanin, and Petrova. We provide an alternative proof that this system forms a Riesz basis of based on the Gershgorin theorem. We also generalize this system to higher dimensions by a construction, which avoids using (tensor) products. As a consequence, the functions from the new Riesz basis of can be easily represented by neural networks. Moreover, the Riesz constants of this system are independent of d, making it an attractive building block regarding future multivariate analysis of neural networks.
{"title":"A multivariate Riesz basis of ReLU neural networks","authors":"Cornelia Schneider , Jan Vybíral","doi":"10.1016/j.acha.2023.101605","DOIUrl":"10.1016/j.acha.2023.101605","url":null,"abstract":"<div><p><span>We consider the trigonometric-like system of piecewise linear<span> functions introduced recently by Daubechies, DeVore, Foucart, Hanin, and Petrova. We provide an alternative proof that this system forms a Riesz basis of </span></span><span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>(</mo><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo><mo>)</mo></math></span><span> based on the Gershgorin theorem. We also generalize this system to higher dimensions </span><span><math><mi>d</mi><mo>></mo><mn>1</mn></math></span> by a construction, which avoids using (tensor) products. As a consequence, the functions from the new Riesz basis of <span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>(</mo><msup><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span><span> can be easily represented by neural networks. Moreover, the Riesz constants of this system are independent of </span><em>d</em><span>, making it an attractive building block regarding future multivariate analysis of neural networks.</span></p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101605"},"PeriodicalIF":2.5,"publicationDate":"2023-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71518637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-13DOI: 10.1016/j.acha.2023.101603
Anton Kutsenko , Sergey Danilov , Stephan Juricke , Marcel Oliver
We discuss relations between the expansion coefficients of a discrete random field when analyzed with respect to different hierarchical bases. Our main focus is on the comparison of two such systems: the Walsh–Rademacher basis and the trigonometric Fourier basis. In general, spectra computed with respect to one basis will look different in the other. In this paper, we prove that, in a statistical sense, the rate of spectral decay computed in one basis can be translated to the other. We further provide explicit expressions for this translation on quadrilateral meshes. The results are illustrated with numerical examples for deterministic and random fields.
{"title":"On the relation between Fourier and Walsh–Rademacher spectra for random fields","authors":"Anton Kutsenko , Sergey Danilov , Stephan Juricke , Marcel Oliver","doi":"10.1016/j.acha.2023.101603","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101603","url":null,"abstract":"<div><p>We discuss relations between the expansion coefficients of a discrete random field when analyzed with respect to different hierarchical bases. Our main focus is on the comparison of two such systems: the Walsh–Rademacher basis and the trigonometric Fourier basis. In general, spectra computed with respect to one basis will look different in the other. In this paper, we prove that, in a statistical sense, the rate of spectral decay computed in one basis can be translated to the other. We further provide explicit expressions for this translation on quadrilateral meshes. The results are illustrated with numerical examples for deterministic and random fields.</p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101603"},"PeriodicalIF":2.5,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49791518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-12DOI: 10.1016/j.acha.2023.101602
Hao Liu , Alex Havrilla , Rongjie Lai , Wenjing Liao
Autoencoders have demonstrated remarkable success in learning low-dimensional latent features of high-dimensional data across various applications. Assuming that data are sampled near a low-dimensional manifold, we employ chart autoencoders, which encode data into low-dimensional latent features on a collection of charts, preserving the topology and geometry of the data manifold. Our paper establishes statistical guarantees on the generalization error of chart autoencoders, and we demonstrate their denoising capabilities by considering n noisy training samples, along with their noise-free counterparts, on a d-dimensional manifold. By training autoencoders, we show that chart autoencoders can effectively denoise the input data with normal noise. We prove that, under proper network architectures, chart autoencoders achieve a squared generalization error in the order of , which depends on the intrinsic dimension of the manifold and only weakly depends on the ambient dimension and noise level. We further extend our theory on data with noise containing both normal and tangential components, where chart autoencoders still exhibit a denoising effect for the normal component. As a special case, our theory also applies to classical autoencoders, as long as the data manifold has a global parametrization. Our results provide a solid theoretical foundation for the effectiveness of autoencoders, which is further validated through several numerical experiments.
{"title":"Deep nonparametric estimation of intrinsic data structures by chart autoencoders: Generalization error and robustness","authors":"Hao Liu , Alex Havrilla , Rongjie Lai , Wenjing Liao","doi":"10.1016/j.acha.2023.101602","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101602","url":null,"abstract":"<div><p>Autoencoders have demonstrated remarkable success in learning low-dimensional latent features of high-dimensional data across various applications. Assuming that data are sampled near a low-dimensional manifold, we employ chart autoencoders, which encode data into low-dimensional latent features on a collection of charts, preserving the topology and geometry of the data manifold. Our paper establishes statistical guarantees on the generalization error of chart autoencoders, and we demonstrate their denoising capabilities by considering <em>n</em> noisy training samples, along with their noise-free counterparts, on a <em>d</em>-dimensional manifold. By training autoencoders, we show that chart autoencoders can effectively denoise the input data with normal noise. We prove that, under proper network architectures, chart autoencoders achieve a squared generalization error in the order of <span><math><msup><mrow><mi>n</mi></mrow><mrow><mo>−</mo><mfrac><mrow><mn>2</mn></mrow><mrow><mi>d</mi><mo>+</mo><mn>2</mn></mrow></mfrac></mrow></msup><msup><mrow><mi>log</mi></mrow><mrow><mn>4</mn></mrow></msup><mo></mo><mi>n</mi></math></span>, which depends on the intrinsic dimension of the manifold and only weakly depends on the ambient dimension and noise level. We further extend our theory on data with noise containing both normal and tangential components, where chart autoencoders still exhibit a denoising effect for the normal component. As a special case, our theory also applies to classical autoencoders, as long as the data manifold has a global parametrization. Our results provide a solid theoretical foundation for the effectiveness of autoencoders, which is further validated through several numerical experiments.</p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101602"},"PeriodicalIF":2.5,"publicationDate":"2023-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49778386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-10DOI: 10.1016/j.acha.2023.101600
M.M. Castro , F.A. Grünbaum , I. Zurrián
The “time-and-band limiting” commutative property was found and exploited by D. Slepian, H. Landau and H. Pollak at Bell Labs in the 1960's, and independently by M. Mehta and later by C. Tracy and H. Widom in Random matrix theory. The property in question is the existence of local operators with simple spectrum that commute with naturally appearing global ones.
Here we give a general result that insures the existence of a commuting differential operator for a given family of exceptional orthogonal polynomials satisfying the “bispectral property”. As a main tool we go beyond bispectrality and make use of the notion of Fourier Algebras associated to the given sequence of exceptional polynomials. We illustrate this result with two examples, of Hermite and Laguerre type, exhibiting also a nice Perline's form for the commuting differential operator.
{"title":"Time and band limiting for exceptional polynomials","authors":"M.M. Castro , F.A. Grünbaum , I. Zurrián","doi":"10.1016/j.acha.2023.101600","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101600","url":null,"abstract":"<div><p><span>The “time-and-band limiting” commutative property was found and exploited by D. Slepian, H. Landau and H. Pollak at Bell Labs in the 1960's, and independently by M. Mehta and later by C. Tracy and H. Widom in </span>Random matrix theory. The property in question is the existence of local operators with simple spectrum that commute with naturally appearing global ones.</p><p>Here we give a general result that insures the existence of a commuting differential operator<span> for a given family of exceptional orthogonal polynomials satisfying the “bispectral property”. As a main tool we go beyond bispectrality and make use of the notion of Fourier Algebras associated to the given sequence of exceptional polynomials. We illustrate this result with two examples, of Hermite and Laguerre type, exhibiting also a nice Perline's form for the commuting differential operator.</span></p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101600"},"PeriodicalIF":2.5,"publicationDate":"2023-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49778389","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-06DOI: 10.1016/j.acha.2023.101601
Yucong Liu , Simiao Jiao , Lek-Heng Lim
<div><p>Any matrix <em>A</em> has an LU decomposition up to a row or column permutation. Less well-known is the fact that it has a ‘Toeplitz decomposition’ <span><math><mi>A</mi><mo>=</mo><msub><mrow><mi>T</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>T</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>⋯</mo><msub><mrow><mi>T</mi></mrow><mrow><mi>r</mi></mrow></msub></math></span> where <span><math><msub><mrow><mi>T</mi></mrow><mrow><mi>i</mi></mrow></msub></math></span>'s are Toeplitz matrices. We will prove that any continuous function <span><math><mi>f</mi><mo>:</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>n</mi></mrow></msup><mo>→</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>m</mi></mrow></msup></math></span> has an approximation to arbitrary accuracy by a neural network that maps <span><math><mi>x</mi><mo>∈</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>n</mi></mrow></msup></math></span> to <span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>U</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>2</mn></mrow></msub><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>3</mn></mrow></msub><msub><mrow><mi>U</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>⋯</mo><msub><mrow><mi>L</mi></mrow><mrow><mi>r</mi></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>2</mn><mi>r</mi><mo>−</mo><mn>1</mn></mrow></msub><msub><mrow><mi>U</mi></mrow><mrow><mi>r</mi></mrow></msub><mi>x</mi><mo>∈</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>m</mi></mrow></msup></math></span>, i.e., where the weight matrices alternate between lower and upper triangular matrices, <span><math><msub><mrow><mi>σ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>(</mo><mi>x</mi><mo>)</mo><mo>≔</mo><mi>σ</mi><mo>(</mo><mi>x</mi><mo>−</mo><msub><mrow><mi>b</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>)</mo></math></span> for some bias vector <span><math><msub><mrow><mi>b</mi></mrow><mrow><mi>i</mi></mrow></msub></math></span>, and the activation <em>σ</em> may be chosen to be essentially any uniformly continuous nonpolynomial function. The same result also holds with Toeplitz matrices, i.e., <span><math><mi>f</mi><mo>≈</mo><msub><mrow><mi>T</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>T</mi></mrow><mrow><mn>2</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>⋯</mo><msub><mrow><mi>σ</mi></mrow><mrow><mi>r</mi><mo>−</mo><mn>1</mn></mrow></msub><msub><mrow><mi>T</mi></mrow><mrow><mi>r</mi></mrow></msub></math></span> to arbitrary accuracy, and likewise for Hankel matrices. A consequence of our Toeplitz result is a fixed-width universal approximation theorem for convolutional neural networks, which so far have only arbitrary width versions. Since our results apply in particular to the case when <em>f</em> is a general neural network, we ma
{"title":"LU decomposition and Toeplitz decomposition of a neural network","authors":"Yucong Liu , Simiao Jiao , Lek-Heng Lim","doi":"10.1016/j.acha.2023.101601","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101601","url":null,"abstract":"<div><p>Any matrix <em>A</em> has an LU decomposition up to a row or column permutation. Less well-known is the fact that it has a ‘Toeplitz decomposition’ <span><math><mi>A</mi><mo>=</mo><msub><mrow><mi>T</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>T</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>⋯</mo><msub><mrow><mi>T</mi></mrow><mrow><mi>r</mi></mrow></msub></math></span> where <span><math><msub><mrow><mi>T</mi></mrow><mrow><mi>i</mi></mrow></msub></math></span>'s are Toeplitz matrices. We will prove that any continuous function <span><math><mi>f</mi><mo>:</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>n</mi></mrow></msup><mo>→</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>m</mi></mrow></msup></math></span> has an approximation to arbitrary accuracy by a neural network that maps <span><math><mi>x</mi><mo>∈</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>n</mi></mrow></msup></math></span> to <span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>U</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>2</mn></mrow></msub><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>3</mn></mrow></msub><msub><mrow><mi>U</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>⋯</mo><msub><mrow><mi>L</mi></mrow><mrow><mi>r</mi></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>2</mn><mi>r</mi><mo>−</mo><mn>1</mn></mrow></msub><msub><mrow><mi>U</mi></mrow><mrow><mi>r</mi></mrow></msub><mi>x</mi><mo>∈</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>m</mi></mrow></msup></math></span>, i.e., where the weight matrices alternate between lower and upper triangular matrices, <span><math><msub><mrow><mi>σ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>(</mo><mi>x</mi><mo>)</mo><mo>≔</mo><mi>σ</mi><mo>(</mo><mi>x</mi><mo>−</mo><msub><mrow><mi>b</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>)</mo></math></span> for some bias vector <span><math><msub><mrow><mi>b</mi></mrow><mrow><mi>i</mi></mrow></msub></math></span>, and the activation <em>σ</em> may be chosen to be essentially any uniformly continuous nonpolynomial function. The same result also holds with Toeplitz matrices, i.e., <span><math><mi>f</mi><mo>≈</mo><msub><mrow><mi>T</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>1</mn></mrow></msub><msub><mrow><mi>T</mi></mrow><mrow><mn>2</mn></mrow></msub><msub><mrow><mi>σ</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>⋯</mo><msub><mrow><mi>σ</mi></mrow><mrow><mi>r</mi><mo>−</mo><mn>1</mn></mrow></msub><msub><mrow><mi>T</mi></mrow><mrow><mi>r</mi></mrow></msub></math></span> to arbitrary accuracy, and likewise for Hankel matrices. A consequence of our Toeplitz result is a fixed-width universal approximation theorem for convolutional neural networks, which so far have only arbitrary width versions. Since our results apply in particular to the case when <em>f</em> is a general neural network, we ma","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101601"},"PeriodicalIF":2.5,"publicationDate":"2023-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49778387","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-05DOI: 10.1016/j.acha.2023.101596
Peter Balazs, Mitra Shamsabadi, Ali Akbar Arefijamaal, Gilles Chardon
To solve operator equations numerically, matrix representations are employing bases or more recently frames. For finding the numerical solution of operator equations a decomposition in subspaces is needed in many applications. To combine those two approaches, it is necessary to extend the known methods of matrix representation to the utilization of fusion frames. In this paper, we investigate this representation of operators on a Hilbert space with Bessel fusion sequences, fusion frames and fusion Riesz bases. Fusion frames can be considered as a frame-like family of subspaces. Taking the particular property of the duality of fusion frames into account, this allows us to define a matrix representation in a canonical as well as an alternate way, the later being more efficient and well behaved in respect to inversion. We will give the basic definitions and show some structural results, like that the functions assigning the alternate representation to an operator is an algebra homomorphism. We give formulas for pseudo-inverses and the inverses (if existing) of such matrix representations. We apply this idea to Schatten p-class operators. Consequently, we show that tensor products of fusion frames are fusion frames in the space of Hilbert-Schmidt operators. We will show how this can be used for the solution of operator equations and link our approach to the additive Schwarz algorithm. Consequently, we propose some methods for solving an operator equation by iterative methods on the subspaces. Moreover, we implement the alternate Schwarz algorithms employing our perspective and provide small proof-of-concept numerical experiments. Finally, we show the application of this concept to overlapped convolution and the non-standard wavelet representation of operators.
{"title":"Representation of operators using fusion frames","authors":"Peter Balazs, Mitra Shamsabadi, Ali Akbar Arefijamaal, Gilles Chardon","doi":"10.1016/j.acha.2023.101596","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101596","url":null,"abstract":"<div><p><span><span>To solve operator equations numerically, matrix representations are employing bases or more recently frames. For finding the numerical solution of operator equations a decomposition in subspaces is needed in many applications. To combine those two approaches, it is necessary to extend the known methods of matrix representation to the utilization of fusion frames. In this paper, we investigate this representation of operators on a </span>Hilbert space </span><span><math><mi>H</mi></math></span><span><span> with Bessel<span> fusion sequences, fusion frames and fusion Riesz bases. Fusion frames can be considered as a frame-like family of subspaces. Taking the particular property of the duality of fusion frames into account, this allows us to define a matrix representation in a canonical as well as an alternate way, the later being more efficient and well behaved in respect to inversion. We will give the basic definitions and show some structural results, like that the functions assigning the alternate representation to an operator is an algebra </span></span>homomorphism. We give formulas for pseudo-inverses and the inverses (if existing) of such matrix representations. We apply this idea to Schatten </span><em>p</em><span>-class operators. Consequently, we show that tensor products of fusion frames are fusion frames in the space of Hilbert-Schmidt operators. We will show how this can be used for the solution of operator equations and link our approach to the additive Schwarz algorithm. Consequently, we propose some methods for solving an operator equation by iterative methods on the subspaces. Moreover, we implement the alternate Schwarz algorithms employing our perspective and provide small proof-of-concept numerical experiments. Finally, we show the application of this concept to overlapped convolution and the non-standard wavelet representation of operators.</span></p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101596"},"PeriodicalIF":2.5,"publicationDate":"2023-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49778388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-09DOI: 10.1016/j.acha.2023.101593
Ryan Vaughn , Tyrus Berry , Harbir Antil
Given only a finite collection of points sampled from a Riemannian manifold embedded in a Euclidean space, in this paper we propose a new method to numerically solve elliptic and parabolic partial differential equations (PDEs) supplemented with boundary conditions. Since the construction of triangulations on unknown manifolds can be both difficult and expensive, both in terms of computational and data requirements, our goal is to solve these problems without a triangulation. Instead, we rely only on using the sample points to define quadrature formulas on the unknown manifold. Our main tool is the diffusion maps algorithm. We re-analyze this well-known method in a variational sense for manifolds with boundary. Our main result is that the variational diffusion maps graph Laplacian is a consistent estimator of the Dirichlet energy on the manifold. This improves upon previous results and provides a rigorous justification of the well-known relationship between diffusion maps and the Neumann eigenvalue problem. Moreover, using semigeodesic coordinates we derive the first uniform asymptotic expansion of the diffusion maps kernel integral operator for manifolds with boundary. This expansion relies on a novel lemma which relates the extrinsic Euclidean distance to the coordinate norm in a normal collar of the boundary. We then use a recently developed method of estimating the distance to boundary function (notice that the boundary location is assumed to be unknown) to construct a consistent estimator for boundary integrals. Finally, by combining these various estimators, we illustrate how to impose Dirichlet and Neumann conditions for some common PDEs based on the Laplacian. Several numerical examples illustrate our theoretical findings.
{"title":"Diffusion maps for embedded manifolds with boundary with applications to PDEs","authors":"Ryan Vaughn , Tyrus Berry , Harbir Antil","doi":"10.1016/j.acha.2023.101593","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101593","url":null,"abstract":"<div><p><span><span><span>Given only a finite collection of points sampled from a Riemannian manifold embedded in a </span>Euclidean space<span><span>, in this paper we propose a new method to numerically solve elliptic and parabolic partial differential equations (PDEs) supplemented with boundary conditions. Since the construction of triangulations on unknown manifolds can be both difficult and expensive, both in terms of computational and data requirements, our goal is to solve these problems without a triangulation. Instead, we rely only on using the sample points to define </span>quadrature formulas<span> on the unknown manifold. Our main tool is the diffusion maps algorithm. We re-analyze this well-known method in a variational sense for manifolds with boundary. Our main result is that the variational diffusion maps graph Laplacian<span> is a consistent estimator of the Dirichlet energy on the manifold. This improves upon previous results and provides a rigorous justification of the well-known relationship between diffusion maps and the Neumann </span></span></span></span>eigenvalue problem<span>. Moreover, using semigeodesic coordinates we derive the first uniform asymptotic expansion of the diffusion maps kernel integral operator for manifolds with boundary. This expansion relies on a novel lemma which relates the extrinsic </span></span>Euclidean distance to the coordinate norm in a normal collar of the boundary. We then use a recently developed method of estimating the distance to boundary function (notice that the boundary location is assumed to be unknown) to construct a consistent estimator for boundary integrals. Finally, by combining these various estimators, we illustrate how to impose Dirichlet and Neumann conditions for some common PDEs based on the Laplacian. Several numerical examples illustrate our theoretical findings.</p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101593"},"PeriodicalIF":2.5,"publicationDate":"2023-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49778390","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-09DOI: 10.1016/j.acha.2023.101594
Elena Cordero , Gianluca Giacchi
We introduce new frames, called metaplectic Gabor frames, as natural generalizations of Gabor frames in the framework of metaplectic Wigner distributions, cf. [7], [8], [5], [17], [27], [28]. Namely, we develop the theory of metaplectic atoms in a full-general setting and prove an inversion formula for metaplectic Wigner distributions on . Its discretization provides metaplectic Gabor frames.
Next, we deepen the understanding of the so-called shift-invertible metaplectic Wigner distributions, showing that they can be represented, up to chirps, as rescaled short-time Fourier transforms. As an application, we derive a new characterization of modulation and Wiener amalgam spaces. Thus, these metaplectic distributions (and related frames) provide meaningful definitions of local frequencies and can be used to measure effectively the local frequency content of signals.
{"title":"Metaplectic Gabor frames and symplectic analysis of time-frequency spaces","authors":"Elena Cordero , Gianluca Giacchi","doi":"10.1016/j.acha.2023.101594","DOIUrl":"https://doi.org/10.1016/j.acha.2023.101594","url":null,"abstract":"<div><p>We introduce new frames, called <em>metaplectic Gabor frames</em><span>, as natural generalizations of Gabor frames in the framework of metaplectic Wigner distributions, cf. </span><span>[7]</span>, <span>[8]</span>, <span>[5]</span>, <span>[17]</span>, <span>[27]</span>, <span>[28]</span><span>. Namely, we develop the theory of metaplectic atoms in a full-general setting and prove an inversion formula for metaplectic Wigner distributions on </span><span><math><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup></math></span><span>. Its discretization provides metaplectic Gabor frames.</span></p><p>Next, we deepen the understanding of the so-called shift-invertible metaplectic Wigner distributions, showing that they can be represented, up to chirps, as rescaled short-time Fourier transforms. As an application, we derive a new characterization of modulation and Wiener amalgam spaces. Thus, these metaplectic distributions (and related frames) provide meaningful definitions of local frequencies and can be used to measure effectively the local frequency content of signals.</p></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101594"},"PeriodicalIF":2.5,"publicationDate":"2023-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49819289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}