Pub Date : 2024-08-06DOI: 10.1109/TIT.2024.3439408
Ziling Heng;Xiaoru Li;Yansheng Wu;Qi Wang
Linear codes are widely studied in coding theory as they have nice applications in distributed storage, combinatorics, lattices, cryptography and so on. Constructing linear codes with desirable properties is an interesting research topic. In this paper, based on the augmentation technique, we present two families of linear codes from some functions over finite fields. The first family of linear codes is constructed from monomial functions over finite fields. The weight distribution of the codes is determined in some cases. The codes are proved to be both optimally or almost optimally extendable and self-orthogonal under certain conditions. The localities of the codes and their duals are also studied and we obtain an infinite family of optimal or almost optimal locally recoverable codes. The second family of linear codes is constructed from weakly regular bent functions over finite fields and its weight distribution is explicitly determined. This family of codes is also proved to be both optimally or almost optimally extendable and self-orthogonal. Besides, this family of codes has been proven to have locality 2 or 3 under certain conditions. Particularly, we derive two infinite families of optimal locally recoverable codes. Some infinite families of 2-designs are obtained from the codes in this paper as byproducts.
{"title":"Two Families of Linear Codes With Desirable Properties From Some Functions Over Finite Fields","authors":"Ziling Heng;Xiaoru Li;Yansheng Wu;Qi Wang","doi":"10.1109/TIT.2024.3439408","DOIUrl":"10.1109/TIT.2024.3439408","url":null,"abstract":"Linear codes are widely studied in coding theory as they have nice applications in distributed storage, combinatorics, lattices, cryptography and so on. Constructing linear codes with desirable properties is an interesting research topic. In this paper, based on the augmentation technique, we present two families of linear codes from some functions over finite fields. The first family of linear codes is constructed from monomial functions over finite fields. The weight distribution of the codes is determined in some cases. The codes are proved to be both optimally or almost optimally extendable and self-orthogonal under certain conditions. The localities of the codes and their duals are also studied and we obtain an infinite family of optimal or almost optimal locally recoverable codes. The second family of linear codes is constructed from weakly regular bent functions over finite fields and its weight distribution is explicitly determined. This family of codes is also proved to be both optimally or almost optimally extendable and self-orthogonal. Besides, this family of codes has been proven to have locality 2 or 3 under certain conditions. Particularly, we derive two infinite families of optimal locally recoverable codes. Some infinite families of 2-designs are obtained from the codes in this paper as byproducts.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"8320-8342"},"PeriodicalIF":2.2,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141947531","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-05DOI: 10.1109/tit.2024.3435008
Dor Elimelech, Wasim Huleihel
{"title":"Detection of Correlated Random Vectors","authors":"Dor Elimelech, Wasim Huleihel","doi":"10.1109/tit.2024.3435008","DOIUrl":"https://doi.org/10.1109/tit.2024.3435008","url":null,"abstract":"","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"62 1","pages":""},"PeriodicalIF":2.5,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141947534","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-05DOI: 10.1109/TIT.2024.3439136
Yuesheng Xu;Haizhang Zhang
We consider deep neural networks (DNNs) with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of DNNs to a meaningful function as the number of their layers tends to infinity. In the framework, special results on uniform convergence of DNNs with a fixed width, bounded widths and unbounded widths are presented. In particular, as convolutional neural networks are special DNNs with weight matrices of increasing widths, we put forward conditions on the mask sequence which lead to uniform convergence of the resulting convolutional neural networks. The Lipschitz continuity assumption on the activation functions allows us to include in our theory most of commonly used activation functions in applications.
{"title":"Uniform Convergence of Deep Neural Networks With Lipschitz Continuous Activation Functions and Variable Widths","authors":"Yuesheng Xu;Haizhang Zhang","doi":"10.1109/TIT.2024.3439136","DOIUrl":"10.1109/TIT.2024.3439136","url":null,"abstract":"We consider deep neural networks (DNNs) with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of DNNs to a meaningful function as the number of their layers tends to infinity. In the framework, special results on uniform convergence of DNNs with a fixed width, bounded widths and unbounded widths are presented. In particular, as convolutional neural networks are special DNNs with weight matrices of increasing widths, we put forward conditions on the mask sequence which lead to uniform convergence of the resulting convolutional neural networks. The Lipschitz continuity assumption on the activation functions allows us to include in our theory most of commonly used activation functions in applications.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 10","pages":"7125-7142"},"PeriodicalIF":2.2,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10623495","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141947533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-01DOI: 10.1109/TIT.2024.3436923
Minjia Shi;Shitao Li;Tor Helleseth
Determining the weight distribution of a code is an old and fundamental topic in coding theory that has been thoroughly studied. In 1977, Helleseth, Kløve, and Mykkeltveit presented a weight enumerator polynomial of the lifted code over ${mathbb {F}}_{q^{ell } }$ of a q-ary linear code with significant combinatorial properties, which can determine the support weight distribution of this linear code. The Solomon-Stiffler codes are a family of famous Griesmer codes, which were proposed by Solomon and Stiffler in 1965. In this paper, we determine the weight enumerator polynomials of the lifted codes of the projective Solomon-Stiffler codes using some combinatorial properties of subspaces. As a result, we determine the support weight distributions of the projective Solomon-Stiffler codes. In particular, we determine the weight hierarchies of the projective Solomon-Stiffler codes.
{"title":"The Weight Enumerator Polynomials of the Lifted Codes of the Projective Solomon-Stiffler Codes","authors":"Minjia Shi;Shitao Li;Tor Helleseth","doi":"10.1109/TIT.2024.3436923","DOIUrl":"10.1109/TIT.2024.3436923","url":null,"abstract":"Determining the weight distribution of a code is an old and fundamental topic in coding theory that has been thoroughly studied. In 1977, Helleseth, Kløve, and Mykkeltveit presented a weight enumerator polynomial of the lifted code over \u0000<inline-formula> <tex-math>${mathbb {F}}_{q^{ell } }$ </tex-math></inline-formula>\u0000 of a q-ary linear code with significant combinatorial properties, which can determine the support weight distribution of this linear code. The Solomon-Stiffler codes are a family of famous Griesmer codes, which were proposed by Solomon and Stiffler in 1965. In this paper, we determine the weight enumerator polynomials of the lifted codes of the projective Solomon-Stiffler codes using some combinatorial properties of subspaces. As a result, we determine the support weight distributions of the projective Solomon-Stiffler codes. In particular, we determine the weight hierarchies of the projective Solomon-Stiffler codes.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 9","pages":"6316-6325"},"PeriodicalIF":2.2,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141886463","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Most low-density parity-check (LDPC) code constructions are considered over finite fields. In this work, we focus on regular LDPC codes over integer residue rings and analyze their performance with respect to the Lee metric. Their error-correction performance is studied over two channel models, in the Lee metric. The first channel model is a discrete memoryless channel, whereas in the second channel model an error vector is drawn uniformly at random from all vectors of a fixed Lee weight. It is known that the two channel laws coincide in the asymptotic regime, meaning that their marginal distributions match. For both channel models, we derive upper bounds on the block error probability in terms of a random coding union bound as well as sphere packing bounds that make use of the marginal distribution of the considered channels. We estimate the decoding error probability of regular LDPC code ensembles over the channels using the marginal distribution and determining the expected Lee weight distribution of a random LDPC code over a finite integer ring. By means of density evolution and finite-length simulations, we estimate the error-correction performance of selected LDPC code ensembles under belief propagation decoding and a low-complexity symbol message passing decoding algorithm and compare the performances. The analysis developed in this paper may serve to design regular low-density parity-check (LDPC) codes over integer residue rings for storage and cryptographic application.
{"title":"Error-Correction Performance of Regular Ring-Linear LDPC Codes Over Lee Channels","authors":"Jessica Bariffi;Hannes Bartz;Gianluigi Liva;Joachim Rosenthal","doi":"10.1109/TIT.2024.3436938","DOIUrl":"10.1109/TIT.2024.3436938","url":null,"abstract":"Most low-density parity-check (LDPC) code constructions are considered over finite fields. In this work, we focus on regular LDPC codes over integer residue rings and analyze their performance with respect to the Lee metric. Their error-correction performance is studied over two channel models, in the Lee metric. The first channel model is a discrete memoryless channel, whereas in the second channel model an error vector is drawn uniformly at random from all vectors of a fixed Lee weight. It is known that the two channel laws coincide in the asymptotic regime, meaning that their marginal distributions match. For both channel models, we derive upper bounds on the block error probability in terms of a random coding union bound as well as sphere packing bounds that make use of the marginal distribution of the considered channels. We estimate the decoding error probability of regular LDPC code ensembles over the channels using the marginal distribution and determining the expected Lee weight distribution of a random LDPC code over a finite integer ring. By means of density evolution and finite-length simulations, we estimate the error-correction performance of selected LDPC code ensembles under belief propagation decoding and a low-complexity symbol message passing decoding algorithm and compare the performances. The analysis developed in this paper may serve to design regular low-density parity-check (LDPC) codes over integer residue rings for storage and cryptographic application.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"7820-7839"},"PeriodicalIF":2.2,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141886462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-30DOI: 10.1109/TIT.2024.3435849
Anjana A. Mahesh;Charul Rajput;Bobbadi Rupa;B. Sundar Rajan
Ong and Ho developed optimal linear index codes for single uniprior index coding problems (ICPs) by finding a spanning tree for each strongly connected component of their information-flow graphs, following which Thomas et al. considered the same class of ICPs over Rayleigh fading channels. They developed the min-max probability of error criterion for choosing an index code from the set of bandwidth-optimal linear index codes. Motivated by the above works, this paper deals with single uniprior ICPs over binary-input continuous-output channels. Minimizing the average probability of error is introduced as a criterion for further selection of index codes which is shown to be equivalent to minimizing the total number of transmissions used for decoding the message requests at all the receivers. An algorithm that generates a spanning tree with a lower value of this metric than the optimal star graph is also presented. A couple of lower bounds for the total number of transmissions, used by any optimal index code, are derived, and two classes of ICPs for which these bounds are tight are identified. An improvement of the proposed algorithm for information-flow graphs with bridges and a generalization of the improved algorithm for information-flow graphs obtainable as the union of strongly connected sub-graphs are presented, and some optimality results are derived.
Ong 和 Ho 通过为其信息流图中的每个强连接分量找到一棵生成树，开发出了单前索引编码问题 (ICP) 的最优线性索引编码，随后 Thomas 等人考虑了瑞利衰减信道上的同一类 ICP。他们提出了从带宽最优线性索引码集合中选择索引码的最小-最大错误概率准则。受上述著作的启发，本文讨论了二元输入连续输出信道上的单前导 ICP。本文将平均错误概率最小化作为进一步选择索引编码的标准，并证明这等同于将所有接收器用于解码信息请求的传输总数最小化。此外，还介绍了一种生成生成树的算法，该生成树的指标值低于最优星形图。此外，还推导出了任何最优索引代码所使用的传输总数的几个下限，并确定了这些下限较小的两类 ICP。此外，还提出了针对具有桥的信息流图的改进算法，以及针对可作为强连接子图联盟的信息流图的改进算法的推广，并得出了一些最优结果。
{"title":"Average Probability of Error for Single Uniprior Index Coding Over Binary-Input Continuous-Output Channels","authors":"Anjana A. Mahesh;Charul Rajput;Bobbadi Rupa;B. Sundar Rajan","doi":"10.1109/TIT.2024.3435849","DOIUrl":"10.1109/TIT.2024.3435849","url":null,"abstract":"Ong and Ho developed optimal linear index codes for single uniprior index coding problems (ICPs) by finding a spanning tree for each strongly connected component of their information-flow graphs, following which Thomas et al. considered the same class of ICPs over Rayleigh fading channels. They developed the min-max probability of error criterion for choosing an index code from the set of bandwidth-optimal linear index codes. Motivated by the above works, this paper deals with single uniprior ICPs over binary-input continuous-output channels. Minimizing the average probability of error is introduced as a criterion for further selection of index codes which is shown to be equivalent to minimizing the total number of transmissions used for decoding the message requests at all the receivers. An algorithm that generates a spanning tree with a lower value of this metric than the optimal star graph is also presented. A couple of lower bounds for the total number of transmissions, used by any optimal index code, are derived, and two classes of ICPs for which these bounds are tight are identified. An improvement of the proposed algorithm for information-flow graphs with bridges and a generalization of the improved algorithm for information-flow graphs obtainable as the union of strongly connected sub-graphs are presented, and some optimality results are derived.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 9","pages":"6297-6315"},"PeriodicalIF":2.2,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141864887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-29DOI: 10.1109/TIT.2024.3435346
Xiangliang Kong
Modern large-scale distributed storage systems use erasure codes to protect against node failures with low storage overhead. In practice, the failure rate and other factors of storage devices in the system may vary significantly over time, and leads to changes of the ideal code parameters. To maintain the storage efficiency, this requires the system to adjust parameters of the currently used codes. The changing process of code parameters on encoded data is called code conversion. As an important class of storage codes, locally repairable codes (LRCs) can repair any codeword symbol using a small number of other symbols. This feature makes LRCs highly efficient for addressing single node failures in the storage systems. In this paper, we investigate the code conversions for locally repairable codes in the merge regime. We establish a lower bound on the access cost of code conversion for general LRCs and propose a construction of LRCs that can perform code conversions with access cost matching this bound. This construction yields a family of LRCs with optimal conversion processes over a field size linear in the code length. As a special case, it provides a family of RS codes with optimal conversion processes, which could be of particular practical interest.
{"title":"Locally Repairable Convertible Codes With Optimal Access Costs","authors":"Xiangliang Kong","doi":"10.1109/TIT.2024.3435346","DOIUrl":"10.1109/TIT.2024.3435346","url":null,"abstract":"Modern large-scale distributed storage systems use erasure codes to protect against node failures with low storage overhead. In practice, the failure rate and other factors of storage devices in the system may vary significantly over time, and leads to changes of the ideal code parameters. To maintain the storage efficiency, this requires the system to adjust parameters of the currently used codes. The changing process of code parameters on encoded data is called code conversion. As an important class of storage codes, locally repairable codes (LRCs) can repair any codeword symbol using a small number of other symbols. This feature makes LRCs highly efficient for addressing single node failures in the storage systems. In this paper, we investigate the code conversions for locally repairable codes in the merge regime. We establish a lower bound on the access cost of code conversion for general LRCs and propose a construction of LRCs that can perform code conversions with access cost matching this bound. This construction yields a family of LRCs with optimal conversion processes over a field size linear in the code length. As a special case, it provides a family of RS codes with optimal conversion processes, which could be of particular practical interest.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 9","pages":"6239-6257"},"PeriodicalIF":2.2,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141864893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-29DOI: 10.1109/tit.2024.3435412
Zhiwei Song, Lin Chen, Dragomir Ž Ðoković
{"title":"The existence of distinguishable bases in three-dimensional subspaces of qutrit-qudit systems under one-way local projective measurements and classical communication","authors":"Zhiwei Song, Lin Chen, Dragomir Ž Ðoković","doi":"10.1109/tit.2024.3435412","DOIUrl":"https://doi.org/10.1109/tit.2024.3435412","url":null,"abstract":"","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"49 1","pages":""},"PeriodicalIF":2.5,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141864890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-29DOI: 10.1109/TIT.2024.3435522
Jeongyeol Kwon;Wei Qian;Yudong Chen;Constantine Caramanis;Damek Davis;Nhat Ho
Recent results established that EM enjoys global convergence for Gaussian Mixture Models. For Mixed Linear Regression, however, only local convergence results have been established, and those only for the high signal-to-noise ratio (SNR) regime. In this work, we completely characterize the global optimality of EM: we show that starting from any randomly initialized point, the EM algorithm converges to the true parameter ${beta }^{*}$ at the minimax statistical rates under all SNR regimes. Toward this goal, we first show the global convergence of the EM algorithm at the population level. Then we provide a complete characterization of statistical and computational behaviors of EM under all SNR regimes with finite samples. In particular: (i) When the SNR is sufficiently large, the EM updates converge to the true parameter $ {beta }^{*}$ at the standard parametric convergence rate $O((d/n)^{1/2})$ after $O(log (n/d))$ iterations. (ii) In the regime where the SNR is above $O((d/n)^{1/4})$ and below some constant, the EM iterates converge to a $O({mathrm { SNR}}^{-1} (d/n)^{1/2})$ neighborhood of the true parameter, when the number of iterations is of the order $O({mathrm { SNR}}^{-2} log (n/d))$ . (iii) In the low SNR regime where the SNR is below $O((d/n)^{1/4})$ , we show that EM converges to a $O((d/n)^{1/4})$ neighborhood of the true parameters, after $O((n/d)^{1/2})$ iterations. By providing tight convergence guarantees of the EM algorithm in middle-to-low SNR regimes, we reveal that in low SNR, EM changes rate, matching the $n^{-1/4}$ rate of the MLE, a behavior that previous work had been unable to show.
最近的研究结果表明，EM 对高斯混合模型具有全局收敛性。然而，对于混合线性回归，目前只确定了局部收敛结果，而且只针对高信噪比（SNR）机制。在这项工作中，我们完全描述了 EM 的全局最优性：我们证明了从任意随机初始化点开始，EM 算法在所有信噪比条件下都能以最小统计率收敛到真实参数 ${beta }^{*}$。为了实现这一目标，我们首先展示了 EM 算法在群体水平上的全局收敛性。然后，我们提供了有限样本下 EM 在所有信噪比情况下的统计和计算行为的完整特征。具体而言：(i) 当信噪比足够大时，经过 $O(log (n/d))$ 次迭代后，EM 更新以标准参数收敛速率 $O((d/n)^{1/2})$收敛到真实参数 $ {beta }^{*}$。(ii) 在信噪比高于 $O((d/n)^{1/4})$且低于某个常数的情况下，当迭代次数为 $O({mathrm { SNR}}^{-2} log (n/d))$ 时，EM 迭代收敛到真实参数的 $O({mathrm { SNR}}^{-1} (d/n)^{1/2})$ 邻域。(iii) 在信噪比低于 $O((d/n)^{1/4})$的低信噪比条件下，我们证明 EM 在经过 $O((n/d)^{1/2})$迭代后，会收敛到真实参数的 $O((d/n)^{1/4})$邻域。通过提供 EM 算法在中低信噪比条件下的严格收敛保证，我们揭示了在低信噪比条件下，EM 的速率会发生变化，与 MLE 的 $n^{-1/4}$ 速率相匹配，这是以前的工作无法显示的。
{"title":"Global Optimality of the EM Algorithm for Mixtures of Two-Component Linear Regressions","authors":"Jeongyeol Kwon;Wei Qian;Yudong Chen;Constantine Caramanis;Damek Davis;Nhat Ho","doi":"10.1109/TIT.2024.3435522","DOIUrl":"10.1109/TIT.2024.3435522","url":null,"abstract":"Recent results established that EM enjoys global convergence for Gaussian Mixture Models. For Mixed Linear Regression, however, only local convergence results have been established, and those only for the high signal-to-noise ratio (SNR) regime. In this work, we completely characterize the global optimality of EM: we show that starting from any randomly initialized point, the EM algorithm converges to the true parameter \u0000<inline-formula> <tex-math>${beta }^{*}$ </tex-math></inline-formula>\u0000 at the minimax statistical rates under all SNR regimes. Toward this goal, we first show the global convergence of the EM algorithm at the population level. Then we provide a complete characterization of statistical and computational behaviors of EM under all SNR regimes with finite samples. In particular: (i) When the SNR is sufficiently large, the EM updates converge to the true parameter \u0000<inline-formula> <tex-math>$ {beta }^{*}$ </tex-math></inline-formula>\u0000 at the standard parametric convergence rate \u0000<inline-formula> <tex-math>$O((d/n)^{1/2})$ </tex-math></inline-formula>\u0000 after \u0000<inline-formula> <tex-math>$O(log (n/d))$ </tex-math></inline-formula>\u0000 iterations. (ii) In the regime where the SNR is above \u0000<inline-formula> <tex-math>$O((d/n)^{1/4})$ </tex-math></inline-formula>\u0000 and below some constant, the EM iterates converge to a \u0000<inline-formula> <tex-math>$O({mathrm { SNR}}^{-1} (d/n)^{1/2})$ </tex-math></inline-formula>\u0000 neighborhood of the true parameter, when the number of iterations is of the order \u0000<inline-formula> <tex-math>$O({mathrm { SNR}}^{-2} log (n/d))$ </tex-math></inline-formula>\u0000. (iii) In the low SNR regime where the SNR is below \u0000<inline-formula> <tex-math>$O((d/n)^{1/4})$ </tex-math></inline-formula>\u0000, we show that EM converges to a \u0000<inline-formula> <tex-math>$O((d/n)^{1/4})$ </tex-math></inline-formula>\u0000 neighborhood of the true parameters, after \u0000<inline-formula> <tex-math>$O((n/d)^{1/2})$ </tex-math></inline-formula>\u0000 iterations. By providing tight convergence guarantees of the EM algorithm in middle-to-low SNR regimes, we reveal that in low SNR, EM changes rate, matching the \u0000<inline-formula> <tex-math>$n^{-1/4}$ </tex-math></inline-formula>\u0000 rate of the MLE, a behavior that previous work had been unable to show.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 9","pages":"6519-6546"},"PeriodicalIF":2.2,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141864889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-29DOI: 10.1109/tit.2024.3434481
Ayça Çeşmelioğlu, Wilfried Meidl
{"title":"Vectorial bent functions with non-weakly regular components","authors":"Ayça Çeşmelioğlu, Wilfried Meidl","doi":"10.1109/tit.2024.3434481","DOIUrl":"https://doi.org/10.1109/tit.2024.3434481","url":null,"abstract":"","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"10 1","pages":""},"PeriodicalIF":2.5,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141864891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：