Pub Date : 2024-08-19DOI: 10.1109/TIT.2024.3446000
Dror Chawin;Ishay Haviv
The index coding problem is concerned with broadcasting encoded information to a collection of receivers in a way that enables each receiver to discover its required data based on its side information, which comprises the data required by some of the others. Given the side information map, represented by a graph in the symmetric case and by a digraph otherwise, the goal is to devise a coding scheme of minimum broadcast length. We present a general method for developing efficient algorithms for approximating the index coding rate for prescribed families of instances. As applications, we obtain polynomial-time algorithms that approximate the index coding rate of graphs and digraphs on n vertices to within factors of $O(n/log ^{2} n)$ and $O(n/log n)$ respectively. This improves on the approximation factors of $O(n/log n)$ for graphs and $O(n cdot log log n/log n)$ for digraphs achieved by Blasiak, Kleinberg, and Lubetzky. For the family of quasi-line graphs, we exhibit a polynomial-time algorithm that approximates the index coding rate to within a factor of 2. This improves on the approximation factor of $O(n^{2/3})$ achieved by Arbabjolfaei and Kim for graphs on n vertices taken from certain sub-families of quasi-line graphs. Our approach is applicable for approximating a variety of additional graph and digraph quantities to within the same approximation factors. Specifically, it captures every graph quantity sandwiched between the independence number and the clique cover number and every digraph quantity sandwiched between the maximum size of an acyclic induced sub-digraph and the directed clique cover number.
{"title":"Improved Approximation Algorithms for Index Coding","authors":"Dror Chawin;Ishay Haviv","doi":"10.1109/TIT.2024.3446000","DOIUrl":"10.1109/TIT.2024.3446000","url":null,"abstract":"The index coding problem is concerned with broadcasting encoded information to a collection of receivers in a way that enables each receiver to discover its required data based on its side information, which comprises the data required by some of the others. Given the side information map, represented by a graph in the symmetric case and by a digraph otherwise, the goal is to devise a coding scheme of minimum broadcast length. We present a general method for developing efficient algorithms for approximating the index coding rate for prescribed families of instances. As applications, we obtain polynomial-time algorithms that approximate the index coding rate of graphs and digraphs on n vertices to within factors of \u0000<inline-formula> <tex-math>$O(n/log ^{2} n)$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$O(n/log n)$ </tex-math></inline-formula>\u0000 respectively. This improves on the approximation factors of \u0000<inline-formula> <tex-math>$O(n/log n)$ </tex-math></inline-formula>\u0000 for graphs and \u0000<inline-formula> <tex-math>$O(n cdot log log n/log n)$ </tex-math></inline-formula>\u0000 for digraphs achieved by Blasiak, Kleinberg, and Lubetzky. For the family of quasi-line graphs, we exhibit a polynomial-time algorithm that approximates the index coding rate to within a factor of 2. This improves on the approximation factor of \u0000<inline-formula> <tex-math>$O(n^{2/3})$ </tex-math></inline-formula>\u0000 achieved by Arbabjolfaei and Kim for graphs on n vertices taken from certain sub-families of quasi-line graphs. Our approach is applicable for approximating a variety of additional graph and digraph quantities to within the same approximation factors. Specifically, it captures every graph quantity sandwiched between the independence number and the clique cover number and every digraph quantity sandwiched between the maximum size of an acyclic induced sub-digraph and the directed clique cover number.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"8266-8275"},"PeriodicalIF":2.2,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-19DOI: 10.1109/TIT.2024.3445929
Yubo Sun;Gennian Ge
An edit refers to a single insertion, deletion, or substitution. This paper aims to construct binary codes that can correct two edits. To do this, a necessary and sufficient condition for a code to be two-edit correctable is provided, showing that a code is a two-edit correcting code if and only if it can correct two deletions, up to two substitutions, and one deletion and up to one substitution, separately. This criterion allows for the construction of two-edit correcting codes leveraging these three types of error correcting codes. In the field of constructing codes for correcting two deletions, we present a construction with $4log n+O(log log n)$ redundant bits that can be viewed as a subcode proposed by Guruswami and Håstad, and provide an alternative proof. Moreover, our two-deletion correcting codes can also correct up to two substitutions after making a slight modification. In the field of constructing codes for correcting one deletion and up to one substitution, we present a construction with $4 log n+O(log log n)$ redundant bits, which outperforms the best previously known results $6 log n+O(1)$ . Leveraging these codes, we obtain a construction of two-edit correcting codes with $6 log n+O(log log n)$ redundant bits. This outperforms the best previously known result, which requires at least $8log n$ redundant bits. Moreover, we also consider the list-decoding problem under the two-edit channel and construct a two-edit list-decodable code with a list size of two employing $4 log n+O(log log n)$ redundant bits.
{"title":"Binary Codes for Correcting Two Edits","authors":"Yubo Sun;Gennian Ge","doi":"10.1109/TIT.2024.3445929","DOIUrl":"10.1109/TIT.2024.3445929","url":null,"abstract":"An edit refers to a single insertion, deletion, or substitution. This paper aims to construct binary codes that can correct two edits. To do this, a necessary and sufficient condition for a code to be two-edit correctable is provided, showing that a code is a two-edit correcting code if and only if it can correct two deletions, up to two substitutions, and one deletion and up to one substitution, separately. This criterion allows for the construction of two-edit correcting codes leveraging these three types of error correcting codes. In the field of constructing codes for correcting two deletions, we present a construction with \u0000<inline-formula> <tex-math>$4log n+O(log log n)$ </tex-math></inline-formula>\u0000 redundant bits that can be viewed as a subcode proposed by Guruswami and Håstad, and provide an alternative proof. Moreover, our two-deletion correcting codes can also correct up to two substitutions after making a slight modification. In the field of constructing codes for correcting one deletion and up to one substitution, we present a construction with \u0000<inline-formula> <tex-math>$4 log n+O(log log n)$ </tex-math></inline-formula>\u0000 redundant bits, which outperforms the best previously known results \u0000<inline-formula> <tex-math>$6 log n+O(1)$ </tex-math></inline-formula>\u0000. Leveraging these codes, we obtain a construction of two-edit correcting codes with \u0000<inline-formula> <tex-math>$6 log n+O(log log n)$ </tex-math></inline-formula>\u0000 redundant bits. This outperforms the best previously known result, which requires at least \u0000<inline-formula> <tex-math>$8log n$ </tex-math></inline-formula>\u0000 redundant bits. Moreover, we also consider the list-decoding problem under the two-edit channel and construct a two-edit list-decodable code with a list size of two employing \u0000<inline-formula> <tex-math>$4 log n+O(log log n)$ </tex-math></inline-formula>\u0000 redundant bits.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 10","pages":"6877-6898"},"PeriodicalIF":2.2,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-19DOI: 10.1109/TIT.2024.3440929
Leighton P. Barnes;Alex Dytso;Jingbo Liu;H. Vincent Poor
Consider the problem of estimating a random variable X from noisy observations $Y = X+ Z$ , where Z is standard normal, under the $L^{1}$ fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on X that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution $P_{X|Y=y}$ is symmetric for all y, then X must follow a Gaussian distribution. Additionally, we consider other $L^{p}$ losses and observe the following phenomenon: for $p in [{1,2}]$ , Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for $p in (2,infty)$ , infinitely many prior distributions on X can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.
考虑在 $L^{1}$ 保真度准则下，从有噪声的观测值 $Y = X+ Z$ 中估计随机变量 X 的问题，其中 Z 为标准正态分布。众所周知，这种情况下的最优贝叶斯估计器是条件中值。本研究表明，唯一能诱导条件中值线性的 X 先验分布是高斯分布。同时，还提出了其他一些结果。特别是，它证明了如果条件分布 $P_{X|Y=y}$ 对所有 y 都是对称的，那么 X 必须遵循高斯分布。此外，我们还考虑了其他 $L^{p}$ 损失，并观察到以下现象：对于 $p in [{1,2}]$ 来说，高斯分布是唯一能诱导线性最优贝叶斯估计器的先验分布；而对于 $p in (2,infty)$ 来说，X 上有无限多的先验分布能诱导线性。最后，本文还提供了一些扩展，以涵盖导致某些指数族条件分布的噪声模型。
{"title":"L1 Estimation: On the Optimality of Linear Estimators","authors":"Leighton P. Barnes;Alex Dytso;Jingbo Liu;H. Vincent Poor","doi":"10.1109/TIT.2024.3440929","DOIUrl":"10.1109/TIT.2024.3440929","url":null,"abstract":"Consider the problem of estimating a random variable X from noisy observations \u0000<inline-formula> <tex-math>$Y = X+ Z$ </tex-math></inline-formula>\u0000, where Z is standard normal, under the \u0000<inline-formula> <tex-math>$L^{1}$ </tex-math></inline-formula>\u0000 fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on X that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution \u0000<inline-formula> <tex-math>$P_{X|Y=y}$ </tex-math></inline-formula>\u0000 is symmetric for all y, then X must follow a Gaussian distribution. Additionally, we consider other \u0000<inline-formula> <tex-math>$L^{p}$ </tex-math></inline-formula>\u0000 losses and observe the following phenomenon: for \u0000<inline-formula> <tex-math>$p in [{1,2}]$ </tex-math></inline-formula>\u0000, Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for \u0000<inline-formula> <tex-math>$p in (2,infty)$ </tex-math></inline-formula>\u0000, infinitely many prior distributions on X can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"8026-8039"},"PeriodicalIF":2.2,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-19DOI: 10.1109/TIT.2024.3446191
Krzysztof Szczygielski;Michał Studziński
In this paper, we present new families of quantum channels for which corresponding minimum output Rényi p-entropy is not additive. Our manuscript is motivated by the results of Grudka et al. and we focus on channels characterized by both extensions and subspaces of the antisymmetric subspace in $mathbb {C}^{d} otimes mathbb {C} ^{d}$ , which exhibit additivity breaking for $pgt 2$ .
{"title":"New Constructive Counterexamples to Additivity of Minimum Output Rényi p-Entropy of Quantum Channels","authors":"Krzysztof Szczygielski;Michał Studziński","doi":"10.1109/TIT.2024.3446191","DOIUrl":"10.1109/TIT.2024.3446191","url":null,"abstract":"In this paper, we present new families of quantum channels for which corresponding minimum output Rényi p-entropy is not additive. Our manuscript is motivated by the results of Grudka et al. and we focus on channels characterized by both extensions and subspaces of the antisymmetric subspace in \u0000<inline-formula> <tex-math>$mathbb {C}^{d} otimes mathbb {C} ^{d}$ </tex-math></inline-formula>\u0000, which exhibit additivity breaking for \u0000<inline-formula> <tex-math>$pgt 2$ </tex-math></inline-formula>\u0000.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 10","pages":"7023-7035"},"PeriodicalIF":2.2,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-16DOI: 10.1109/TIT.2024.3444892
Maosheng Xiong;Haode Yan
The cross-correlation problem is a classic problem in sequence design. In this paper we compute the cross-correlation distribution of the Niho-type decimation $d=3(p^{m}-1)+1$ over ${mathrm {GF}}(p^{2m})$ for any prime $p ge 5$ . Previously this problem was solved by Xia et al. only for $p=2$ and $p=3$ in a series of papers. The main difficulty of this problem for $p ge 5$ , as pointed out by Xia et al., is to count the number of codewords of “pure weight” 5 in p-ary Zetterberg codes. It turns out this counting problem can be transformed by the MacWilliams identity into counting codewords of weight at most 5 in p-ary Melas codes, the most difficult of which is related to a K3 surface well studied in the literature and can be computed. When $p ge 7$ , the theory of elliptic curves over finite fields also plays an important role in the resolution of this problem.
{"title":"On Correlation Distribution of Niho-Type Decimation d = 3(pm - 1) + 1","authors":"Maosheng Xiong;Haode Yan","doi":"10.1109/TIT.2024.3444892","DOIUrl":"10.1109/TIT.2024.3444892","url":null,"abstract":"The cross-correlation problem is a classic problem in sequence design. In this paper we compute the cross-correlation distribution of the Niho-type decimation \u0000<inline-formula> <tex-math>$d=3(p^{m}-1)+1$ </tex-math></inline-formula>\u0000 over \u0000<inline-formula> <tex-math>${mathrm {GF}}(p^{2m})$ </tex-math></inline-formula>\u0000 for any prime \u0000<inline-formula> <tex-math>$p ge 5$ </tex-math></inline-formula>\u0000. Previously this problem was solved by Xia et al. only for \u0000<inline-formula> <tex-math>$p=2$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$p=3$ </tex-math></inline-formula>\u0000 in a series of papers. The main difficulty of this problem for \u0000<inline-formula> <tex-math>$p ge 5$ </tex-math></inline-formula>\u0000, as pointed out by Xia et al., is to count the number of codewords of “pure weight” 5 in p-ary Zetterberg codes. It turns out this counting problem can be transformed by the MacWilliams identity into counting codewords of weight at most 5 in p-ary Melas codes, the most difficult of which is related to a K3 surface well studied in the literature and can be computed. When \u0000<inline-formula> <tex-math>$p ge 7$ </tex-math></inline-formula>\u0000, the theory of elliptic curves over finite fields also plays an important role in the resolution of this problem.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"8289-8302"},"PeriodicalIF":2.2,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-16DOI: 10.1109/TIT.2024.3445122
Verónica Becher;Olivier Carton
We introduce two-dimensional periodic arrays that are a variant of the de Bruijn tori. We call them nested perfect arrays. Instead of asking that every array of a given size has exactly one occurrence, we partition the positions in congruence classes and we ask exactly one occurrence in each congruence class. We also ask that this property applies recursively to each of the subarrays. We give a method to construct nested perfect arrays based on Pascal triangle matrix modulo 2. For the two-symbol alphabet, and for n being a power of 2, we partition the positions of the arrays in $n^{2}$ many congruence classes by taking the row number modulo n and the column number modulo n. We construct arrays where each possible $ntimes n$ array occurs $n^{2}$ times, once in each congruence class. Our method yields exponentially many (in $n^{2}$ ) different nested perfect arrays.
我们介绍的二维周期阵列是 de Bruijn tori 的一种变体。我们称之为嵌套完美数组。我们并不要求给定大小的每个数组都恰好有一次出现，而是将位置划分为全等类，并要求在每个全等类中都恰好有一次出现。我们还要求这一属性递归地适用于每个子数组。我们给出了一种基于帕斯卡三角形矩阵模 2 的嵌套完美数组构造方法。对于双符号字母表，并且 n 是 2 的幂次，我们通过取行数 modulo n 和列数 modulo n 将数组的位置划分为 $n^{2}$ 多个同类。我们构建的数组中，每个可能的 $ntimes n$ 数组出现了 $n^{2}$次，在每个同类中出现一次。我们的方法会产生指数级数量（以 $n^{2}$ 为单位）的不同嵌套完美数组。
{"title":"Nested Perfect Arrays","authors":"Verónica Becher;Olivier Carton","doi":"10.1109/TIT.2024.3445122","DOIUrl":"10.1109/TIT.2024.3445122","url":null,"abstract":"We introduce two-dimensional periodic arrays that are a variant of the de Bruijn tori. We call them nested perfect arrays. Instead of asking that every array of a given size has exactly one occurrence, we partition the positions in congruence classes and we ask exactly one occurrence in each congruence class. We also ask that this property applies recursively to each of the subarrays. We give a method to construct nested perfect arrays based on Pascal triangle matrix modulo 2. For the two-symbol alphabet, and for n being a power of 2, we partition the positions of the arrays in \u0000<inline-formula> <tex-math>$n^{2}$ </tex-math></inline-formula>\u0000 many congruence classes by taking the row number modulo n and the column number modulo n. We construct arrays where each possible \u0000<inline-formula> <tex-math>$ntimes n$ </tex-math></inline-formula>\u0000 array occurs \u0000<inline-formula> <tex-math>$n^{2}$ </tex-math></inline-formula>\u0000 times, once in each congruence class. Our method yields exponentially many (in \u0000<inline-formula> <tex-math>$n^{2}$ </tex-math></inline-formula>\u0000) different nested perfect arrays.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 10","pages":"7463-7471"},"PeriodicalIF":2.2,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-15DOI: 10.1109/TIT.2024.3444043
Sergei L. Semakov
We propose a scheme for finding the probabilities of events related to crossings of a level by a random process. Using this scheme, we estimate the probability that the first achievement of a given level by the component $y_{1}(x)$ of an n-dimensional continuous process ${mathbf { y}}(x)!=!{y_{1}(x),ldots,y_{n}(x)}$ occurs at some moment $x^{*}$ from a given interval $(x',x'')$ and, at this moment $x^{*}$ , the other components $y_{2}(x^{*}),ldots,y_{n}(x^{*})$ satisfy given constraints. The need for estimating the above-mentioned probability arises, in particular, in the problems of ensuring the safety of an aircraft landing.
我们提出了一种方法，用于计算与随机过程越级相关的事件概率。利用这个方案，我们可以估算出一个 n 维连续过程 ${mathbf { y}}(x)!= 的分量 $y_{1}(x)$ 首次达到给定水平的概率！{y_{1}(x),ldots,y_{n}(x)}$发生在给定区间$(x',x''')$的某个时刻$x^{*}$，并且在这个时刻$x^{*}$，其他分量$y_{2}(x^{*}),ldots,y_{n}(x^{*})$满足给定的约束条件。在确保飞机着陆安全的问题中，尤其需要估算上述概率。
{"title":"The First Achievement of a Given Level by a Random Process","authors":"Sergei L. Semakov","doi":"10.1109/TIT.2024.3444043","DOIUrl":"10.1109/TIT.2024.3444043","url":null,"abstract":"We propose a scheme for finding the probabilities of events related to crossings of a level by a random process. Using this scheme, we estimate the probability that the first achievement of a given level by the component \u0000<inline-formula> <tex-math>$y_{1}(x)$ </tex-math></inline-formula>\u0000 of an n-dimensional continuous process \u0000<inline-formula> <tex-math>${mathbf { y}}(x)!=!{y_{1}(x),ldots,y_{n}(x)}$ </tex-math></inline-formula>\u0000 occurs at some moment \u0000<inline-formula> <tex-math>$x^{*}$ </tex-math></inline-formula>\u0000 from a given interval \u0000<inline-formula> <tex-math>$(x',x'')$ </tex-math></inline-formula>\u0000 and, at this moment \u0000<inline-formula> <tex-math>$x^{*}$ </tex-math></inline-formula>\u0000, the other components \u0000<inline-formula> <tex-math>$y_{2}(x^{*}),ldots,y_{n}(x^{*})$ </tex-math></inline-formula>\u0000 satisfy given constraints. The need for estimating the above-mentioned probability arises, in particular, in the problems of ensuring the safety of an aircraft landing.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 10","pages":"7162-7178"},"PeriodicalIF":2.2,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-15DOI: 10.1109/TIT.2024.3444458
Tyron Lardy;Peter Grünwald;Peter Harremoës
Information projections have found important applications in probability theory, statistics, and related areas. In the field of hypothesis testing in particular, the reverse information projection (RIPr) has recently been shown to lead to growth-rate optimal (GRO) e-statistics for testing simple alternatives against composite null hypotheses. However, the RIPr as well as the GRO criterion are undefined whenever the infimum information divergence between the null and alternative is infinite. We show that in such scenarios, under some assumptions, there still exists a measure in the null that is closest to the alternative in a specific sense. Whenever the information divergence is finite, this measure coincides with the usual RIPr. It therefore gives a natural extension of the RIPr to certain cases where the latter was previously not defined. This extended notion of the RIPr is shown to lead to optimal e-statistics in a sense that is a novel, but natural, extension of the GRO criterion. We also give conditions under which the (extension of the) RIPr is a strict sub-probability measure, as well as conditions under which an approximation of the RIPr leads to approximate e-statistics. For this case we provide tight relations between the corresponding approximation rates.
{"title":"Reverse Information Projections and Optimal E-Statistics","authors":"Tyron Lardy;Peter Grünwald;Peter Harremoës","doi":"10.1109/TIT.2024.3444458","DOIUrl":"10.1109/TIT.2024.3444458","url":null,"abstract":"Information projections have found important applications in probability theory, statistics, and related areas. In the field of hypothesis testing in particular, the reverse information projection (RIPr) has recently been shown to lead to growth-rate optimal (GRO) e-statistics for testing simple alternatives against composite null hypotheses. However, the RIPr as well as the GRO criterion are undefined whenever the infimum information divergence between the null and alternative is infinite. We show that in such scenarios, under some assumptions, there still exists a measure in the null that is closest to the alternative in a specific sense. Whenever the information divergence is finite, this measure coincides with the usual RIPr. It therefore gives a natural extension of the RIPr to certain cases where the latter was previously not defined. This extended notion of the RIPr is shown to lead to optimal e-statistics in a sense that is a novel, but natural, extension of the GRO criterion. We also give conditions under which the (extension of the) RIPr is a strict sub-probability measure, as well as conditions under which an approximation of the RIPr leads to approximate e-statistics. For this case we provide tight relations between the corresponding approximation rates.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"7616-7631"},"PeriodicalIF":2.2,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-14DOI: 10.1109/TIT.2024.3441574
Xuetong Wu;Jonathan H. Manton;Uwe Aickelin;Jingge Zhu
Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different probability distributions. In this work, we give an information-theoretic analysis of the generalization error and excess risk of transfer learning algorithms. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence $D(mu |mu ')$ plays an important role in the characterizations where $mu $ and $mu '$ denote the distribution of the training data and the testing data, respectively. Specifically, we provide generalization error and excess risk upper bounds for learning algorithms where data from both distributions are available in the training phase. Recognizing that the bounds could be sub-optimal in general, we provide improved excess risk upper bounds for a certain class of algorithms, including the empirical risk minimization (ERM) algorithm, by making stronger assumptions through the central condition. To demonstrate the usefulness of the bounds, we further extend the analysis to the Gibbs algorithm and the noisy stochastic gradient descent method. We then generalize the mutual information bound with other divergences such as $phi $ -divergence and Wasserstein distance, which may lead to tighter bounds and can handle the case when $mu $ is not absolutely continuous with respect to $mu '$ . Several numerical results are provided to demonstrate our theoretical findings. Lastly, to address the problem that the bounds are often not directly applicable in practice due to the absence of the distributional knowledge of the data, we develop an algorithm (called InfoBoost) that dynamically adjusts the importance weights for both source and target data based on certain information measures. The empirical results show the effectiveness of the proposed algorithm.
{"title":"On the Generalization for Transfer Learning: An Information-Theoretic Analysis","authors":"Xuetong Wu;Jonathan H. Manton;Uwe Aickelin;Jingge Zhu","doi":"10.1109/TIT.2024.3441574","DOIUrl":"10.1109/TIT.2024.3441574","url":null,"abstract":"Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different probability distributions. In this work, we give an information-theoretic analysis of the generalization error and excess risk of transfer learning algorithms. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence \u0000<inline-formula> <tex-math>$D(mu |mu ')$ </tex-math></inline-formula>\u0000 plays an important role in the characterizations where \u0000<inline-formula> <tex-math>$mu $ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$mu '$ </tex-math></inline-formula>\u0000 denote the distribution of the training data and the testing data, respectively. Specifically, we provide generalization error and excess risk upper bounds for learning algorithms where data from both distributions are available in the training phase. Recognizing that the bounds could be sub-optimal in general, we provide improved excess risk upper bounds for a certain class of algorithms, including the empirical risk minimization (ERM) algorithm, by making stronger assumptions through the central condition. To demonstrate the usefulness of the bounds, we further extend the analysis to the Gibbs algorithm and the noisy stochastic gradient descent method. We then generalize the mutual information bound with other divergences such as \u0000<inline-formula> <tex-math>$phi $ </tex-math></inline-formula>\u0000-divergence and Wasserstein distance, which may lead to tighter bounds and can handle the case when \u0000<inline-formula> <tex-math>$mu $ </tex-math></inline-formula>\u0000 is not absolutely continuous with respect to \u0000<inline-formula> <tex-math>$mu '$ </tex-math></inline-formula>\u0000. Several numerical results are provided to demonstrate our theoretical findings. Lastly, to address the problem that the bounds are often not directly applicable in practice due to the absence of the distributional knowledge of the data, we develop an algorithm (called InfoBoost) that dynamically adjusts the importance weights for both source and target data based on certain information measures. The empirical results show the effectiveness of the proposed algorithm.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 10","pages":"7089-7124"},"PeriodicalIF":2.2,"publicationDate":"2024-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-13DOI: 10.1109/tit.2024.3443090
Shitao Li, Minjia Shi, San Ling
{"title":"An open problem and a conjecture on binary linear complementary pairs of codes","authors":"Shitao Li, Minjia Shi, San Ling","doi":"10.1109/tit.2024.3443090","DOIUrl":"https://doi.org/10.1109/tit.2024.3443090","url":null,"abstract":"","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"168 1","pages":""},"PeriodicalIF":2.5,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142194901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：