Pub Date : 2024-09-20DOI: 10.1109/JSAIT.2024.3465022
Michael Gastpar;Erixhen Sula
The Shannon lower bound has been the subject of several important contributions by Berger. This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger’s techniques. Moreover, as a new result, the Gray-Wyner network is added to the canon of settings for which such bounds are known. In the Shannon bounding technique, elegant lower bounds are expressed in terms of the source entropy power. Moreover, there is often a complementary upper bound that involves the source variance in such a way that the bounds coincide in the special case of Gaussian statistics. Such pairs of bounds are sometimes referred to as Shannon bounds. The present paper puts Berger’s work on many aspects of this problem in the context of more recent developments, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.
香农下界是伯杰数次重要贡献的主题。本文研究了均方误差失真条件下速率失真问题的香农下界，并特别强调了伯杰的技术。此外，作为一项新成果，格雷-维纳网络也被添加到已知此类约束的环境中。在香农约束技术中，优雅的下限用源熵功率表示。此外，在高斯统计的特殊情况下，通常会有一个涉及源方差的互补上界，使两者的边界重合。这样的边界对有时被称为香农边界。本文将伯杰在这一问题的许多方面所做的工作与最近的发展结合起来，包括间接和远程源编码，如伯杰最初提出的 CEO 问题，以及作为新贡献的格雷-惠纳网络。
{"title":"Shannon Bounds for Quadratic Rate-Distortion Problems","authors":"Michael Gastpar;Erixhen Sula","doi":"10.1109/JSAIT.2024.3465022","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3465022","url":null,"abstract":"The Shannon lower bound has been the subject of several important contributions by Berger. This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger’s techniques. Moreover, as a new result, the Gray-Wyner network is added to the canon of settings for which such bounds are known. In the Shannon bounding technique, elegant lower bounds are expressed in terms of the source entropy power. Moreover, there is often a complementary upper bound that involves the source variance in such a way that the bounds coincide in the special case of Gaussian statistics. Such pairs of bounds are sometimes referred to as Shannon bounds. The present paper puts Berger’s work on many aspects of this problem in the context of more recent developments, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"597-608"},"PeriodicalIF":0.0,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142452676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-02DOI: 10.1109/JSAIT.2024.3453273
Ruze Zhang;Xuan Guang;Shenghao Yang;Xueyan Niu;Bo Bai
In this paper, the problem of zero-error network function computation is considered, where in a directed acyclic network, a single sink node is required to compute with zero error a function of the source messages that are separately generated by multiple source nodes. From the information-theoretic point of view, we are interested in the fundamental computing capacity, which is defined as the average number of times that the function can be computed with zero error for one use of the network. The explicit characterization of the computing capacity in general is overwhelming difficult. The best known upper bound applicable to arbitrary network topologies and arbitrary target functions is the one proved by Guang et al. in using an approach of the cut-set strong partition. This bound is tight for all previously considered network function computation problems whose computing capacities are known. In this paper, we consider the model of computing the binary arithmetic sum over an asymmetric diamond network, which is of great importance to illustrate the combinatorial nature of network function computation problem. First, we prove a corrected upper bound 1 by using a linear programming approach, which corrects an invalid bound previously claimed in the literature. Nevertheless, this upper bound cannot bring any improvement over the best known upper bound for this model, which is also equal to 1. Further, by developing a different graph coloring approach, we obtain an improved upper bound ${}frac {1}{log _{3} 2+log 3-1}~(approx 0.822)$ . We thus show that the best known upper bound proved by Guang et al. is not tight for this model which answers the open problem that whether this bound in general is tight. On the other hand, we present an explicit code construction, which implies a lower bound ${}frac {1}{2}log _{3}6~(approx 0.815)$ on the computing capacity. Comparing the improved upper and lower bounds thus obtained, there exists a rough 0.007 gap between them.
{"title":"Computation of Binary Arithmetic Sum Over an Asymmetric Diamond Network","authors":"Ruze Zhang;Xuan Guang;Shenghao Yang;Xueyan Niu;Bo Bai","doi":"10.1109/JSAIT.2024.3453273","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3453273","url":null,"abstract":"In this paper, the problem of zero-error network function computation is considered, where in a directed acyclic network, a single sink node is required to compute with zero error a function of the source messages that are separately generated by multiple source nodes. From the information-theoretic point of view, we are interested in the fundamental computing capacity, which is defined as the average number of times that the function can be computed with zero error for one use of the network. The explicit characterization of the computing capacity in general is overwhelming difficult. The best known upper bound applicable to arbitrary network topologies and arbitrary target functions is the one proved by Guang et al. in using an approach of the cut-set strong partition. This bound is tight for all previously considered network function computation problems whose computing capacities are known. In this paper, we consider the model of computing the binary arithmetic sum over an asymmetric diamond network, which is of great importance to illustrate the combinatorial nature of network function computation problem. First, we prove a corrected upper bound 1 by using a linear programming approach, which corrects an invalid bound previously claimed in the literature. Nevertheless, this upper bound cannot bring any improvement over the best known upper bound for this model, which is also equal to 1. Further, by developing a different graph coloring approach, we obtain an improved upper bound \u0000<inline-formula> <tex-math>${}frac {1}{log _{3} 2+log 3-1}~(approx 0.822)$ </tex-math></inline-formula>\u0000. We thus show that the best known upper bound proved by Guang et al. is not tight for this model which answers the open problem that whether this bound in general is tight. On the other hand, we present an explicit code construction, which implies a lower bound \u0000<inline-formula> <tex-math>${}frac {1}{2}log _{3}6~(approx 0.815)$ </tex-math></inline-formula>\u0000 on the computing capacity. Comparing the improved upper and lower bounds thus obtained, there exists a rough 0.007 gap between them.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"585-596"},"PeriodicalIF":0.0,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142320442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-28DOI: 10.1109/JSAIT.2024.3451240
Nadim Ghaddar;Lele Wang
The problem of coding for the uplink and downlink of cloud radio access networks (C-RAN’s) with K users and L relays is considered. It is shown that low-complexity coding schemes that achieve any point in the rate-fronthaul region of joint coding and compression can be constructed starting from at most $4(K+L)-2$ point-to-point codes designed for symmetric channels. This reduces the seemingly hard task of constructing good codes for C-RAN’s to the much better understood task of finding good codes for single-user channels. To show this result, an equivalence between the achievable rate-fronthaul regions of joint coding and successive coding is established. Then, rate-splitting and quantization-splitting techniques are used to show that the task of achieving a rate-fronthaul point in the joint coding region can be simplified to that of achieving a corner point in a higher-dimensional C-RAN problem. As a by-product, some interesting properties of the rate-fronthaul region of joint decoding for uplink C-RAN’s are also derived.
本文研究了具有 K 个用户和 L 个中继站的云无线接入网（C-RAN）的上下行链路编码问题。研究表明，可以从最多 $4(K+L)-2$ 为对称信道设计的点对点编码开始，构建低复杂度编码方案，以实现联合编码和压缩的速率-链路区域内的任意点。这就把为 C-RAN 构建良好编码这一看似艰巨的任务简化为为单用户信道寻找良好编码这一更好理解的任务。为了证明这一结果，我们建立了联合编码和连续编码的可实现速率-频带区域之间的等价关系。然后，利用速率拆分和量化拆分技术来说明，在联合编码区域实现速率-频带点的任务可以简化为在高维 C-RAN 问题中实现角点的任务。作为副产品，还得出了上行链路 C-RAN 联合解码区域的一些有趣特性。
{"title":"Low-Complexity Coding Techniques for Cloud Radio Access Networks","authors":"Nadim Ghaddar;Lele Wang","doi":"10.1109/JSAIT.2024.3451240","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3451240","url":null,"abstract":"The problem of coding for the uplink and downlink of cloud radio access networks (C-RAN’s) with K users and L relays is considered. It is shown that low-complexity coding schemes that achieve any point in the rate-fronthaul region of joint coding and compression can be constructed starting from at most \u0000<inline-formula> <tex-math>$4(K+L)-2$ </tex-math></inline-formula>\u0000 point-to-point codes designed for symmetric channels. This reduces the seemingly hard task of constructing good codes for C-RAN’s to the much better understood task of finding good codes for single-user channels. To show this result, an equivalence between the achievable rate-fronthaul regions of joint coding and successive coding is established. Then, rate-splitting and quantization-splitting techniques are used to show that the task of achieving a rate-fronthaul point in the joint coding region can be simplified to that of achieving a corner point in a higher-dimensional C-RAN problem. As a by-product, some interesting properties of the rate-fronthaul region of joint decoding for uplink C-RAN’s are also derived.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"572-584"},"PeriodicalIF":0.0,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142275002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article addresses errors in [1] . Equation (2) contained an error wherein x was not bold. It is corrected below.
本文解决了 [1] 中的错误。公式(2)中有一处错误，其中 x 没有加粗。现更正如下。
{"title":"Erratum to “LightVeriFL: A Lightweight and Verifiable Secure Aggregation for Federated Learning”","authors":"Baturalp Buyukates;Jinhyun So;Hessam Mahdavifar;Salman Avestimehr","doi":"10.1109/JSAIT.2024.3413928","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3413928","url":null,"abstract":"This article addresses errors in \u0000<xref>[1]</xref>\u0000. \u0000<xref>Equation (2)</xref>\u0000 contained an error wherein x was not bold. It is corrected below.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"570-571"},"PeriodicalIF":0.0,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10648286","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142077637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-04DOI: 10.1109/JSAIT.2024.3422011
Ahmed H. Salamah;Kaixiang Zheng;Linfeng Ye;En-Hui Yang
Conventional image compression techniques are primarily developed for the human visual system. However, with the extensive use of deep neural networks (DNNs) for computer vision, more and more images will be consumed by DNN-based intelligent machines, which makes it crucial to develop image compression techniques customized for DNN vision while being JPEG compliant. In this paper, we revisit the JPEG rate distortion theory for DNN vision. First, we propose a novel distortion measure, dubbed the sensitivity weighted error (SWE), for DNN vision. Second, we incorporate SWE into the soft decision quantization (SDQ) process of JPEG to trade SWE for rate. Finally, we develop an algorithm, called OptS, for designing optimal quantization tables for the luminance channel and chrominance channels, respectively. To test the performance of the resulting DNN-oriented compression framework and algorithm, experiments of image classification are conducted on the ImageNet dataset for four prevalent DNN models. Results demonstrate that our proposed framework and algorithm achieve better rate-accuracy (R-A) performance than the default JPEG. For some DNN models, our proposed framework and algorithm provide a significant reduction in the compression rate up to 67.84% with no accuracy loss compared to the default JPEG.
{"title":"JPEG Compliant Compression for DNN Vision","authors":"Ahmed H. Salamah;Kaixiang Zheng;Linfeng Ye;En-Hui Yang","doi":"10.1109/JSAIT.2024.3422011","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3422011","url":null,"abstract":"Conventional image compression techniques are primarily developed for the human visual system. However, with the extensive use of deep neural networks (DNNs) for computer vision, more and more images will be consumed by DNN-based intelligent machines, which makes it crucial to develop image compression techniques customized for DNN vision while being JPEG compliant. In this paper, we revisit the JPEG rate distortion theory for DNN vision. First, we propose a novel distortion measure, dubbed the sensitivity weighted error (SWE), for DNN vision. Second, we incorporate SWE into the soft decision quantization (SDQ) process of JPEG to trade SWE for rate. Finally, we develop an algorithm, called OptS, for designing optimal quantization tables for the luminance channel and chrominance channels, respectively. To test the performance of the resulting DNN-oriented compression framework and algorithm, experiments of image classification are conducted on the ImageNet dataset for four prevalent DNN models. Results demonstrate that our proposed framework and algorithm achieve better rate-accuracy (R-A) performance than the default JPEG. For some DNN models, our proposed framework and algorithm provide a significant reduction in the compression rate up to 67.84% with no accuracy loss compared to the default JPEG.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"520-533"},"PeriodicalIF":0.0,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141966001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-25DOI: 10.1109/JSAIT.2024.3419054
Yanyan Dong;Shenghao Yang;Jie Wang;Fan Cheng
Wireless communication links suffer from outage events caused by fading and interference. To facilitate a tractable analysis of network communication throughput and latency, we propose an outage link model to represent a communication link in the slow fading phenomenon. For a line-topology network with outage links, we study three types of intermediate network node schemes: random linear network coding, store-and-forward, and hop-by-hop retransmission. We provide the analytical formulas for the maximum throughputs and the end-to-end latency for each scheme. To gain a more explicit understanding, we perform a scalability analysis of the throughput and latency as the network length increases. We observe that the same order of throughput/latency holds across a wide range of outage functions for each scheme. We illustrate how our exact formulae and scalability results can be applied to compare different schemes.
{"title":"Throughput and Latency Analysis for Line Networks With Outage Links","authors":"Yanyan Dong;Shenghao Yang;Jie Wang;Fan Cheng","doi":"10.1109/JSAIT.2024.3419054","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3419054","url":null,"abstract":"Wireless communication links suffer from outage events caused by fading and interference. To facilitate a tractable analysis of network communication throughput and latency, we propose an outage link model to represent a communication link in the slow fading phenomenon. For a line-topology network with outage links, we study three types of intermediate network node schemes: random linear network coding, store-and-forward, and hop-by-hop retransmission. We provide the analytical formulas for the maximum throughputs and the end-to-end latency for each scheme. To gain a more explicit understanding, we perform a scalability analysis of the throughput and latency as the network length increases. We observe that the same order of throughput/latency holds across a wide range of outage functions for each scheme. We illustrate how our exact formulae and scalability results can be applied to compare different schemes.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"464-477"},"PeriodicalIF":0.0,"publicationDate":"2024-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10571545","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141966000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-19DOI: 10.1109/JSAIT.2024.3415670
Monica Welfert;Gowtham R. Kurri;Kyle Otstot;Lalitha Sankar
Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and f-GANs which minimize f-divergences. We also show that all symmetric f-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to $alpha $ -GANs, defined using $alpha $ -loss, a tunable CPE loss family parametrized by $alpha in (0,infty $ ]. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player’s objective using $alpha $ -loss to obtain $(alpha _{D},alpha _{G})$ -GANs. We show that the resulting non-zero sum game simplifies to minimizing an f-divergence under appropriate conditions on $(alpha _{D},alpha _{G})$ . Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning $(alpha _{D},alpha _{G})$ in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.
生成式对抗网络（GAN）被模拟为生成器（G）和判别器（D）之间的零和博弈，可以生成具有形式保证的合成数据。注意到 D 是一个分类器，我们首先使用类概率估计（CPE）损失重新表述了 GAN 的价值函数。我们证明了 CPE 损失 GAN 与 f-GAN 之间的双向对应关系，后者最大限度地减小了 f 分歧。我们还证明了所有对称的 f-divergences 在收敛性上是等价的。在有限样本和模型容量设置中，我们定义并获得了估计误差和泛化误差的界限。我们将这些结果专门应用于$alpha $ -GANs，使用$alpha $ -loss定义，$alpha in (0,infty $ ]是一个参数为$alpha in (0,infty $ ]的可调CPE损失族。接下来，我们引入了一类双目标 GAN，通过使用 $alpha $ -loss 对每个参与者的目标进行建模，得到 $(alpha _{D},alpha _{G})$ -GAN，从而解决 GAN 的训练不稳定性问题。我们证明，由此产生的非零和博弈在 $(alpha _{D},alpha _{G})$ 的适当条件下简化为最小化 f-发散。通过使用 CPE 损失对这一双目标表述进行推广，我们定义并获得了适当定义的估计误差上限。最后，我们强调了调整 $(alpha _{D},alpha _{G})$ 在缓解合成二维高斯混合环以及大型公开 Celeb-A 和 LSUN 课堂图像数据集的训练不稳定性方面的价值。
{"title":"Addressing GAN Training Instabilities via Tunable Classification Losses","authors":"Monica Welfert;Gowtham R. Kurri;Kyle Otstot;Lalitha Sankar","doi":"10.1109/JSAIT.2024.3415670","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3415670","url":null,"abstract":"Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and f-GANs which minimize f-divergences. We also show that all symmetric f-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to \u0000<inline-formula> <tex-math>$alpha $ </tex-math></inline-formula>\u0000-GANs, defined using \u0000<inline-formula> <tex-math>$alpha $ </tex-math></inline-formula>\u0000-loss, a tunable CPE loss family parametrized by \u0000<inline-formula> <tex-math>$alpha in (0,infty $ </tex-math></inline-formula>\u0000]. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player’s objective using \u0000<inline-formula> <tex-math>$alpha $ </tex-math></inline-formula>\u0000-loss to obtain \u0000<inline-formula> <tex-math>$(alpha _{D},alpha _{G})$ </tex-math></inline-formula>\u0000-GANs. We show that the resulting non-zero sum game simplifies to minimizing an f-divergence under appropriate conditions on \u0000<inline-formula> <tex-math>$(alpha _{D},alpha _{G})$ </tex-math></inline-formula>\u0000. Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning \u0000<inline-formula> <tex-math>$(alpha _{D},alpha _{G})$ </tex-math></inline-formula>\u0000 in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"534-553"},"PeriodicalIF":0.0,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
While much of the rapidly growing literature on fair decision-making focuses on metrics for one-shot decisions, recent work has raised the intriguing possibility of designing sequential decision-making to positively impact long-term social fairness. In selection processes such as college admissions or hiring, biasing slightly towards applicants from under-represented groups is hypothesized to provide positive feedback that increases the pool of under-represented applicants in future selection rounds, thus enhancing fairness in the long term. In this paper, we examine this hypothesis and its consequences in a setting in which multiple agents are selecting from a common pool of applicants. We propose the Multi-agent Fair-Greedy policy, that balances greedy score maximization and fairness. Under this policy, we prove that the resource pool and the admissions converge to a long-term fairness target set by the agents when the score distributions across the groups in the population are identical. We provide empirical evidence of existence of equilibria under non-identical score distributions through synthetic and adapted real-world datasets. We then sound a cautionary note for more complex applicant pool evolution models, under which uncoordinated behavior by the agents can cause negative reinforcement, leading to a reduction in the fraction of under-represented applicants. Our results indicate that, while positive reinforcement is a promising mechanism for long-term fairness, policies must be designed carefully to be robust to variations in the evolution model, with a number of open issues that remain to be explored by algorithm designers, social scientists, and policymakers.
{"title":"Long-Term Fairness in Sequential Multi-Agent Selection With Positive Reinforcement","authors":"Bhagyashree Puranik;Ozgur Guldogan;Upamanyu Madhow;Ramtin Pedarsani","doi":"10.1109/JSAIT.2024.3416078","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3416078","url":null,"abstract":"While much of the rapidly growing literature on fair decision-making focuses on metrics for one-shot decisions, recent work has raised the intriguing possibility of designing sequential decision-making to positively impact long-term social fairness. In selection processes such as college admissions or hiring, biasing slightly towards applicants from under-represented groups is hypothesized to provide positive feedback that increases the pool of under-represented applicants in future selection rounds, thus enhancing fairness in the long term. In this paper, we examine this hypothesis and its consequences in a setting in which multiple agents are selecting from a common pool of applicants. We propose the Multi-agent Fair-Greedy policy, that balances greedy score maximization and fairness. Under this policy, we prove that the resource pool and the admissions converge to a long-term fairness target set by the agents when the score distributions across the groups in the population are identical. We provide empirical evidence of existence of equilibria under non-identical score distributions through synthetic and adapted real-world datasets. We then sound a cautionary note for more complex applicant pool evolution models, under which uncoordinated behavior by the agents can cause negative reinforcement, leading to a reduction in the fraction of under-represented applicants. Our results indicate that, while positive reinforcement is a promising mechanism for long-term fairness, policies must be designed carefully to be robust to variations in the evolution model, with a number of open issues that remain to be explored by algorithm designers, social scientists, and policymakers.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"424-441"},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141624148","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-18DOI: 10.1109/JSAIT.2024.3416089
Shahrzad Kiani;Franziska Boenisch;Stark C. Draper
Federated Learning (FL) is the standard protocol for collaborative learning. In FL, multiple workers jointly train a shared model. They exchange model updates calculated on their data, while keeping the raw data itself local. Since workers naturally form groups based on common interests and privacy policies, we are motivated to extend standard FL to reflect a setting with multiple, potentially overlapping groups. In this setup where workers can belong and contribute to more than one group at a time, complexities arise in understanding privacy leakage and in adhering to privacy policies. To address the challenges, we propose differential private overlapping grouped learning (DP-OGL), a novel method to implement privacy guarantees within overlapping groups. Under the honest-but-curious threat model, we derive novel privacy guarantees between arbitrary pairs of workers. These privacy guarantees describe and quantify two key effects of privacy leakage in DP-OGL: propagation delay, i.e., the fact that information from one group will leak to other groups only with temporal offset through the common workers and information degradation, i.e., the fact that noise addition over model updates limits information leakage between workers. Our experiments show that applying DP-OGL enhances utility while maintaining strong privacy compared to standard FL setups.
{"title":"Controlled Privacy Leakage Propagation Throughout Overlapping Grouped Learning","authors":"Shahrzad Kiani;Franziska Boenisch;Stark C. Draper","doi":"10.1109/JSAIT.2024.3416089","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3416089","url":null,"abstract":"Federated Learning (FL) is the standard protocol for collaborative learning. In FL, multiple workers jointly train a shared model. They exchange model updates calculated on their data, while keeping the raw data itself local. Since workers naturally form groups based on common interests and privacy policies, we are motivated to extend standard FL to reflect a setting with multiple, potentially overlapping groups. In this setup where workers can belong and contribute to more than one group at a time, complexities arise in understanding privacy leakage and in adhering to privacy policies. To address the challenges, we propose differential private overlapping grouped learning (DP-OGL), a novel method to implement privacy guarantees within overlapping groups. Under the honest-but-curious threat model, we derive novel privacy guarantees between arbitrary pairs of workers. These privacy guarantees describe and quantify two key effects of privacy leakage in DP-OGL: propagation delay, i.e., the fact that information from one group will leak to other groups only with temporal offset through the common workers and information degradation, i.e., the fact that noise addition over model updates limits information leakage between workers. Our experiments show that applying DP-OGL enhances utility while maintaining strong privacy compared to standard FL setups.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"442-463"},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141624126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We consider a line network of nodes, connected by additive white noise channels, equipped with local feedback. We study the velocity at which information spreads over this network. For transmission of a data packet, we give an explicit positive lower bound on the velocity, for any packet size. Furthermore, we consider streaming, that is, transmission of data packets generated at a given average arrival rate. We show that a positive velocity exists as long as the arrival rate is below the individual Gaussian channel capacity, and provide an explicit lower bound. Our analysis involves applying pulse-amplitude modulation to the data (successively in the streaming case), and using linear mean-squared error estimation at the network nodes. For general white noise, we derive exponential error-probability bounds. For single-packet transmission over channels with (sub-)Gaussian noise, we show a doubly-exponential behavior, which reduces to the celebrated Schalkwijk–Kailath scheme when considering a single node. Viewing the constellation as an “analog source”, we also provide bounds on the exponential decay of the mean-squared error of source transmission over the network.
{"title":"Information Velocity of Cascaded Gaussian Channels With Feedback","authors":"Elad Domanovitz;Anatoly Khina;Tal Philosof;Yuval Kochman","doi":"10.1109/JSAIT.2024.3416310","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3416310","url":null,"abstract":"We consider a line network of nodes, connected by additive white noise channels, equipped with local feedback. We study the velocity at which information spreads over this network. For transmission of a data packet, we give an explicit positive lower bound on the velocity, for any packet size. Furthermore, we consider streaming, that is, transmission of data packets generated at a given average arrival rate. We show that a positive velocity exists as long as the arrival rate is below the individual Gaussian channel capacity, and provide an explicit lower bound. Our analysis involves applying pulse-amplitude modulation to the data (successively in the streaming case), and using linear mean-squared error estimation at the network nodes. For general white noise, we derive exponential error-probability bounds. For single-packet transmission over channels with (sub-)Gaussian noise, we show a doubly-exponential behavior, which reduces to the celebrated Schalkwijk–Kailath scheme when considering a single node. Viewing the constellation as an “analog source”, we also provide bounds on the exponential decay of the mean-squared error of source transmission over the network.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"554-569"},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141980053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：