Pub Date : 2024-03-09DOI: 10.1109/JSAIT.2024.3397305
Shahab Asoodeh;Huanyu Zhang
We investigate the contraction properties of locally differentially private mechanisms. More specifically, we derive tight upper bounds on the divergence between $P{mathsf K}$ and $Q{mathsf K}$ output distributions of an $varepsilon $ -LDP mechanism $mathsf K$ in terms of a divergence between the corresponding input distributions P and Q, respectively. Our first main technical result presents a sharp upper bound on the $chi ^{2}$ -divergence $chi ^{2}(P{mathsf K}|Q{mathsf K})$ in terms of $chi ^{2}(P|Q)$ and $varepsilon $ . We also show that the same result holds for a large family of divergences, including KL-divergence and squared Hellinger distance. The second main technical result gives an upper bound on $chi ^{2}(P{mathsf K}|Q{mathsf K})$ in terms of total variation distance ${textsf {TV}}(P, Q)$ and $varepsilon $ . We then utilize these bounds to establish locally private versions of the van Trees inequality, Le Cam’s, Assouad’s, and the mutual information methods —powerful tools for bounding minimax estimation risks. These results are shown to lead to tighter privacy analyses than the state-of-the-arts in several statistical problems such as entropy and discrete distribution estimation, non-parametric density estimation, and hypothesis testing.
{"title":"Contraction of Locally Differentially Private Mechanisms","authors":"Shahab Asoodeh;Huanyu Zhang","doi":"10.1109/JSAIT.2024.3397305","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3397305","url":null,"abstract":"We investigate the contraction properties of locally differentially private mechanisms. More specifically, we derive tight upper bounds on the divergence between \u0000<inline-formula> <tex-math>$P{mathsf K}$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$Q{mathsf K}$ </tex-math></inline-formula>\u0000 output distributions of an \u0000<inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>\u0000-LDP mechanism \u0000<inline-formula> <tex-math>$mathsf K$ </tex-math></inline-formula>\u0000 in terms of a divergence between the corresponding input distributions P and Q, respectively. Our first main technical result presents a sharp upper bound on the \u0000<inline-formula> <tex-math>$chi ^{2}$ </tex-math></inline-formula>\u0000-divergence \u0000<inline-formula> <tex-math>$chi ^{2}(P{mathsf K}|Q{mathsf K})$ </tex-math></inline-formula>\u0000 in terms of \u0000<inline-formula> <tex-math>$chi ^{2}(P|Q)$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>\u0000. We also show that the same result holds for a large family of divergences, including KL-divergence and squared Hellinger distance. The second main technical result gives an upper bound on \u0000<inline-formula> <tex-math>$chi ^{2}(P{mathsf K}|Q{mathsf K})$ </tex-math></inline-formula>\u0000 in terms of total variation distance \u0000<inline-formula> <tex-math>${textsf {TV}}(P, Q)$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>\u0000. We then utilize these bounds to establish locally private versions of the van Trees inequality, Le Cam’s, Assouad’s, and the mutual information methods —powerful tools for bounding minimax estimation risks. These results are shown to lead to tighter privacy analyses than the state-of-the-arts in several statistical problems such as entropy and discrete distribution estimation, non-parametric density estimation, and hypothesis testing.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"385-395"},"PeriodicalIF":0.0,"publicationDate":"2024-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141494817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-09DOI: 10.1109/JSAIT.2024.3397741
Lucas Monteiro Paes;Ananda Theertha Suresh;Alex Beutel;Flavio P. Calmon;Ahmad Beirami
Machine learning (ML) models used in prediction and classification tasks may display performance disparities across population groups determined by sensitive attributes (e.g., race, sex, age). We consider the problem of evaluating the performance of a fixed ML model across population groups defined by multiple sensitive attributes (e.g., race and sex and age). Here, the sample complexity for estimating the worst-case performance gap across groups (e.g., the largest difference in error rates) increases exponentially with the number of group-denoting sensitive attributes. To address this issue, we propose an approach to test for performance disparities based on Conditional Value-at-Risk (CVaR). By allowing a small probabilistic slack on the groups over which a model has approximately equal performance, we show that the sample complexity required for discovering performance violations is reduced exponentially to be at most upper bounded by the square root of the number of groups. As a byproduct of our analysis, when the groups are weighted by a specific prior distribution, we show that Rényi entropy of order 2/3 of the prior distribution captures the sample complexity of the proposed CVaR test algorithm. Finally, we also show that there exists a non-i.i.d. data collection strategy that results in a sample complexity independent of the number of groups.
在预测和分类任务中使用的机器学习（ML）模型可能会在由敏感属性（如种族、性别、年龄）决定的人群中显示出性能差异。我们考虑的问题是，如何评估固定 ML 模型在由多个敏感属性（如种族、性别和年龄）决定的人群中的性能。在这种情况下，估计不同群体间最坏情况下的性能差距（例如，误差率的最大差异）的样本复杂度会随着群体敏感属性的数量呈指数增长。为了解决这个问题，我们提出了一种基于条件风险值（CVaR）的性能差距测试方法。通过允许模型在性能大致相同的组别上有较小的概率松弛，我们证明发现性能违规所需的样本复杂度会以指数形式降低，最多为组别数量平方根的上限。作为我们分析的副产品，当各组由特定的先验分布加权时，我们证明先验分布的 2/3 阶雷尼熵可以捕捉到所提 CVaR 检验算法的样本复杂度。最后，我们还证明，存在一种非 i.i.d. 数据收集策略，其样本复杂度与组数无关。
{"title":"Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testing","authors":"Lucas Monteiro Paes;Ananda Theertha Suresh;Alex Beutel;Flavio P. Calmon;Ahmad Beirami","doi":"10.1109/JSAIT.2024.3397741","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3397741","url":null,"abstract":"Machine learning (ML) models used in prediction and classification tasks may display performance disparities across population groups determined by sensitive attributes (e.g., race, sex, age). We consider the problem of evaluating the performance of a fixed ML model across population groups defined by multiple sensitive attributes (e.g., race and sex and age). Here, the sample complexity for estimating the worst-case performance gap across groups (e.g., the largest difference in error rates) increases exponentially with the number of group-denoting sensitive attributes. To address this issue, we propose an approach to test for performance disparities based on Conditional Value-at-Risk (CVaR). By allowing a small probabilistic slack on the groups over which a model has approximately equal performance, we show that the sample complexity required for discovering performance violations is reduced exponentially to be at most upper bounded by the square root of the number of groups. As a byproduct of our analysis, when the groups are weighted by a specific prior distribution, we show that Rényi entropy of order 2/3 of the prior distribution captures the sample complexity of the proposed CVaR test algorithm. Finally, we also show that there exists a non-i.i.d. data collection strategy that results in a sample complexity independent of the number of groups.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"659-674"},"PeriodicalIF":0.0,"publicationDate":"2024-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142736343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3397187
Mark Beliaev;Payam Delgosha;Hamed Hassani;Ramtin Pedarsani
Over the past two decades, the rise in adoption of neural networks has surged in parallel with their performance. Concurrently, we have observed the inherent fragility of these prediction models: small changes to the inputs can induce classification errors across entire datasets. In the following study, we examine perturbations constrained by the $ell _{0}$ –norm, a potent attack model in the domains of computer vision, malware detection, and natural language processing. To combat this adversary, we introduce a novel defense technique comprised of two components: “truncation” and “adversarial training”. Subsequently, we conduct a theoretical analysis of the Gaussian mixture setting and establish the asymptotic optimality of our proposed defense. Based on this obtained insight, we broaden the application of our technique to neural networks. Lastly, we empirically validate our results in the domain of computer vision, demonstrating substantial enhancements in the robust classification error of neural networks.
{"title":"Efficient and Robust Classification for Sparse Attacks","authors":"Mark Beliaev;Payam Delgosha;Hamed Hassani;Ramtin Pedarsani","doi":"10.1109/JSAIT.2024.3397187","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3397187","url":null,"abstract":"Over the past two decades, the rise in adoption of neural networks has surged in parallel with their performance. Concurrently, we have observed the inherent fragility of these prediction models: small changes to the inputs can induce classification errors across entire datasets. In the following study, we examine perturbations constrained by the \u0000<inline-formula> <tex-math>$ell _{0}$ </tex-math></inline-formula>\u0000–norm, a potent attack model in the domains of computer vision, malware detection, and natural language processing. To combat this adversary, we introduce a novel defense technique comprised of two components: “truncation” and “adversarial training”. Subsequently, we conduct a theoretical analysis of the Gaussian mixture setting and establish the asymptotic optimality of our proposed defense. Based on this obtained insight, we broaden the application of our technique to neural networks. Lastly, we empirically validate our results in the domain of computer vision, demonstrating substantial enhancements in the robust classification error of neural networks.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"261-272"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3373595
Zirui Yan;Arpan Mukherjee;Burak Varıcı;Ali Tajer
The sequential design of experiments for optimizing a reward function in causal systems can be effectively modeled by the sequential design of interventions in causal bandits (CBs). In the existing literature on CBs, a critical assumption is that the causal models remain constant over time. However, this assumption does not necessarily hold in complex systems, which constantly undergo temporal model fluctuations. This paper addresses the robustness of CBs to such model fluctuations. The focus is on causal systems with linear structural equation models (SEMs). The SEMs and the time-varying pre- and post-interventional statistical models are all unknown. Cumulative regret is adopted as the design criteria, based on which the objective is to design a sequence of interventions that incur the smallest cumulative regret with respect to an oracle aware of the entire causal model and its fluctuations. First, it is established that the existing approaches fail to maintain regret sub-linearity with even a few instances of model deviation. Specifically, when the number of instances with model deviation is as few as $T^{frac {1}{2L}}$ , where $T$ is the time horizon and $L$ is the length of the longest causal path in the graph, the existing algorithms will have linear regret in $T$ . For instance, when $T=10^{5}$ and $L=3$ , model deviations in 6 out of 105 instances result in a linear regret. Next, a robust CB algorithm is designed, and its regret is analyzed, where upper and information-theoretic lower bounds on the regret are established. Specifically, in a graph with $N$ nodes and maximum degree $d$ , under a general measure of model deviation $C$ , the cumulative regret is upper bounded by $tilde {mathcal {O}}left({d^{L-{}frac {1}{2}}(sqrt {NT} + NC)}right)$ and lower bounded by $Omega left({d^{frac {L}{2}-2}max {sqrt {T};, ; d^{2}C}}right)$ . Comparing these bounds establishes that the proposed algorithm achieves nearly optimal $tilde{mathcal {O}} (sqrt {T})$ regret when $C$ is $o(sqrt {T})$ and maintains sub-linear regret for a broader range of $C$ .
{"title":"Robust Causal Bandits for Linear Models","authors":"Zirui Yan;Arpan Mukherjee;Burak Varıcı;Ali Tajer","doi":"10.1109/JSAIT.2024.3373595","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3373595","url":null,"abstract":"The sequential design of experiments for optimizing a reward function in causal systems can be effectively modeled by the sequential design of interventions in causal bandits (CBs). In the existing literature on CBs, a critical assumption is that the causal models remain constant over time. However, this assumption does not necessarily hold in complex systems, which constantly undergo temporal model fluctuations. This paper addresses the robustness of CBs to such model fluctuations. The focus is on causal systems with linear structural equation models (SEMs). The SEMs and the time-varying pre- and post-interventional statistical models are all unknown. Cumulative regret is adopted as the design criteria, based on which the objective is to design a sequence of interventions that incur the smallest cumulative regret with respect to an oracle aware of the entire causal model and its fluctuations. First, it is established that the existing approaches fail to maintain regret sub-linearity with even a few instances of model deviation. Specifically, when the number of instances with model deviation is as few as \u0000<inline-formula> <tex-math>$T^{frac {1}{2L}}$ </tex-math></inline-formula>\u0000, where \u0000<inline-formula> <tex-math>$T$ </tex-math></inline-formula>\u0000 is the time horizon and \u0000<inline-formula> <tex-math>$L$ </tex-math></inline-formula>\u0000 is the length of the longest causal path in the graph, the existing algorithms will have linear regret in \u0000<inline-formula> <tex-math>$T$ </tex-math></inline-formula>\u0000. For instance, when \u0000<inline-formula> <tex-math>$T=10^{5}$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$L=3$ </tex-math></inline-formula>\u0000, model deviations in 6 out of 105 instances result in a linear regret. Next, a robust CB algorithm is designed, and its regret is analyzed, where upper and information-theoretic lower bounds on the regret are established. Specifically, in a graph with \u0000<inline-formula> <tex-math>$N$ </tex-math></inline-formula>\u0000 nodes and maximum degree \u0000<inline-formula> <tex-math>$d$ </tex-math></inline-formula>\u0000, under a general measure of model deviation \u0000<inline-formula> <tex-math>$C$ </tex-math></inline-formula>\u0000, the cumulative regret is upper bounded by \u0000<inline-formula> <tex-math>$tilde {mathcal {O}}left({d^{L-{}frac {1}{2}}(sqrt {NT} + NC)}right)$ </tex-math></inline-formula>\u0000 and lower bounded by \u0000<inline-formula> <tex-math>$Omega left({d^{frac {L}{2}-2}max {sqrt {T};, ; d^{2}C}}right)$ </tex-math></inline-formula>\u0000. Comparing these bounds establishes that the proposed algorithm achieves nearly optimal \u0000<inline-formula> <tex-math>$tilde{mathcal {O}} (sqrt {T})$ </tex-math></inline-formula>\u0000 regret when \u0000<inline-formula> <tex-math>$C$ </tex-math></inline-formula>\u0000 is \u0000<inline-formula> <tex-math>$o(sqrt {T})$ </tex-math></inline-formula>\u0000 and maintains sub-linear regret for a broader range of \u0000<inline-formula> <tex-math>$C$ </tex-math></inline-formula>\u0000.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"78-93"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140605970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3396400
Mohamed Seif;Yanxi Chen;Andrea J. Goldsmith;H. Vincent Poor
We study the community detection problem over binary symmetric stochastic block models (SBMs) while preserving the privacy of the individual connections between the vertices. We present and analyze the associated information-theoretic tradeoff for differentially private exact recovery of the underlying communities by deriving sufficient separation conditions between the intra-community and inter-community connection probabilities while taking into account the privacy budget and graph sketching as a speed-up technique to improve the computational complexity of maximum likelihood (ML) based recovery algorithms.
{"title":"Differentially Private Sketch-and-Solve for Community Detection via Semidefinite Programming","authors":"Mohamed Seif;Yanxi Chen;Andrea J. Goldsmith;H. Vincent Poor","doi":"10.1109/JSAIT.2024.3396400","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3396400","url":null,"abstract":"We study the community detection problem over binary symmetric stochastic block models (SBMs) while preserving the privacy of the individual connections between the vertices. We present and analyze the associated information-theoretic tradeoff for differentially private exact recovery of the underlying communities by deriving sufficient separation conditions between the intra-community and inter-community connection probabilities while taking into account the privacy budget and graph sketching as a speed-up technique to improve the computational complexity of maximum likelihood (ML) based recovery algorithms.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"331-346"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141251112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3396787
Banghua Zhu;Ziao Wang;Nadim Ghaddar;Jiantao Jiao;Lele Wang
We consider the problem of computing a function of n variables using noisy queries, where each query is incorrect with some fixed and known probability $p in (0,1/2)$ . Specifically, we consider the computation of the $textsf {OR}$ function of n bits (where queries correspond to noisy readings of the bits) and the $textsf {MAX}$ function of n real numbers (where queries correspond to noisy pairwise comparisons). We show that an expected number of queries of $(1 pm o(1)) {}frac {nlog {}frac {1}{delta }}{D_{textsf {KL}}(p | 1-p)}$ is both sufficient and necessary to compute both functions with a vanishing error probability $delta = o(1)$ , where $D_{textsf {KL}}(p | 1-p)$ denotes the Kullback-Leibler divergence between $textsf {Bern}(p)$ and $textsf {Bern}(1-p)$ distributions. Compared to previous work, our results tighten the dependence on p in both the upper and lower bounds for the two functions.
我们考虑的问题是使用噪声查询计算 n 个变量的函数，其中每个查询都是不正确的，其概率是固定且已知的 $p in (0,1/2)$。具体来说，我们考虑计算 n 个比特的 $textsf {OR}$ 函数（其中查询对应于比特的噪声读数）和 n 个实数的 $textsf {MAX}$ 函数（其中查询对应于噪声成对比较）。我们证明，$(1 pm o(1)) {}frac {nlog {}frac {1}{delta }}{D_{textsf {KL}}(p | 1-p)}$ 的预期查询次数对于以消失的错误概率 $delta = o(1)$ 计算这两个函数来说既充分又必要、其中，$D_{textsf {KL}}(p | 1-p)$ 表示 $textsf {Bern}(p)$ 和 $textsf {Bern}(1-p)$ 分布之间的库尔贝-莱布勒发散。与之前的工作相比，我们的结果在两个函数的上界和下界中都加强了对 p 的依赖。
{"title":"Noisy Computing of the OR and MAX Functions","authors":"Banghua Zhu;Ziao Wang;Nadim Ghaddar;Jiantao Jiao;Lele Wang","doi":"10.1109/JSAIT.2024.3396787","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3396787","url":null,"abstract":"We consider the problem of computing a function of n variables using noisy queries, where each query is incorrect with some fixed and known probability \u0000<inline-formula> <tex-math>$p in (0,1/2)$ </tex-math></inline-formula>\u0000. Specifically, we consider the computation of the \u0000<inline-formula> <tex-math>$textsf {OR}$ </tex-math></inline-formula>\u0000 function of n bits (where queries correspond to noisy readings of the bits) and the \u0000<inline-formula> <tex-math>$textsf {MAX}$ </tex-math></inline-formula>\u0000 function of n real numbers (where queries correspond to noisy pairwise comparisons). We show that an expected number of queries of \u0000<inline-formula> <tex-math>$(1 pm o(1)) {}frac {nlog {}frac {1}{delta }}{D_{textsf {KL}}(p | 1-p)}$ </tex-math></inline-formula>\u0000 is both sufficient and necessary to compute both functions with a vanishing error probability \u0000<inline-formula> <tex-math>$delta = o(1)$ </tex-math></inline-formula>\u0000, where \u0000<inline-formula> <tex-math>$D_{textsf {KL}}(p | 1-p)$ </tex-math></inline-formula>\u0000 denotes the Kullback-Leibler divergence between \u0000<inline-formula> <tex-math>$textsf {Bern}(p)$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$textsf {Bern}(1-p)$ </tex-math></inline-formula>\u0000 distributions. Compared to previous work, our results tighten the dependence on p in both the upper and lower bounds for the two functions.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"302-313"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141084767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3396079
Ruizhi Zhang
Detection of sparse signals arises in many modern applications such as signal processing, bioinformatics, finance, and disease surveillance. However, in many of these applications, the data may contain sensitive personal information, which is desirable to be protected during the data analysis. In this article, we consider the problem of $(epsilon,delta)$ -differentially private detection of a general sparse mixture with a focus on how privacy affects the detection power. By investigating the nonasymptotic upper bound for the summation of error probabilities, we find any $(epsilon,delta)$ -differentially private test cannot detect the sparse signal if the privacy constraint is too strong or if the model parameters are in the undetectable region (Cai and Wu, 2014). Moreover, we study the private clamped log-likelihood ratio test proposed by Canonne et al., 2019 and show it achieves vanishing error probabilities in some conditions on the model parameters and privacy parameters. Then, for the case when the null distribution is a standard normal distribution, we propose an adaptive $(epsilon,delta)$ -differentially private test, which achieves vanishing error probabilities in the same detectable region (Cai and Wu, 2014) when the privacy parameters satisfy certain sufficient conditions. Several numerical experiments are conducted to verify our theoretical results and illustrate the performance of our proposed test.
稀疏信号的检测在信号处理、生物信息学、金融和疾病监测等许多现代应用中都会出现。然而，在许多此类应用中，数据可能包含敏感的个人信息，这就需要在数据分析过程中加以保护。在本文中，我们考虑了一般稀疏混合物的$(epsilon,delta)$差异隐私检测问题，重点关注隐私如何影响检测能力。通过研究误差概率求和的非渐近上界，我们发现如果隐私约束太强或模型参数处于不可检测区域，任何$(epsilon,delta)$-差异隐私检测都无法检测到稀疏信号（Cai and Wu，2014）。此外，我们还研究了 Canonne 等人 2019 年提出的私有钳位对数似然比检验，结果表明它在模型参数和隐私参数的某些条件下实现了虚化误差概率。然后，对于空分布是标准正态分布的情况，我们提出了一种自适应的$(epsilon,delta)$-差异私有检验，当隐私参数满足某些充分条件时，它在相同的可检测区域内实现了消失的误差概率（Cai and Wu, 2014）。我们进行了一些数值实验来验证我们的理论结果，并说明我们提出的测试的性能。
{"title":"Detection of Sparse Mixtures With Differential Privacy","authors":"Ruizhi Zhang","doi":"10.1109/JSAIT.2024.3396079","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3396079","url":null,"abstract":"Detection of sparse signals arises in many modern applications such as signal processing, bioinformatics, finance, and disease surveillance. However, in many of these applications, the data may contain sensitive personal information, which is desirable to be protected during the data analysis. In this article, we consider the problem of \u0000<inline-formula> <tex-math>$(epsilon,delta)$ </tex-math></inline-formula>\u0000-differentially private detection of a general sparse mixture with a focus on how privacy affects the detection power. By investigating the nonasymptotic upper bound for the summation of error probabilities, we find any \u0000<inline-formula> <tex-math>$(epsilon,delta)$ </tex-math></inline-formula>\u0000-differentially private test cannot detect the sparse signal if the privacy constraint is too strong or if the model parameters are in the undetectable region (Cai and Wu, 2014). Moreover, we study the private clamped log-likelihood ratio test proposed by Canonne et al., 2019 and show it achieves vanishing error probabilities in some conditions on the model parameters and privacy parameters. Then, for the case when the null distribution is a standard normal distribution, we propose an adaptive \u0000<inline-formula> <tex-math>$(epsilon,delta)$ </tex-math></inline-formula>\u0000-differentially private test, which achieves vanishing error probabilities in the same detectable region (Cai and Wu, 2014) when the privacy parameters satisfy certain sufficient conditions. Several numerical experiments are conducted to verify our theoretical results and illustrate the performance of our proposed test.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"347-356"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141435289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-27DOI: 10.1109/JSAIT.2024.3370699
Chen Xu;Jonghyeok Lee;Xiuyuan Cheng;Yao Xie
We present a computationally efficient framework, called FlowDRO , for solving flow-based distributionally robust optimization (DRO) problems with Wasserstein uncertainty sets while aiming to find continuous worst-case distribution (also called the Least Favorable Distribution, LFD) and sample from it. The requirement for LFD to be continuous is so that the algorithm can be scalable to problems with larger sample sizes and achieve better generalization capability for the induced robust algorithms. To tackle the computationally challenging infinitely dimensional optimization problem, we leverage flow-based models and continuous-time invertible transport maps between the data distribution and the target distribution and develop a Wasserstein proximal gradient flow type algorithm. In theory, we establish the equivalence of the solution by optimal transport map to the original formulation, as well as the dual form of the problem through Wasserstein calculus and Brenier theorem. In practice, we parameterize the transport maps by a sequence of neural networks progressively trained in blocks by gradient descent. We demonstrate its usage in adversarial learning, distributionally robust hypothesis testing, and a new mechanism for data-driven distribution perturbation differential privacy, where the proposed method gives strong empirical performance on high-dimensional real data.
{"title":"Flow-Based Distributionally Robust Optimization","authors":"Chen Xu;Jonghyeok Lee;Xiuyuan Cheng;Yao Xie","doi":"10.1109/JSAIT.2024.3370699","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3370699","url":null,"abstract":"We present a computationally efficient framework, called \u0000<monospace>FlowDRO</monospace>\u0000, for solving flow-based distributionally robust optimization (DRO) problems with Wasserstein uncertainty sets while aiming to find continuous worst-case distribution (also called the Least Favorable Distribution, LFD) and sample from it. The requirement for LFD to be continuous is so that the algorithm can be scalable to problems with larger sample sizes and achieve better generalization capability for the induced robust algorithms. To tackle the computationally challenging infinitely dimensional optimization problem, we leverage flow-based models and continuous-time invertible transport maps between the data distribution and the target distribution and develop a Wasserstein proximal gradient flow type algorithm. In theory, we establish the equivalence of the solution by optimal transport map to the original formulation, as well as the dual form of the problem through Wasserstein calculus and Brenier theorem. In practice, we parameterize the transport maps by a sequence of neural networks progressively trained in blocks by gradient descent. We demonstrate its usage in adversarial learning, distributionally robust hypothesis testing, and a new mechanism for data-driven distribution perturbation differential privacy, where the proposed method gives strong empirical performance on high-dimensional real data.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"62-77"},"PeriodicalIF":0.0,"publicationDate":"2024-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1109/JSAIT.2024.3368229
Matteo Zecchin;Sangwoo Park;Osvaldo Simeone
In many real-world problems, predictions are leveraged to monitor and control cyber-physical systems, demanding guarantees on the satisfaction of reliability and safety requirements. However, predictions are inherently uncertain, and managing prediction uncertainty presents significant challenges in environments characterized by complex dynamics and forking trajectories. In this work, we assume access to a pre-designed probabilistic implicit or explicit sequence model, which may have been obtained using model-based or model-free methods. We introduce probabilistic time series-conformal risk prediction (PTS-CRC), a novel post-hoc calibration procedure that operates on the predictions produced by any pre-designed probabilistic forecaster to yield reliable error bars. In contrast to existing art, PTS-CRC produces predictive sets based on an ensemble of multiple prototype trajectories sampled from the sequence model, supporting the efficient representation of forking uncertainties. Furthermore, unlike the state of the art, PTS-CRC can satisfy reliability definitions beyond coverage. This property is leveraged to devise a novel model predictive control (MPC) framework that addresses open-loop and closed-loop control problems under general average constraints on the quality or safety of the control policy. We experimentally validate the performance of PTS-CRC prediction and control by studying a number of use cases in the context of wireless networking. Across all the considered tasks, PTS-CRC predictors are shown to provide more informative predictive sets, as well as safe control policies with larger returns.
{"title":"Forking Uncertainties: Reliable Prediction and Model Predictive Control With Sequence Models via Conformal Risk Control","authors":"Matteo Zecchin;Sangwoo Park;Osvaldo Simeone","doi":"10.1109/JSAIT.2024.3368229","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3368229","url":null,"abstract":"In many real-world problems, predictions are leveraged to monitor and control cyber-physical systems, demanding guarantees on the satisfaction of reliability and safety requirements. However, predictions are inherently uncertain, and managing prediction uncertainty presents significant challenges in environments characterized by complex dynamics and forking trajectories. In this work, we assume access to a pre-designed probabilistic implicit or explicit sequence model, which may have been obtained using model-based or model-free methods. We introduce probabilistic time series-conformal risk prediction (PTS-CRC), a novel post-hoc calibration procedure that operates on the predictions produced by any pre-designed probabilistic forecaster to yield reliable error bars. In contrast to existing art, PTS-CRC produces predictive sets based on an ensemble of multiple prototype trajectories sampled from the sequence model, supporting the efficient representation of forking uncertainties. Furthermore, unlike the state of the art, PTS-CRC can satisfy reliability definitions beyond coverage. This property is leveraged to devise a novel model predictive control (MPC) framework that addresses open-loop and closed-loop control problems under general average constraints on the quality or safety of the control policy. We experimentally validate the performance of PTS-CRC prediction and control by studying a number of use cases in the context of wireless networking. Across all the considered tasks, PTS-CRC predictors are shown to provide more informative predictive sets, as well as safe control policies with larger returns.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"44-61"},"PeriodicalIF":0.0,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-22DOI: 10.1109/JSAIT.2024.3366086
Anand Jerry George;Lekshmi Ramesh;Aditya Vikram Singh;Himanshu Tyagi
We consider the problem of continually releasing an estimate of the population mean of a stream of samples that is user-level differentially private (DP). At each time instant, a user contributes a sample, and the users can arrive in arbitrary order. Until now these requirements of continual release and user-level privacy were considered in isolation. But, in practice, both these requirements come together as the users often contribute data repeatedly and multiple queries are made. We provide an algorithm that outputs a mean estimate at every time instant t such that the overall release is user-level $varepsilon $ -DP and has the following error guarantee: Denoting by $m_{t}$ the maximum number of samples contributed by a user, as long as $tilde {Omega }(1/varepsilon)$ users have $m_{t}/2$ samples each, the error at time t is $tilde {O}(1/sqrt {t}+sqrt {m}_{t}/tvarepsilon)$ . This is a universal error guarantee which is valid for all arrival patterns of the users. Furthermore, it (almost) matches the existing lower bounds for the single-release setting at all time instants when users have contributed equal number of samples.
{"title":"Continual Mean Estimation Under User-Level Privacy","authors":"Anand Jerry George;Lekshmi Ramesh;Aditya Vikram Singh;Himanshu Tyagi","doi":"10.1109/JSAIT.2024.3366086","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3366086","url":null,"abstract":"We consider the problem of continually releasing an estimate of the population mean of a stream of samples that is user-level differentially private (DP). At each time instant, a user contributes a sample, and the users can arrive in arbitrary order. Until now these requirements of continual release and user-level privacy were considered in isolation. But, in practice, both these requirements come together as the users often contribute data repeatedly and multiple queries are made. We provide an algorithm that outputs a mean estimate at every time instant t such that the overall release is user-level \u0000<inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>\u0000-DP and has the following error guarantee: Denoting by \u0000<inline-formula> <tex-math>$m_{t}$ </tex-math></inline-formula>\u0000 the maximum number of samples contributed by a user, as long as \u0000<inline-formula> <tex-math>$tilde {Omega }(1/varepsilon)$ </tex-math></inline-formula>\u0000 users have \u0000<inline-formula> <tex-math>$m_{t}/2$ </tex-math></inline-formula>\u0000 samples each, the error at time t is \u0000<inline-formula> <tex-math>$tilde {O}(1/sqrt {t}+sqrt {m}_{t}/tvarepsilon)$ </tex-math></inline-formula>\u0000. This is a universal error guarantee which is valid for all arrival patterns of the users. Furthermore, it (almost) matches the existing lower bounds for the single-release setting at all time instants when users have contributed equal number of samples.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"28-43"},"PeriodicalIF":0.0,"publicationDate":"2024-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：