Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3397187
Mark Beliaev;Payam Delgosha;Hamed Hassani;Ramtin Pedarsani
Over the past two decades, the rise in adoption of neural networks has surged in parallel with their performance. Concurrently, we have observed the inherent fragility of these prediction models: small changes to the inputs can induce classification errors across entire datasets. In the following study, we examine perturbations constrained by the $ell _{0}$ –norm, a potent attack model in the domains of computer vision, malware detection, and natural language processing. To combat this adversary, we introduce a novel defense technique comprised of two components: “truncation” and “adversarial training”. Subsequently, we conduct a theoretical analysis of the Gaussian mixture setting and establish the asymptotic optimality of our proposed defense. Based on this obtained insight, we broaden the application of our technique to neural networks. Lastly, we empirically validate our results in the domain of computer vision, demonstrating substantial enhancements in the robust classification error of neural networks.
{"title":"Efficient and Robust Classification for Sparse Attacks","authors":"Mark Beliaev;Payam Delgosha;Hamed Hassani;Ramtin Pedarsani","doi":"10.1109/JSAIT.2024.3397187","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3397187","url":null,"abstract":"Over the past two decades, the rise in adoption of neural networks has surged in parallel with their performance. Concurrently, we have observed the inherent fragility of these prediction models: small changes to the inputs can induce classification errors across entire datasets. In the following study, we examine perturbations constrained by the \u0000<inline-formula> <tex-math>$ell _{0}$ </tex-math></inline-formula>\u0000–norm, a potent attack model in the domains of computer vision, malware detection, and natural language processing. To combat this adversary, we introduce a novel defense technique comprised of two components: “truncation” and “adversarial training”. Subsequently, we conduct a theoretical analysis of the Gaussian mixture setting and establish the asymptotic optimality of our proposed defense. Based on this obtained insight, we broaden the application of our technique to neural networks. Lastly, we empirically validate our results in the domain of computer vision, demonstrating substantial enhancements in the robust classification error of neural networks.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"261-272"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3373595
Zirui Yan;Arpan Mukherjee;Burak Varıcı;Ali Tajer
The sequential design of experiments for optimizing a reward function in causal systems can be effectively modeled by the sequential design of interventions in causal bandits (CBs). In the existing literature on CBs, a critical assumption is that the causal models remain constant over time. However, this assumption does not necessarily hold in complex systems, which constantly undergo temporal model fluctuations. This paper addresses the robustness of CBs to such model fluctuations. The focus is on causal systems with linear structural equation models (SEMs). The SEMs and the time-varying pre- and post-interventional statistical models are all unknown. Cumulative regret is adopted as the design criteria, based on which the objective is to design a sequence of interventions that incur the smallest cumulative regret with respect to an oracle aware of the entire causal model and its fluctuations. First, it is established that the existing approaches fail to maintain regret sub-linearity with even a few instances of model deviation. Specifically, when the number of instances with model deviation is as few as $T^{frac {1}{2L}}$ , where $T$ is the time horizon and $L$ is the length of the longest causal path in the graph, the existing algorithms will have linear regret in $T$ . For instance, when $T=10^{5}$ and $L=3$ , model deviations in 6 out of 105 instances result in a linear regret. Next, a robust CB algorithm is designed, and its regret is analyzed, where upper and information-theoretic lower bounds on the regret are established. Specifically, in a graph with $N$ nodes and maximum degree $d$ , under a general measure of model deviation $C$ , the cumulative regret is upper bounded by $tilde {mathcal {O}}left({d^{L-{}frac {1}{2}}(sqrt {NT} + NC)}right)$ and lower bounded by $Omega left({d^{frac {L}{2}-2}max {sqrt {T};, ; d^{2}C}}right)$ . Comparing these bounds establishes that the proposed algorithm achieves nearly optimal $tilde{mathcal {O}} (sqrt {T})$ regret when $C$ is $o(sqrt {T})$ and maintains sub-linear regret for a broader range of $C$ .
{"title":"Robust Causal Bandits for Linear Models","authors":"Zirui Yan;Arpan Mukherjee;Burak Varıcı;Ali Tajer","doi":"10.1109/JSAIT.2024.3373595","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3373595","url":null,"abstract":"The sequential design of experiments for optimizing a reward function in causal systems can be effectively modeled by the sequential design of interventions in causal bandits (CBs). In the existing literature on CBs, a critical assumption is that the causal models remain constant over time. However, this assumption does not necessarily hold in complex systems, which constantly undergo temporal model fluctuations. This paper addresses the robustness of CBs to such model fluctuations. The focus is on causal systems with linear structural equation models (SEMs). The SEMs and the time-varying pre- and post-interventional statistical models are all unknown. Cumulative regret is adopted as the design criteria, based on which the objective is to design a sequence of interventions that incur the smallest cumulative regret with respect to an oracle aware of the entire causal model and its fluctuations. First, it is established that the existing approaches fail to maintain regret sub-linearity with even a few instances of model deviation. Specifically, when the number of instances with model deviation is as few as \u0000<inline-formula> <tex-math>$T^{frac {1}{2L}}$ </tex-math></inline-formula>\u0000, where \u0000<inline-formula> <tex-math>$T$ </tex-math></inline-formula>\u0000 is the time horizon and \u0000<inline-formula> <tex-math>$L$ </tex-math></inline-formula>\u0000 is the length of the longest causal path in the graph, the existing algorithms will have linear regret in \u0000<inline-formula> <tex-math>$T$ </tex-math></inline-formula>\u0000. For instance, when \u0000<inline-formula> <tex-math>$T=10^{5}$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$L=3$ </tex-math></inline-formula>\u0000, model deviations in 6 out of 105 instances result in a linear regret. Next, a robust CB algorithm is designed, and its regret is analyzed, where upper and information-theoretic lower bounds on the regret are established. Specifically, in a graph with \u0000<inline-formula> <tex-math>$N$ </tex-math></inline-formula>\u0000 nodes and maximum degree \u0000<inline-formula> <tex-math>$d$ </tex-math></inline-formula>\u0000, under a general measure of model deviation \u0000<inline-formula> <tex-math>$C$ </tex-math></inline-formula>\u0000, the cumulative regret is upper bounded by \u0000<inline-formula> <tex-math>$tilde {mathcal {O}}left({d^{L-{}frac {1}{2}}(sqrt {NT} + NC)}right)$ </tex-math></inline-formula>\u0000 and lower bounded by \u0000<inline-formula> <tex-math>$Omega left({d^{frac {L}{2}-2}max {sqrt {T};, ; d^{2}C}}right)$ </tex-math></inline-formula>\u0000. Comparing these bounds establishes that the proposed algorithm achieves nearly optimal \u0000<inline-formula> <tex-math>$tilde{mathcal {O}} (sqrt {T})$ </tex-math></inline-formula>\u0000 regret when \u0000<inline-formula> <tex-math>$C$ </tex-math></inline-formula>\u0000 is \u0000<inline-formula> <tex-math>$o(sqrt {T})$ </tex-math></inline-formula>\u0000 and maintains sub-linear regret for a broader range of \u0000<inline-formula> <tex-math>$C$ </tex-math></inline-formula>\u0000.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"78-93"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140605970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3396400
Mohamed Seif;Yanxi Chen;Andrea J. Goldsmith;H. Vincent Poor
We study the community detection problem over binary symmetric stochastic block models (SBMs) while preserving the privacy of the individual connections between the vertices. We present and analyze the associated information-theoretic tradeoff for differentially private exact recovery of the underlying communities by deriving sufficient separation conditions between the intra-community and inter-community connection probabilities while taking into account the privacy budget and graph sketching as a speed-up technique to improve the computational complexity of maximum likelihood (ML) based recovery algorithms.
{"title":"Differentially Private Sketch-and-Solve for Community Detection via Semidefinite Programming","authors":"Mohamed Seif;Yanxi Chen;Andrea J. Goldsmith;H. Vincent Poor","doi":"10.1109/JSAIT.2024.3396400","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3396400","url":null,"abstract":"We study the community detection problem over binary symmetric stochastic block models (SBMs) while preserving the privacy of the individual connections between the vertices. We present and analyze the associated information-theoretic tradeoff for differentially private exact recovery of the underlying communities by deriving sufficient separation conditions between the intra-community and inter-community connection probabilities while taking into account the privacy budget and graph sketching as a speed-up technique to improve the computational complexity of maximum likelihood (ML) based recovery algorithms.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"331-346"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141251112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3396787
Banghua Zhu;Ziao Wang;Nadim Ghaddar;Jiantao Jiao;Lele Wang
We consider the problem of computing a function of n variables using noisy queries, where each query is incorrect with some fixed and known probability $p in (0,1/2)$ . Specifically, we consider the computation of the $textsf {OR}$ function of n bits (where queries correspond to noisy readings of the bits) and the $textsf {MAX}$ function of n real numbers (where queries correspond to noisy pairwise comparisons). We show that an expected number of queries of $(1 pm o(1)) {}frac {nlog {}frac {1}{delta }}{D_{textsf {KL}}(p | 1-p)}$ is both sufficient and necessary to compute both functions with a vanishing error probability $delta = o(1)$ , where $D_{textsf {KL}}(p | 1-p)$ denotes the Kullback-Leibler divergence between $textsf {Bern}(p)$ and $textsf {Bern}(1-p)$ distributions. Compared to previous work, our results tighten the dependence on p in both the upper and lower bounds for the two functions.
我们考虑的问题是使用噪声查询计算 n 个变量的函数，其中每个查询都是不正确的，其概率是固定且已知的 $p in (0,1/2)$。具体来说，我们考虑计算 n 个比特的 $textsf {OR}$ 函数（其中查询对应于比特的噪声读数）和 n 个实数的 $textsf {MAX}$ 函数（其中查询对应于噪声成对比较）。我们证明，$(1 pm o(1)) {}frac {nlog {}frac {1}{delta }}{D_{textsf {KL}}(p | 1-p)}$ 的预期查询次数对于以消失的错误概率 $delta = o(1)$ 计算这两个函数来说既充分又必要、其中，$D_{textsf {KL}}(p | 1-p)$ 表示 $textsf {Bern}(p)$ 和 $textsf {Bern}(1-p)$ 分布之间的库尔贝-莱布勒发散。与之前的工作相比，我们的结果在两个函数的上界和下界中都加强了对 p 的依赖。
{"title":"Noisy Computing of the OR and MAX Functions","authors":"Banghua Zhu;Ziao Wang;Nadim Ghaddar;Jiantao Jiao;Lele Wang","doi":"10.1109/JSAIT.2024.3396787","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3396787","url":null,"abstract":"We consider the problem of computing a function of n variables using noisy queries, where each query is incorrect with some fixed and known probability \u0000<inline-formula> <tex-math>$p in (0,1/2)$ </tex-math></inline-formula>\u0000. Specifically, we consider the computation of the \u0000<inline-formula> <tex-math>$textsf {OR}$ </tex-math></inline-formula>\u0000 function of n bits (where queries correspond to noisy readings of the bits) and the \u0000<inline-formula> <tex-math>$textsf {MAX}$ </tex-math></inline-formula>\u0000 function of n real numbers (where queries correspond to noisy pairwise comparisons). We show that an expected number of queries of \u0000<inline-formula> <tex-math>$(1 pm o(1)) {}frac {nlog {}frac {1}{delta }}{D_{textsf {KL}}(p | 1-p)}$ </tex-math></inline-formula>\u0000 is both sufficient and necessary to compute both functions with a vanishing error probability \u0000<inline-formula> <tex-math>$delta = o(1)$ </tex-math></inline-formula>\u0000, where \u0000<inline-formula> <tex-math>$D_{textsf {KL}}(p | 1-p)$ </tex-math></inline-formula>\u0000 denotes the Kullback-Leibler divergence between \u0000<inline-formula> <tex-math>$textsf {Bern}(p)$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$textsf {Bern}(1-p)$ </tex-math></inline-formula>\u0000 distributions. Compared to previous work, our results tighten the dependence on p in both the upper and lower bounds for the two functions.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"302-313"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141084767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/JSAIT.2024.3396079
Ruizhi Zhang
Detection of sparse signals arises in many modern applications such as signal processing, bioinformatics, finance, and disease surveillance. However, in many of these applications, the data may contain sensitive personal information, which is desirable to be protected during the data analysis. In this article, we consider the problem of $(epsilon,delta)$ -differentially private detection of a general sparse mixture with a focus on how privacy affects the detection power. By investigating the nonasymptotic upper bound for the summation of error probabilities, we find any $(epsilon,delta)$ -differentially private test cannot detect the sparse signal if the privacy constraint is too strong or if the model parameters are in the undetectable region (Cai and Wu, 2014). Moreover, we study the private clamped log-likelihood ratio test proposed by Canonne et al., 2019 and show it achieves vanishing error probabilities in some conditions on the model parameters and privacy parameters. Then, for the case when the null distribution is a standard normal distribution, we propose an adaptive $(epsilon,delta)$ -differentially private test, which achieves vanishing error probabilities in the same detectable region (Cai and Wu, 2014) when the privacy parameters satisfy certain sufficient conditions. Several numerical experiments are conducted to verify our theoretical results and illustrate the performance of our proposed test.
稀疏信号的检测在信号处理、生物信息学、金融和疾病监测等许多现代应用中都会出现。然而，在许多此类应用中，数据可能包含敏感的个人信息，这就需要在数据分析过程中加以保护。在本文中，我们考虑了一般稀疏混合物的$(epsilon,delta)$差异隐私检测问题，重点关注隐私如何影响检测能力。通过研究误差概率求和的非渐近上界，我们发现如果隐私约束太强或模型参数处于不可检测区域，任何$(epsilon,delta)$-差异隐私检测都无法检测到稀疏信号（Cai and Wu，2014）。此外，我们还研究了 Canonne 等人 2019 年提出的私有钳位对数似然比检验，结果表明它在模型参数和隐私参数的某些条件下实现了虚化误差概率。然后，对于空分布是标准正态分布的情况，我们提出了一种自适应的$(epsilon,delta)$-差异私有检验，当隐私参数满足某些充分条件时，它在相同的可检测区域内实现了消失的误差概率（Cai and Wu, 2014）。我们进行了一些数值实验来验证我们的理论结果，并说明我们提出的测试的性能。
{"title":"Detection of Sparse Mixtures With Differential Privacy","authors":"Ruizhi Zhang","doi":"10.1109/JSAIT.2024.3396079","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3396079","url":null,"abstract":"Detection of sparse signals arises in many modern applications such as signal processing, bioinformatics, finance, and disease surveillance. However, in many of these applications, the data may contain sensitive personal information, which is desirable to be protected during the data analysis. In this article, we consider the problem of \u0000<inline-formula> <tex-math>$(epsilon,delta)$ </tex-math></inline-formula>\u0000-differentially private detection of a general sparse mixture with a focus on how privacy affects the detection power. By investigating the nonasymptotic upper bound for the summation of error probabilities, we find any \u0000<inline-formula> <tex-math>$(epsilon,delta)$ </tex-math></inline-formula>\u0000-differentially private test cannot detect the sparse signal if the privacy constraint is too strong or if the model parameters are in the undetectable region (Cai and Wu, 2014). Moreover, we study the private clamped log-likelihood ratio test proposed by Canonne et al., 2019 and show it achieves vanishing error probabilities in some conditions on the model parameters and privacy parameters. Then, for the case when the null distribution is a standard normal distribution, we propose an adaptive \u0000<inline-formula> <tex-math>$(epsilon,delta)$ </tex-math></inline-formula>\u0000-differentially private test, which achieves vanishing error probabilities in the same detectable region (Cai and Wu, 2014) when the privacy parameters satisfy certain sufficient conditions. Several numerical experiments are conducted to verify our theoretical results and illustrate the performance of our proposed test.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"347-356"},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141435289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-27DOI: 10.1109/JSAIT.2024.3370699
Chen Xu;Jonghyeok Lee;Xiuyuan Cheng;Yao Xie
We present a computationally efficient framework, called FlowDRO , for solving flow-based distributionally robust optimization (DRO) problems with Wasserstein uncertainty sets while aiming to find continuous worst-case distribution (also called the Least Favorable Distribution, LFD) and sample from it. The requirement for LFD to be continuous is so that the algorithm can be scalable to problems with larger sample sizes and achieve better generalization capability for the induced robust algorithms. To tackle the computationally challenging infinitely dimensional optimization problem, we leverage flow-based models and continuous-time invertible transport maps between the data distribution and the target distribution and develop a Wasserstein proximal gradient flow type algorithm. In theory, we establish the equivalence of the solution by optimal transport map to the original formulation, as well as the dual form of the problem through Wasserstein calculus and Brenier theorem. In practice, we parameterize the transport maps by a sequence of neural networks progressively trained in blocks by gradient descent. We demonstrate its usage in adversarial learning, distributionally robust hypothesis testing, and a new mechanism for data-driven distribution perturbation differential privacy, where the proposed method gives strong empirical performance on high-dimensional real data.
{"title":"Flow-Based Distributionally Robust Optimization","authors":"Chen Xu;Jonghyeok Lee;Xiuyuan Cheng;Yao Xie","doi":"10.1109/JSAIT.2024.3370699","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3370699","url":null,"abstract":"We present a computationally efficient framework, called \u0000<monospace>FlowDRO</monospace>\u0000, for solving flow-based distributionally robust optimization (DRO) problems with Wasserstein uncertainty sets while aiming to find continuous worst-case distribution (also called the Least Favorable Distribution, LFD) and sample from it. The requirement for LFD to be continuous is so that the algorithm can be scalable to problems with larger sample sizes and achieve better generalization capability for the induced robust algorithms. To tackle the computationally challenging infinitely dimensional optimization problem, we leverage flow-based models and continuous-time invertible transport maps between the data distribution and the target distribution and develop a Wasserstein proximal gradient flow type algorithm. In theory, we establish the equivalence of the solution by optimal transport map to the original formulation, as well as the dual form of the problem through Wasserstein calculus and Brenier theorem. In practice, we parameterize the transport maps by a sequence of neural networks progressively trained in blocks by gradient descent. We demonstrate its usage in adversarial learning, distributionally robust hypothesis testing, and a new mechanism for data-driven distribution perturbation differential privacy, where the proposed method gives strong empirical performance on high-dimensional real data.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"62-77"},"PeriodicalIF":0.0,"publicationDate":"2024-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1109/JSAIT.2024.3368229
Matteo Zecchin;Sangwoo Park;Osvaldo Simeone
In many real-world problems, predictions are leveraged to monitor and control cyber-physical systems, demanding guarantees on the satisfaction of reliability and safety requirements. However, predictions are inherently uncertain, and managing prediction uncertainty presents significant challenges in environments characterized by complex dynamics and forking trajectories. In this work, we assume access to a pre-designed probabilistic implicit or explicit sequence model, which may have been obtained using model-based or model-free methods. We introduce probabilistic time series-conformal risk prediction (PTS-CRC), a novel post-hoc calibration procedure that operates on the predictions produced by any pre-designed probabilistic forecaster to yield reliable error bars. In contrast to existing art, PTS-CRC produces predictive sets based on an ensemble of multiple prototype trajectories sampled from the sequence model, supporting the efficient representation of forking uncertainties. Furthermore, unlike the state of the art, PTS-CRC can satisfy reliability definitions beyond coverage. This property is leveraged to devise a novel model predictive control (MPC) framework that addresses open-loop and closed-loop control problems under general average constraints on the quality or safety of the control policy. We experimentally validate the performance of PTS-CRC prediction and control by studying a number of use cases in the context of wireless networking. Across all the considered tasks, PTS-CRC predictors are shown to provide more informative predictive sets, as well as safe control policies with larger returns.
{"title":"Forking Uncertainties: Reliable Prediction and Model Predictive Control With Sequence Models via Conformal Risk Control","authors":"Matteo Zecchin;Sangwoo Park;Osvaldo Simeone","doi":"10.1109/JSAIT.2024.3368229","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3368229","url":null,"abstract":"In many real-world problems, predictions are leveraged to monitor and control cyber-physical systems, demanding guarantees on the satisfaction of reliability and safety requirements. However, predictions are inherently uncertain, and managing prediction uncertainty presents significant challenges in environments characterized by complex dynamics and forking trajectories. In this work, we assume access to a pre-designed probabilistic implicit or explicit sequence model, which may have been obtained using model-based or model-free methods. We introduce probabilistic time series-conformal risk prediction (PTS-CRC), a novel post-hoc calibration procedure that operates on the predictions produced by any pre-designed probabilistic forecaster to yield reliable error bars. In contrast to existing art, PTS-CRC produces predictive sets based on an ensemble of multiple prototype trajectories sampled from the sequence model, supporting the efficient representation of forking uncertainties. Furthermore, unlike the state of the art, PTS-CRC can satisfy reliability definitions beyond coverage. This property is leveraged to devise a novel model predictive control (MPC) framework that addresses open-loop and closed-loop control problems under general average constraints on the quality or safety of the control policy. We experimentally validate the performance of PTS-CRC prediction and control by studying a number of use cases in the context of wireless networking. Across all the considered tasks, PTS-CRC predictors are shown to provide more informative predictive sets, as well as safe control policies with larger returns.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"44-61"},"PeriodicalIF":0.0,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-22DOI: 10.1109/JSAIT.2024.3366086
Anand Jerry George;Lekshmi Ramesh;Aditya Vikram Singh;Himanshu Tyagi
We consider the problem of continually releasing an estimate of the population mean of a stream of samples that is user-level differentially private (DP). At each time instant, a user contributes a sample, and the users can arrive in arbitrary order. Until now these requirements of continual release and user-level privacy were considered in isolation. But, in practice, both these requirements come together as the users often contribute data repeatedly and multiple queries are made. We provide an algorithm that outputs a mean estimate at every time instant t such that the overall release is user-level $varepsilon $ -DP and has the following error guarantee: Denoting by $m_{t}$ the maximum number of samples contributed by a user, as long as $tilde {Omega }(1/varepsilon)$ users have $m_{t}/2$ samples each, the error at time t is $tilde {O}(1/sqrt {t}+sqrt {m}_{t}/tvarepsilon)$ . This is a universal error guarantee which is valid for all arrival patterns of the users. Furthermore, it (almost) matches the existing lower bounds for the single-release setting at all time instants when users have contributed equal number of samples.
{"title":"Continual Mean Estimation Under User-Level Privacy","authors":"Anand Jerry George;Lekshmi Ramesh;Aditya Vikram Singh;Himanshu Tyagi","doi":"10.1109/JSAIT.2024.3366086","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3366086","url":null,"abstract":"We consider the problem of continually releasing an estimate of the population mean of a stream of samples that is user-level differentially private (DP). At each time instant, a user contributes a sample, and the users can arrive in arbitrary order. Until now these requirements of continual release and user-level privacy were considered in isolation. But, in practice, both these requirements come together as the users often contribute data repeatedly and multiple queries are made. We provide an algorithm that outputs a mean estimate at every time instant t such that the overall release is user-level \u0000<inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>\u0000-DP and has the following error guarantee: Denoting by \u0000<inline-formula> <tex-math>$m_{t}$ </tex-math></inline-formula>\u0000 the maximum number of samples contributed by a user, as long as \u0000<inline-formula> <tex-math>$tilde {Omega }(1/varepsilon)$ </tex-math></inline-formula>\u0000 users have \u0000<inline-formula> <tex-math>$m_{t}/2$ </tex-math></inline-formula>\u0000 samples each, the error at time t is \u0000<inline-formula> <tex-math>$tilde {O}(1/sqrt {t}+sqrt {m}_{t}/tvarepsilon)$ </tex-math></inline-formula>\u0000. This is a universal error guarantee which is valid for all arrival patterns of the users. Furthermore, it (almost) matches the existing lower bounds for the single-release setting at all time instants when users have contributed equal number of samples.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"28-43"},"PeriodicalIF":0.0,"publicationDate":"2024-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-15DOI: 10.1109/JSAIT.2024.3366225
Antonious M. Girgis;Suhas Diggavi
We study the distributed mean estimation (DME) problem under privacy and communication constraints in the local differential privacy (LDP) and multi-message shuffled (MMS) privacy frameworks. The DME has wide applications in both federated learning and analytics. We propose a communication-efficient and differentially private algorithm for DME of bounded $ell _{2}$ -norm and $ell _{infty }$ -norm vectors. We analyze our proposed DME schemes showing that our algorithms have order-optimal privacy-communication-performance trade-offs. Our algorithms are designed by giving unequal privacy assignments at different resolutions of the vector (through binary expansion) and appropriately combining it with coordinate sampling. These results are directly applied to give guarantees on private federated learning algorithms. We also numerically evaluate the performance of our private DME algorithms.
{"title":"Multi-Message Shuffled Privacy in Federated Learning","authors":"Antonious M. Girgis;Suhas Diggavi","doi":"10.1109/JSAIT.2024.3366225","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3366225","url":null,"abstract":"We study the distributed mean estimation (DME) problem under privacy and communication constraints in the local differential privacy (LDP) and multi-message shuffled (MMS) privacy frameworks. The DME has wide applications in both federated learning and analytics. We propose a communication-efficient and differentially private algorithm for DME of bounded \u0000<inline-formula> <tex-math>$ell _{2}$ </tex-math></inline-formula>\u0000-norm and \u0000<inline-formula> <tex-math>$ell _{infty }$ </tex-math></inline-formula>\u0000-norm vectors. We analyze our proposed DME schemes showing that our algorithms have order-optimal privacy-communication-performance trade-offs. Our algorithms are designed by giving unequal privacy assignments at different resolutions of the vector (through binary expansion) and appropriately combining it with coordinate sampling. These results are directly applied to give guarantees on private federated learning algorithms. We also numerically evaluate the performance of our private DME algorithms.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"12-27"},"PeriodicalIF":0.0,"publicationDate":"2024-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140052943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-06DOI: 10.1109/JSAIT.2024.3362324
Venugopal V. Veeravalli;Georgios Fellouris;George V. Moustakides
In the problem of quickest change detection, a change occurs at some unknown time in the distribution of a sequence of random vectors that are monitored in real time, and the goal is to detect this change as quickly as possible subject to a certain false alarm constraint. In this work we consider this problem in the presence of parametric uncertainty in the post-change regime and controlled sensing. That is, the post-change distribution contains an unknown parameter, and the distribution of each observation, before and after the change, is affected by a control action. In this context, in addition to a stopping rule that determines the time at which it is declared that the change has occurred, one also needs to determine a sequential control policy, which chooses the control action at each time based on the already collected observations. We formulate this problem mathematically using Lorden’s minimax criterion, and assuming that there are finitely many possible actions and post-change parameter values. We then propose a specific procedure for this problem that employs an adaptive CuSum statistic in which (i) the estimate of the parameter is based on a fixed number of the more recent observations, and (ii) each action is selected to maximize the Kullback-Leibler divergence of the next observation based on the current parameter estimate, apart from a small number of exploration times. We show that this procedure, which we call the Windowed Chernoff-CuSum (WCC), is first-order asymptotically optimal under Lorden’s minimax criterion, for every possible value of the unknown post-change parameter, as the mean time to false alarm goes to infinity. We also provide simulation results to illustrate the performance of the WCC procedure.
{"title":"Quickest Change Detection With Controlled Sensing","authors":"Venugopal V. Veeravalli;Georgios Fellouris;George V. Moustakides","doi":"10.1109/JSAIT.2024.3362324","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3362324","url":null,"abstract":"In the problem of quickest change detection, a change occurs at some unknown time in the distribution of a sequence of random vectors that are monitored in real time, and the goal is to detect this change as quickly as possible subject to a certain false alarm constraint. In this work we consider this problem in the presence of parametric uncertainty in the post-change regime and controlled sensing. That is, the post-change distribution contains an unknown parameter, and the distribution of each observation, before and after the change, is affected by a control action. In this context, in addition to a stopping rule that determines the time at which it is declared that the change has occurred, one also needs to determine a sequential control policy, which chooses the control action at each time based on the already collected observations. We formulate this problem mathematically using Lorden’s minimax criterion, and assuming that there are finitely many possible actions and post-change parameter values. We then propose a specific procedure for this problem that employs an adaptive CuSum statistic in which (i) the estimate of the parameter is based on a fixed number of the more recent observations, and (ii) each action is selected to maximize the Kullback-Leibler divergence of the next observation based on the current parameter estimate, apart from a small number of exploration times. We show that this procedure, which we call the Windowed Chernoff-CuSum (WCC), is first-order asymptotically optimal under Lorden’s minimax criterion, for every possible value of the unknown post-change parameter, as the mean time to false alarm goes to infinity. We also provide simulation results to illustrate the performance of the WCC procedure.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"1-11"},"PeriodicalIF":0.0,"publicationDate":"2024-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10423411","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140052897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：