首页 > 最新文献

2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)最新文献

英文 中文
An empirical comparison of sampling techniques for matrix column subset selection 抽样技术的经验比较,为矩阵列子集的选择
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447127
Yining Wang, Aarti Singh
Column subset selection (CSS) is the problem of selecting a small portion of columns from a large data matrix as one form of interpretable data summarization. Leverage score sampling, which enjoys both sound theoretical guarantee and superior empirical performance, is widely recognized as the state-of-the-art algorithm for column subset selection. In this paper, we revisit iterative norm sampling, another sampling based CSS algorithm proposed even before leverage score sampling, and demonstrate its competitive performance under a wide range of experimental settings. We also compare iterative norm sampling with several of its other competitors and show its superior performance in terms of both approximation accuracy and computational efficiency. We conclude that further theoretical investigation and practical consideration should be devoted to iterative norm sampling in column subset selection.
列子集选择(CSS)是从大数据矩阵中选择一小部分列作为可解释数据摘要的一种形式的问题。杠杆分数抽样是目前公认的最先进的列子集选择算法,具有较好的理论保证和较好的经验性能。在本文中,我们回顾了迭代范数抽样,这是在杠杆分数抽样之前提出的另一种基于抽样的CSS算法,并在广泛的实验设置下展示了它的竞争性能。我们还将迭代范数抽样与其他几种竞争对手进行了比较,并显示了其在近似精度和计算效率方面的优越性能。我们认为迭代范数抽样在列子集选择中的应用需要进一步的理论研究和实践考虑。
{"title":"An empirical comparison of sampling techniques for matrix column subset selection","authors":"Yining Wang, Aarti Singh","doi":"10.1109/ALLERTON.2015.7447127","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447127","url":null,"abstract":"Column subset selection (CSS) is the problem of selecting a small portion of columns from a large data matrix as one form of interpretable data summarization. Leverage score sampling, which enjoys both sound theoretical guarantee and superior empirical performance, is widely recognized as the state-of-the-art algorithm for column subset selection. In this paper, we revisit iterative norm sampling, another sampling based CSS algorithm proposed even before leverage score sampling, and demonstrate its competitive performance under a wide range of experimental settings. We also compare iterative norm sampling with several of its other competitors and show its superior performance in terms of both approximation accuracy and computational efficiency. We conclude that further theoretical investigation and practical consideration should be devoted to iterative norm sampling in column subset selection.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"249 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123263688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Finite-time analysis of the distributed detection problem 分布式检测问题的有限时间分析
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447059
Shahin Shahrampour, A. Rakhlin, A. Jadbabaie
This paper addresses the problem of distributed detection in fixed and switching networks. A network of agents observe partially informative signals about the unknown state of the world. Hence, they collaborate with each other to identify the true state. We propose an update rule building on distributed, stochastic optimization methods. Our main focus is on the finite-time analysis of the problem. For fixed networks, we bring forward the notion of Kullback-Leibler cost to measure the efficiency of the algorithm versus its centralized analog. We bound the cost in terms of the network size, spectral gap and relative entropy of agents' signal structures. We further consider the problem in random networks where the structure is realized according to a stationary distribution. We then prove that the convergence is exponentially fast (with high probability), and the non-asymptotic rate scales inversely in the spectral gap of the expected network.
本文研究了固定网络和交换网络中的分布式检测问题。智能体网络观察关于未知世界状态的部分信息信号。因此,它们相互协作以识别真实状态。我们提出了一种基于分布式随机优化方法的更新规则构建。我们的主要重点是对问题进行有限时间的分析。对于固定网络,我们提出了Kullback-Leibler成本的概念来衡量算法相对于其集中式模拟的效率。我们根据网络规模、频谱间隙和智能体信号结构的相对熵来确定成本。我们进一步考虑随机网络中结构按平稳分布实现的问题。然后,我们证明了收敛速度是指数级的(具有高概率),并且非渐近速率在期望网络的谱间隙中呈反比。
{"title":"Finite-time analysis of the distributed detection problem","authors":"Shahin Shahrampour, A. Rakhlin, A. Jadbabaie","doi":"10.1109/ALLERTON.2015.7447059","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447059","url":null,"abstract":"This paper addresses the problem of distributed detection in fixed and switching networks. A network of agents observe partially informative signals about the unknown state of the world. Hence, they collaborate with each other to identify the true state. We propose an update rule building on distributed, stochastic optimization methods. Our main focus is on the finite-time analysis of the problem. For fixed networks, we bring forward the notion of Kullback-Leibler cost to measure the efficiency of the algorithm versus its centralized analog. We bound the cost in terms of the network size, spectral gap and relative entropy of agents' signal structures. We further consider the problem in random networks where the structure is realized according to a stationary distribution. We then prove that the convergence is exponentially fast (with high probability), and the non-asymptotic rate scales inversely in the spectral gap of the expected network.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114168504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
On the rate of learning in distributed hypothesis testing 关于分布假设检验中的学习速率
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7446979
Anusha Lalitha, T. Javidi
This paper considers a problem of distributed hypothesis testing and cooperative learning. Individual nodes in a network receive noisy local (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The conditional distributions are known locally at the nodes, but the true parameter/hypothesis is not known. We consider a social (“non-Bayesian”) learning rule from previous literature, in which nodes first perform a Bayesian update of their belief (distribution estimate) of the parameter based on their local observation, communicate these updates to their neighbors, and then perform a “non-Bayesian” linear consensus using the log-beliefs of their neighbors. For this learning rule, we know that under mild assumptions, the belief of any node in any incorrect parameter converges to zero exponentially fast, and the exponential rate of learning is a characterized by the network structure and the divergences between the observations' distributions. Tight bounds on the probability of deviating from this nominal rate in aperiodic networks is derived. The bounds are shown to hold for all conditional distributions which satisfy a mild bounded moment condition.
本文研究了分布式假设检验和合作学习问题。网络中的单个节点接收有噪声的局部(私有)观测,其分布由离散参数(假设)参数化。条件分布在局部节点上是已知的,但真正的参数/假设是未知的。我们从以前的文献中考虑一个社会(“非贝叶斯”)学习规则,其中节点首先根据他们的局部观察对参数的信念(分布估计)执行贝叶斯更新,将这些更新传达给他们的邻居,然后使用他们邻居的对数信念执行“非贝叶斯”线性共识。对于这个学习规则,我们知道,在温和的假设下,任何节点在任何不正确参数下的信念以指数速度收敛于零,并且学习的指数速度是由网络结构和观测值分布之间的发散度表征的。推导了非周期网络中偏离该标称速率的概率的严格界限。对于满足温和有界矩条件的所有条件分布,边界都成立。
{"title":"On the rate of learning in distributed hypothesis testing","authors":"Anusha Lalitha, T. Javidi","doi":"10.1109/ALLERTON.2015.7446979","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7446979","url":null,"abstract":"This paper considers a problem of distributed hypothesis testing and cooperative learning. Individual nodes in a network receive noisy local (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The conditional distributions are known locally at the nodes, but the true parameter/hypothesis is not known. We consider a social (“non-Bayesian”) learning rule from previous literature, in which nodes first perform a Bayesian update of their belief (distribution estimate) of the parameter based on their local observation, communicate these updates to their neighbors, and then perform a “non-Bayesian” linear consensus using the log-beliefs of their neighbors. For this learning rule, we know that under mild assumptions, the belief of any node in any incorrect parameter converges to zero exponentially fast, and the exponential rate of learning is a characterized by the network structure and the divergences between the observations' distributions. Tight bounds on the probability of deviating from this nominal rate in aperiodic networks is derived. The bounds are shown to hold for all conditional distributions which satisfy a mild bounded moment condition.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116136979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Sparse covariance estimation based on sparse-graph codes 基于稀疏图代码的稀疏协方差估计
Pub Date : 2015-09-01 DOI: 10.1109/allerton.2015.7447061
Ramtin Pedarsani, Kangwook Lee, K. Ramchandran
We consider the problem of recovering a sparse covariance matrix Σ∈ℝn×n from m quadratic measurements yi = aiTΣai+wi, 1 ≤ i ≤ m, where ai ∈ ℓn is a measurement vector and wi is additive noise. We assume that ℝ has K non-zero off-diagonal entries. We first consider the simplified noiseless problem where wi = 0 for all i. We introduce two low complexity algorithms, the first a “message-passing” algorithm and the second a “forward” algorithm, that are based on a sparse-graph coding framework. We show that under some simplifying assumptions, the message passing algorithm can recover an arbitrarily-large fraction of the K non-zero components with cK measurements, where c is a small constant that can be precisely characterized. As one instance, the message passing algorithm can recover, with high probability, a fraction 1 - 10-4 of the non-zero components, using only m = 6K quadratic measurements, which is a small constant factor from the fundamental limit, with an optimal O(K) decoding complexity. We further show that the forward algorithm can recover all the K non-zero entries with high probability with m = Θ(K) measurements and O(K log(K)) decoding complexity. However, the forward algorithm suffers from significantly larger constants in terms of the number of required measurements, and is indeed less practical despite providing stronger theoretical guarantees. We then consider the noisy setting, and show that both proposed algorithms can be robustified to noise with m = Θ(K log2(n)) measurements. Finally, we provide extensive simulation results that support our theoretical claims.
我们考虑从m个二次测量yi = aiTΣai+wi, 1≤i≤m中恢复稀疏协方差矩阵Σ∈λ n×n的问题,其中ai∈λ n为测量向量,wi为加性噪声。我们假设它有K个非零的非对角线元素。我们首先考虑简化的无噪声问题,其中wi = 0对所有i。我们引入了两种低复杂度算法,第一个是“消息传递”算法,第二个是“转发”算法,这是基于稀疏图编码框架。我们证明,在一些简化的假设下,消息传递算法可以恢复任意大的K非零分量与cK测量,其中c是一个小的常数,可以精确表征。例如,消息传递算法仅使用m = 6K二次测量(这是基本极限的一个小常数因子),就可以高概率地恢复非零分量的1 - 10-4,具有最优的O(K)解码复杂度。我们进一步证明了前向算法可以高概率地恢复所有K个非零条目,m = Θ(K)测量值和O(K log(K))解码复杂度。然而,前向算法在所需测量的数量方面存在明显较大的常数,尽管提供了更强的理论保证,但确实不太实用。然后我们考虑噪声设置,并表明这两种算法都可以通过m = Θ(K log2(n))测量对噪声进行鲁棒化。最后,我们提供了广泛的模拟结果来支持我们的理论主张。
{"title":"Sparse covariance estimation based on sparse-graph codes","authors":"Ramtin Pedarsani, Kangwook Lee, K. Ramchandran","doi":"10.1109/allerton.2015.7447061","DOIUrl":"https://doi.org/10.1109/allerton.2015.7447061","url":null,"abstract":"We consider the problem of recovering a sparse covariance matrix Σ∈ℝn×n from m quadratic measurements yi = aiTΣai+wi, 1 ≤ i ≤ m, where ai ∈ ℓn is a measurement vector and wi is additive noise. We assume that ℝ has K non-zero off-diagonal entries. We first consider the simplified noiseless problem where wi = 0 for all i. We introduce two low complexity algorithms, the first a “message-passing” algorithm and the second a “forward” algorithm, that are based on a sparse-graph coding framework. We show that under some simplifying assumptions, the message passing algorithm can recover an arbitrarily-large fraction of the K non-zero components with cK measurements, where c is a small constant that can be precisely characterized. As one instance, the message passing algorithm can recover, with high probability, a fraction 1 - 10-4 of the non-zero components, using only m = 6K quadratic measurements, which is a small constant factor from the fundamental limit, with an optimal O(K) decoding complexity. We further show that the forward algorithm can recover all the K non-zero entries with high probability with m = Θ(K) measurements and O(K log(K)) decoding complexity. However, the forward algorithm suffers from significantly larger constants in terms of the number of required measurements, and is indeed less practical despite providing stronger theoretical guarantees. We then consider the noisy setting, and show that both proposed algorithms can be robustified to noise with m = Θ(K log2(n)) measurements. Finally, we provide extensive simulation results that support our theoretical claims.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"161 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115295019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Are generalized cut-set bounds tight for the deterministic interference channel? 确定性干扰信道的广义割集边界紧吗?
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447176
Mehrdad Kiamari, A. Avestimehr
We propose the idea of extended networks, which is constructed by replicating the users in the two-user deterministic interference channel (DIC) and designing the interference structure among them, such that any rate that can be achieved by each user in the original network can also be achieved simultaneously by all replicas of that user in the extended network. We demonstrate that by carefully designing extended networks and applying the generalized cut-set (GCS) bound to them, we can derive a tight converse for the two-user DIC. Furthermore, we generalize our techniques to the three-user DIC, and demonstrate that the proposed approach also results in deriving a tight converse for the three-user DIC in the symmetric case.
我们提出了扩展网络的思想,通过复制双用户确定性干扰通道(DIC)中的用户并设计它们之间的干扰结构来构建扩展网络,使得原始网络中的每个用户可以达到的任何速率也可以被扩展网络中的所有复制用户同时达到。我们证明了通过精心设计扩展网络并将广义割集(GCS)绑定到它们上,我们可以推导出双用户DIC的紧逆。此外,我们将我们的技术推广到三用户DIC,并证明了所提出的方法也导致了对称情况下三用户DIC的紧逆。
{"title":"Are generalized cut-set bounds tight for the deterministic interference channel?","authors":"Mehrdad Kiamari, A. Avestimehr","doi":"10.1109/ALLERTON.2015.7447176","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447176","url":null,"abstract":"We propose the idea of extended networks, which is constructed by replicating the users in the two-user deterministic interference channel (DIC) and designing the interference structure among them, such that any rate that can be achieved by each user in the original network can also be achieved simultaneously by all replicas of that user in the extended network. We demonstrate that by carefully designing extended networks and applying the generalized cut-set (GCS) bound to them, we can derive a tight converse for the two-user DIC. Furthermore, we generalize our techniques to the three-user DIC, and demonstrate that the proposed approach also results in deriving a tight converse for the three-user DIC in the symmetric case.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"66 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116750671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Statistical and computational guarantees for the Baum-Welch algorithm 鲍姆-韦尔奇算法的统计和计算保证
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447067
Fanny Yang, Sivaraman Balakrishnan, M. Wainwright
The Hidden Markov Model (HMM) is one of the main-stays of statistical modeling of discrete time series and is widely used in many applications. Estimating an HMM from its observation process is often addressed via the Baum-Welch algorithm, which performs well empirically when initialized reasonably close to the truth. This behavior could not be explained by existing theory which predicts susceptibility to bad local optima. In this paper we aim at closing the gap and provide a framework to characterize a sufficient basin of attraction for any global optimum in which Baum-Welch is guaranteed to converge linearly to an “optimally” small ball around the global optimum. The framework is then used to determine the linear rate of convergence and a sufficient initialization region for Baum-Welch applied on a two component isotropic hidden Markov mixture of Gaussians.
隐马尔可夫模型(HMM)是离散时间序列统计建模的主要方法之一,有着广泛的应用。从观察过程中估计HMM通常通过Baum-Welch算法来解决,当初始化合理地接近事实时,该算法在经验上表现良好。这种行为不能用现有的预测对坏局部最优的易感性的理论来解释。在本文中,我们的目标是缩小这一差距,并提供一个框架来表征任何全局最优的足够吸引力盆地,其中Baum-Welch保证在全局最优周围线性收敛到一个“最优”小球。然后,该框架用于确定线性收敛速率和一个足够的初始化区域,用于双分量各向同性隐藏马尔可夫混合高斯函数。
{"title":"Statistical and computational guarantees for the Baum-Welch algorithm","authors":"Fanny Yang, Sivaraman Balakrishnan, M. Wainwright","doi":"10.1109/ALLERTON.2015.7447067","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447067","url":null,"abstract":"The Hidden Markov Model (HMM) is one of the main-stays of statistical modeling of discrete time series and is widely used in many applications. Estimating an HMM from its observation process is often addressed via the Baum-Welch algorithm, which performs well empirically when initialized reasonably close to the truth. This behavior could not be explained by existing theory which predicts susceptibility to bad local optima. In this paper we aim at closing the gap and provide a framework to characterize a sufficient basin of attraction for any global optimum in which Baum-Welch is guaranteed to converge linearly to an “optimally” small ball around the global optimum. The framework is then used to determine the linear rate of convergence and a sufficient initialization region for Baum-Welch applied on a two component isotropic hidden Markov mixture of Gaussians.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122674305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 34
Optimal multi-vehicle adaptive search with entropy objectives 具有熵目标的最优多车自适应搜索
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447085
Huanyu Ding, D. Castañón
The problem of searching for an unknown object occurs in important applications, ranging from security, medicine and defense. Modern sensors have significant processing capabilities that allow for in situ processing and exploitation of the information to select what additional information to collect. In this paper, we discuss a class of dynamic, adaptive search problems involving multiple sensors sensing for a single stationary object, and formulate them as stochastic control problems with imperfect information. The objective of these problems is related to information entropy. This allows for a complete characterization of the optimal strategies and the optimal cost for the resulting finite-horizon stochastic control problems. We show that the computation of optimal policies can be reduced to solving a finite number of strictly concave maximization problems. We further show that the solution can be decoupled into a finite number of scalar concave maximization problems. We illustrate our results with experiments using multiple sensors searching for a single object.
搜索未知物体的问题出现在安全、医学和国防等重要应用中。现代传感器具有重要的处理能力,允许对信息进行现场处理和利用,以选择要收集的附加信息。本文讨论了一类涉及多个传感器感知单个静止目标的动态自适应搜索问题,并将其表述为具有不完全信息的随机控制问题。这些问题的目标与信息熵有关。这允许一个完整的表征的最优策略和最优成本为所得的有限视界随机控制问题。我们证明了最优策略的计算可以简化为求解有限个数的严格凹最大化问题。进一步证明了解可以解耦为有限个标量凹最大化问题。我们通过使用多个传感器搜索单个对象的实验来说明我们的结果。
{"title":"Optimal multi-vehicle adaptive search with entropy objectives","authors":"Huanyu Ding, D. Castañón","doi":"10.1109/ALLERTON.2015.7447085","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447085","url":null,"abstract":"The problem of searching for an unknown object occurs in important applications, ranging from security, medicine and defense. Modern sensors have significant processing capabilities that allow for in situ processing and exploitation of the information to select what additional information to collect. In this paper, we discuss a class of dynamic, adaptive search problems involving multiple sensors sensing for a single stationary object, and formulate them as stochastic control problems with imperfect information. The objective of these problems is related to information entropy. This allows for a complete characterization of the optimal strategies and the optimal cost for the resulting finite-horizon stochastic control problems. We show that the computation of optimal policies can be reduced to solving a finite number of strictly concave maximization problems. We further show that the solution can be decoupled into a finite number of scalar concave maximization problems. We illustrate our results with experiments using multiple sensors searching for a single object.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132745198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Is the direction of greater Granger causal influence the same as the direction of information flow? 格兰杰因果影响较大的方向是否与信息流的方向相同?
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447069
Praveen Venkatesh, P. Grover
Granger causality is an established statistical measure of the “causal influence” that one stochastic process X has on another process Y. Along with its more recent generalization - Directed Information - Granger Causality has been used extensively in neuroscience, and in complex interconnected systems in general, to infer statistical causal influences. More recently, many works compare the Granger causality metrics along forward and reverse links (from X to Y and from Y to X), and interpret the direction of greater causal influence as the “direction of information flow”. In this paper, we question whether the direction yielded by comparing Granger Causality or Directed Information along forward and reverse links is always the same as the direction of information flow. We explore this question using two simple theoretical experiments, in which the true direction of information flow (the “ground truth”) is known by design. The experiments are based on a communication system with a feedback channel, and employ a strategy inspired by the work of Schalkwijk and Kailath. We show that in these experiments, the direction of information flow can be opposite to the direction of greater Granger causal influence or Directed Information. We also provide information-theoretic intuition for why such counterexamples are not surprising, and why Granger causality-based information-flow inferences will only get more tenuous in larger networks. We conclude that one must not use comparison/difference of Granger causality to infer the direction of information flow.
格兰杰因果关系是一个随机过程X对另一个过程y的“因果影响”的既定统计度量。随着其最近的推广-定向信息-格兰杰因果关系已广泛用于神经科学,以及一般复杂的相互关联系统,以推断统计因果影响。最近,许多作品沿着正向和反向链接(从X到Y和从Y到X)比较格兰杰因果关系指标,并将更大因果影响的方向解释为“信息流的方向”。在本文中,我们质疑通过比较格兰杰因果关系或有向信息沿正向和反向链接所得到的方向是否总是与信息流的方向相同。我们使用两个简单的理论实验来探索这个问题,其中信息流的真正方向(“基础真相”)是通过设计知道的。实验基于一个带有反馈通道的通信系统,并采用了一种受Schalkwijk和Kailath工作启发的策略。我们表明,在这些实验中,信息流的方向可以与更大的格兰杰因果影响或定向信息的方向相反。我们还提供了信息理论的直觉,说明为什么这样的反例并不令人惊讶,以及为什么基于格兰杰因果关系的信息流推断只会在更大的网络中变得更加脆弱。我们的结论是,不能使用格兰杰因果关系的比较/差异来推断信息流的方向。
{"title":"Is the direction of greater Granger causal influence the same as the direction of information flow?","authors":"Praveen Venkatesh, P. Grover","doi":"10.1109/ALLERTON.2015.7447069","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447069","url":null,"abstract":"Granger causality is an established statistical measure of the “causal influence” that one stochastic process X has on another process Y. Along with its more recent generalization - Directed Information - Granger Causality has been used extensively in neuroscience, and in complex interconnected systems in general, to infer statistical causal influences. More recently, many works compare the Granger causality metrics along forward and reverse links (from X to Y and from Y to X), and interpret the direction of greater causal influence as the “direction of information flow”. In this paper, we question whether the direction yielded by comparing Granger Causality or Directed Information along forward and reverse links is always the same as the direction of information flow. We explore this question using two simple theoretical experiments, in which the true direction of information flow (the “ground truth”) is known by design. The experiments are based on a communication system with a feedback channel, and employ a strategy inspired by the work of Schalkwijk and Kailath. We show that in these experiments, the direction of information flow can be opposite to the direction of greater Granger causal influence or Directed Information. We also provide information-theoretic intuition for why such counterexamples are not surprising, and why Granger causality-based information-flow inferences will only get more tenuous in larger networks. We conclude that one must not use comparison/difference of Granger causality to infer the direction of information flow.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133970165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
Inferning trees Inferning树
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447164
Mina Karzand, Guy Bresler
We consider the problem of learning an Ising model for the purpose of subsequently performing inference from partial observations. This is in contrast to most other work on graphical model learning, which tries to learn the true underlying graph. This objective requires a lower bound on the strength of edges for identifiability of the model. We show that in the relatively simple case of tree models, the Chow-Liu algorithm learns a distribution with accurate low-order marginals despite the model possibly being non-identifiable. In other words, a model that appears rather different from the truth nevertheless allows to carry out inference accurately.
我们考虑了学习伊辛模型的问题,目的是随后从部分观测进行推理。这与大多数其他关于图形模型学习的工作形成对比,这些工作试图学习真正的底层图。这个目标需要一个边缘强度的下界来保证模型的可识别性。我们表明,在相对简单的树模型情况下,尽管模型可能不可识别,但Chow-Liu算法学习了具有精确低阶边际的分布。换句话说,一个看起来与事实相当不同的模型,却可以准确地进行推理。
{"title":"Inferning trees","authors":"Mina Karzand, Guy Bresler","doi":"10.1109/ALLERTON.2015.7447164","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447164","url":null,"abstract":"We consider the problem of learning an Ising model for the purpose of subsequently performing inference from partial observations. This is in contrast to most other work on graphical model learning, which tries to learn the true underlying graph. This objective requires a lower bound on the strength of edges for identifiability of the model. We show that in the relatively simple case of tree models, the Chow-Liu algorithm learns a distribution with accurate low-order marginals despite the model possibly being non-identifiable. In other words, a model that appears rather different from the truth nevertheless allows to carry out inference accurately.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"369 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114861281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Robust regularized ZF in decentralized Broadcast Channel with correlated CSI noise 具有相关CSI噪声的分散广播信道鲁棒正则化ZF
Pub Date : 2015-09-01 DOI: 10.1109/ALLERTON.2015.7447023
Qianrui Li, Paul de Kerret, D. Gesbert, N. Gresset
We consider in this work the Distributed Channel State Information (DCSI) Broadcast Channel (BC) setting, in which the various Transmitters (TXs) compute elements of the precoder based on their individual estimates of the global multiuser channel matrix. Previous works relative to the DCSI setting assume the estimation errors at different TXs to be uncorrelated, while we consider in contrast in this work that the CSI noises can be correlated. This generalization bridges the gap between the fully distributed and the centralized setting, and offers an avenue to analyze partially centralized networks. In addition, we generalize the regularized Zero Forcing (ZF) precoding by letting each TX use a different regularization coefficient. Building upon random matrix theory tools, we obtain a deterministic equivalent for the rate achieved in the large system limit from which we can optimize the regularization coefficients at different TXs. This extended precoding scheme in which each TX applies the optimal regularization coefficient is denoted as “DCSI Regularized ZF” and we show by numerical simulations that it allows to significantly reduce the negative impact of the distributed CSI configuration and is robust to the distribution of CSI quality level across all TXs.
在这项工作中,我们考虑了分布式信道状态信息(DCSI)广播信道(BC)设置,其中各种发射机(TXs)基于它们对全局多用户信道矩阵的单独估计来计算预编码器的元素。之前关于DCSI设置的工作假设不同TXs的估计误差是不相关的,而我们在这项工作中相反地认为CSI噪声是可以相关的。这种概括弥合了完全分布式和集中式设置之间的差距,并提供了分析部分集中式网络的途径。此外,我们通过让每个TX使用不同的正则化系数来推广正则化零强制(ZF)预编码。在随机矩阵理论工具的基础上,我们得到了在大系统极限下达到的速率的确定性等价,由此我们可以优化不同TXs下的正则化系数。这种扩展的预编码方案,其中每个TX应用最优正则化系数,被表示为“DCSI正则化ZF”,我们通过数值模拟表明,它允许显着减少分布式CSI配置的负面影响,并且对CSI质量水平在所有TX中的分布具有鲁棒性。
{"title":"Robust regularized ZF in decentralized Broadcast Channel with correlated CSI noise","authors":"Qianrui Li, Paul de Kerret, D. Gesbert, N. Gresset","doi":"10.1109/ALLERTON.2015.7447023","DOIUrl":"https://doi.org/10.1109/ALLERTON.2015.7447023","url":null,"abstract":"We consider in this work the Distributed Channel State Information (DCSI) Broadcast Channel (BC) setting, in which the various Transmitters (TXs) compute elements of the precoder based on their individual estimates of the global multiuser channel matrix. Previous works relative to the DCSI setting assume the estimation errors at different TXs to be uncorrelated, while we consider in contrast in this work that the CSI noises can be correlated. This generalization bridges the gap between the fully distributed and the centralized setting, and offers an avenue to analyze partially centralized networks. In addition, we generalize the regularized Zero Forcing (ZF) precoding by letting each TX use a different regularization coefficient. Building upon random matrix theory tools, we obtain a deterministic equivalent for the rate achieved in the large system limit from which we can optimize the regularization coefficients at different TXs. This extended precoding scheme in which each TX applies the optimal regularization coefficient is denoted as “DCSI Regularized ZF” and we show by numerical simulations that it allows to significantly reduce the negative impact of the distributed CSI configuration and is robust to the distribution of CSI quality level across all TXs.","PeriodicalId":112948,"journal":{"name":"2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"143 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117334191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
期刊
2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1