Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919698
Xuan Wang, S. Mou, S. Sundaram
Consensus-based distributed algorithms have been the key to many problems arising in multi-agent systems including reinforcement learning [1], [2], formation control [3], [4], task allocation [5]and so on. Byconsensushere is meant that all agents in the network reach an agreement regarding a certain quantity of interest [6], [7]. Bydistributedhere is meant that the whole multi-agent system achieve global objectives by only local coordination among nearby neighbors [8]. On one hand, the absence of central controllers in multi-agent systems make them inherently robust against individual agent/link failures. On the other hand, the high dependence of the whole system on local coordination also raises a significant concern that algorithms for multi-agent networks may be crashed down in the presence of even one malicious agent [9]. This has motivated us to develop methodologies to achieveresiliencein order to guarantee nice performance for consensus-based distributed algorithms especially in hostile environment. One challenge along this direction comes from the fact that each agent is usually with locally available information, which makes it very difficult to identify or isolate malicious agents [10]. The authors of [11]–[13]have achieved significant progress by showing that given $N$adversarial nodes under Byzantine attacks, there exists a strategy for normal agents to achieve consensus if the network connectivity is $2 N+1.$These results are usually computationally expensive, assume the network topology to be all-to-all networks, or require normal agents to be aware of non-local information. Most recently the authors of [14], [15]have investigated consensus-based distributed optimizations under adversarial agents. They have introduced a local filtering mechanism which allows each agent to discard the most extreme values in their neighborhood at each step. This is not directly applicable to consensus-based distributed computation algorithms [16]–[19], in which extreme values may come from the local constraints instead of malicious agents. Thus in this talk we will present a new approach developed in [9], which achieves automated resilience without the identification of malicious agents for consensus-based distributed algorithms based on intersection of convex hulls [20].
{"title":"Resilience for Consensus-based Distributed Algorithms in Hostile Environment†","authors":"Xuan Wang, S. Mou, S. Sundaram","doi":"10.1109/ALLERTON.2019.8919698","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919698","url":null,"abstract":"Consensus-based distributed algorithms have been the key to many problems arising in multi-agent systems including reinforcement learning [1], [2], formation control [3], [4], task allocation [5]and so on. Byconsensushere is meant that all agents in the network reach an agreement regarding a certain quantity of interest [6], [7]. Bydistributedhere is meant that the whole multi-agent system achieve global objectives by only local coordination among nearby neighbors [8]. On one hand, the absence of central controllers in multi-agent systems make them inherently robust against individual agent/link failures. On the other hand, the high dependence of the whole system on local coordination also raises a significant concern that algorithms for multi-agent networks may be crashed down in the presence of even one malicious agent [9]. This has motivated us to develop methodologies to achieveresiliencein order to guarantee nice performance for consensus-based distributed algorithms especially in hostile environment. One challenge along this direction comes from the fact that each agent is usually with locally available information, which makes it very difficult to identify or isolate malicious agents [10]. The authors of [11]–[13]have achieved significant progress by showing that given $N$adversarial nodes under Byzantine attacks, there exists a strategy for normal agents to achieve consensus if the network connectivity is $2 N+1.$These results are usually computationally expensive, assume the network topology to be all-to-all networks, or require normal agents to be aware of non-local information. Most recently the authors of [14], [15]have investigated consensus-based distributed optimizations under adversarial agents. They have introduced a local filtering mechanism which allows each agent to discard the most extreme values in their neighborhood at each step. This is not directly applicable to consensus-based distributed computation algorithms [16]–[19], in which extreme values may come from the local constraints instead of malicious agents. Thus in this talk we will present a new approach developed in [9], which achieves automated resilience without the identification of malicious agents for consensus-based distributed algorithms based on intersection of convex hulls [20].","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127446269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919880
T. Doan, J. Romberg
We consider two-time-scale stochastic approximation for finding the solution of a linear system of two equations. Such methods have found broad applications in many areas, especially in machine learning and reinforcement learning. A critical question in this area is to analyze the convergence rates (or sample complexity) of this method, which has not been fully addressed in the existing literature. Our contribution in this paper is, therefore, to provide a new analysis for the finite-time performance of the two-time-scale stochastic approximation. Our key idea is to leverage the common techniques from optimization, in particular, we utilize a residual function to capture the coupling between the two iterates. This will allow us to explicit design the two step sizes used by the two iterations as well as to provide a finite-time error bound on the convergence of the two iterates. Our analysis in this paper provides another aspect to the existing techniques in the literature of two-time-scale stochastic approximation, which we believe is more elegant and can be more applicable to many scenarios.
{"title":"Linear Two-Time-Scale Stochastic Approximation A Finite-Time Analysis","authors":"T. Doan, J. Romberg","doi":"10.1109/ALLERTON.2019.8919880","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919880","url":null,"abstract":"We consider two-time-scale stochastic approximation for finding the solution of a linear system of two equations. Such methods have found broad applications in many areas, especially in machine learning and reinforcement learning. A critical question in this area is to analyze the convergence rates (or sample complexity) of this method, which has not been fully addressed in the existing literature. Our contribution in this paper is, therefore, to provide a new analysis for the finite-time performance of the two-time-scale stochastic approximation. Our key idea is to leverage the common techniques from optimization, in particular, we utilize a residual function to capture the coupling between the two iterates. This will allow us to explicit design the two step sizes used by the two iterations as well as to provide a finite-time error bound on the convergence of the two iterates. Our analysis in this paper provides another aspect to the existing techniques in the literature of two-time-scale stochastic approximation, which we believe is more elegant and can be more applicable to many scenarios.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121186517","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919749
A. Sahai, J. Sanz, Vignesh Subramanian, Caryn Tran, Kailas Vodrahalli
In this work we examine the problem of learning to cooperate in the context of wireless communication. We consider the two agent setting where agents must learn modulation and demodulation schemes that enable them to communicate with each other in the presence of a power-constrained additive white Gaussian noise (AWGN) channel. We investigate whether learning is possible under different levels of information sharing between distributed agents that are not necessarily co-designed. We make use of the “Echo” protocol, a learning protocol where an agent hears, understands, and repeats (echoes) back the message received from another agent. Each agent uses what it sends and receives to train itself to communicate. To capture the idea of cooperation between agents that are “not necessarily co-designed,” we use two different populations of function approximators – neural networks and polynomials. In addition to diverse learning agents, we include non-learning agents that use fixed standardized modulation protocols such as QPSK and 16QAM. This is used to verify that the Echo approach to learning to communicate works independent of the inner workings of the agents, and that learning agents can not only learn to match the communication expectations of others, but can also collaboratively invent a successful communication approach from independent random initializations. In addition to simulation-based experiments, we implement the Echo protocol in physical software-defined radio experiments to verify that it can work with real radios. To explore the continuum between tight co-design of learning agents and independently designed agents, we study how learning is impacted by different levels of information sharing – including sharing training symbols, sharing intermediate loss information, and sharing full gradient information. The resulting learning techniques span supervised learning and reinforcement learning. We find that in general, co-design (increased information sharing) accelerates learning and that this effect becomes more pronounced as the communication task becomes harder.
{"title":"Learning to Communicate with Limited Co-design","authors":"A. Sahai, J. Sanz, Vignesh Subramanian, Caryn Tran, Kailas Vodrahalli","doi":"10.1109/ALLERTON.2019.8919749","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919749","url":null,"abstract":"In this work we examine the problem of learning to cooperate in the context of wireless communication. We consider the two agent setting where agents must learn modulation and demodulation schemes that enable them to communicate with each other in the presence of a power-constrained additive white Gaussian noise (AWGN) channel. We investigate whether learning is possible under different levels of information sharing between distributed agents that are not necessarily co-designed. We make use of the “Echo” protocol, a learning protocol where an agent hears, understands, and repeats (echoes) back the message received from another agent. Each agent uses what it sends and receives to train itself to communicate. To capture the idea of cooperation between agents that are “not necessarily co-designed,” we use two different populations of function approximators – neural networks and polynomials. In addition to diverse learning agents, we include non-learning agents that use fixed standardized modulation protocols such as QPSK and 16QAM. This is used to verify that the Echo approach to learning to communicate works independent of the inner workings of the agents, and that learning agents can not only learn to match the communication expectations of others, but can also collaboratively invent a successful communication approach from independent random initializations. In addition to simulation-based experiments, we implement the Echo protocol in physical software-defined radio experiments to verify that it can work with real radios. To explore the continuum between tight co-design of learning agents and independently designed agents, we study how learning is impacted by different levels of information sharing – including sharing training symbols, sharing intermediate loss information, and sharing full gradient information. The resulting learning techniques span supervised learning and reinforcement learning. We find that in general, co-design (increased information sharing) accelerates learning and that this effect becomes more pronounced as the communication task becomes harder.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"17 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127005449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919868
Nasser Aldaghri, Hessam Mahdavifar
Cryptographic protocols are often implemented at upper layers of communication networks, while error-correcting codes are employed at the physical layer. In this paper, we consider utilizing readily-available physical layer functions, such as encoders and decoders, together with shared keys to provide a threshold-type security scheme. To this end, the effect of physical layer communication is abstracted out and the channels between the legitimate parties, Alice and Bob, and the eaves-dropper Eve are assumed to be noiseless. We introduce a model for threshold-secure coding, where Alice and Bob communicate using a shared key in such a way that Eve does not get any information, in an information-theoretic sense, about the key as well as about any subset of the input symbols of size up to a certain threshold. Then, a framework is provided for constructing threshold-secure codes form linear block codes while characterizing the requirements to satisfy the reliability and security conditions. Moreover, we propose a threshold-secure coding scheme, based on Reed-Muller (RM) codes, that meets security and reliability conditions. Furthermore, it is shown that the encoder and the decoder of the scheme can be implemented efficiently with quasi-linear time complexity. In particular, a low-complexity successive cancellation decoder is shown for the RM-based scheme. Also, the scheme is flexible and can be adapted given any key length.
{"title":"Threshold-Secure Coding with Shared Key","authors":"Nasser Aldaghri, Hessam Mahdavifar","doi":"10.1109/ALLERTON.2019.8919868","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919868","url":null,"abstract":"Cryptographic protocols are often implemented at upper layers of communication networks, while error-correcting codes are employed at the physical layer. In this paper, we consider utilizing readily-available physical layer functions, such as encoders and decoders, together with shared keys to provide a threshold-type security scheme. To this end, the effect of physical layer communication is abstracted out and the channels between the legitimate parties, Alice and Bob, and the eaves-dropper Eve are assumed to be noiseless. We introduce a model for threshold-secure coding, where Alice and Bob communicate using a shared key in such a way that Eve does not get any information, in an information-theoretic sense, about the key as well as about any subset of the input symbols of size up to a certain threshold. Then, a framework is provided for constructing threshold-secure codes form linear block codes while characterizing the requirements to satisfy the reliability and security conditions. Moreover, we propose a threshold-secure coding scheme, based on Reed-Muller (RM) codes, that meets security and reliability conditions. Furthermore, it is shown that the encoder and the decoder of the scheme can be implemented efficiently with quasi-linear time complexity. In particular, a low-complexity successive cancellation decoder is shown for the RM-based scheme. Also, the scheme is flexible and can be adapted given any key length.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"400 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126672137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919795
Mohit Agarwal, Raghupathy Sivakumar
Eye-blinks are known to substantially contaminate EEG signals, and thereby severely impact the decoding of EEG signals in various medical and scientific applications. In this work, we consider the problem of eye-blink detection that can then be employed to reliably remove eye-blinks from EEG signals. We propose a fully automated and unsupervised eyeblink detection algorithm, Blink that self-learns user-specific brainwave profiles for eye-blinks. Hence, Blink does away with any user training or manual inspection requirements. Blink functions on a single channel EEG, and is capable of estimating the start and end timestamps of eye-blinks in a precise manner. We collect four different eye-blink datasets and annotate 2300+ eye-blinks to evaluate the robustness performance of Blink across headsets (OpenBCI and Muse), eye-blink types (voluntary and involuntary), and various user activities (watching a video, reading an article, and attending to an external stimulation). The Blink algorithm performs consistently with an accuracy of over 98% for all the tasks with an average precision of 0.934. The source code and annotated datasets are released publicly for reproducibility and further research. To the best of our knowledge, this is the first ever annotated eye-blink EEG dataset released in the public domain.
{"title":"Blink: A Fully Automated Unsupervised Algorithm for Eye-Blink Detection in EEG Signals","authors":"Mohit Agarwal, Raghupathy Sivakumar","doi":"10.1109/ALLERTON.2019.8919795","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919795","url":null,"abstract":"Eye-blinks are known to substantially contaminate EEG signals, and thereby severely impact the decoding of EEG signals in various medical and scientific applications. In this work, we consider the problem of eye-blink detection that can then be employed to reliably remove eye-blinks from EEG signals. We propose a fully automated and unsupervised eyeblink detection algorithm, Blink that self-learns user-specific brainwave profiles for eye-blinks. Hence, Blink does away with any user training or manual inspection requirements. Blink functions on a single channel EEG, and is capable of estimating the start and end timestamps of eye-blinks in a precise manner. We collect four different eye-blink datasets and annotate 2300+ eye-blinks to evaluate the robustness performance of Blink across headsets (OpenBCI and Muse), eye-blink types (voluntary and involuntary), and various user activities (watching a video, reading an article, and attending to an external stimulation). The Blink algorithm performs consistently with an accuracy of over 98% for all the tasks with an average precision of 0.934. The source code and annotated datasets are released publicly for reproducibility and further research. To the best of our knowledge, this is the first ever annotated eye-blink EEG dataset released in the public domain.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"05 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130603546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919755
Elie Atallah, N. Rahnavard, Chinwendu Enyioha
In this paper, an asynchronous random projection algorithm is introduced to solve a distributed constrained convex optimization problem over a time-varying multi-agent network. In this asynchronous case, each agent computes its estimate by exchanging information with its neighbors within a bounded delay lapse. For diminishing uncoordinated stepsizes and some standard conditions on the gradient errors, we provide a convergence analysis of Distributed Asynchronous Random Projection Algorithm (DARPA) to the same optimal point under an arbitrary uniformly bounded delay.
针对时变多智能体网络中的分布式约束凸优化问题，提出了一种异步随机投影算法。在这种异步情况下，每个代理通过在有限的延迟延时内与其邻居交换信息来计算其估计。为了减少非协调步长和梯度误差的一些标准条件，我们给出了在任意均匀有界延迟下分布式异步随机投影算法(Distributed Asynchronous Random Projection Algorithm, DARPA)收敛到同一最优点的分析。
{"title":"Distributed Asynchronous Random Projection Algorithm (DARPA) with Arbitrary Uniformly Bounded Delay","authors":"Elie Atallah, N. Rahnavard, Chinwendu Enyioha","doi":"10.1109/ALLERTON.2019.8919755","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919755","url":null,"abstract":"In this paper, an asynchronous random projection algorithm is introduced to solve a distributed constrained convex optimization problem over a time-varying multi-agent network. In this asynchronous case, each agent computes its estimate by exchanging information with its neighbors within a bounded delay lapse. For diminishing uncoordinated stepsizes and some standard conditions on the gradient errors, we provide a convergence analysis of Distributed Asynchronous Random Projection Algorithm (DARPA) to the same optimal point under an arbitrary uniformly bounded delay.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125910682","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919782
Kursat Rasim Mestav, L. Tong
The problem of achieving the fast-timescale state estimation for a power system with limited deployment of phasor measurement units is considered. A deep neural-network architecture that integrates bad-data detection, data cleansing, and the minimum mean squared error state estimation is developed. It includes a universal bad-data detection and a Bayesian state estimation subnetworks. A novel universal bad-data detection technique is proposed that requires no knowledge about data distributions under regular and irregular operating conditions. The subnetwork for universal bad-data detection consists of an inverse generative model and a coincidence test. It is implemented through the training of a generative adversary network and an auto-encoder using slow-timescale historical data. The Bayesian state estimation subnetwork is trained through a generative adversary network with embedded physical models of the power system. Comparing with the conventional weighted least squares approach to state estimation, the proposed minimum mean-squared error state estimator does not require observability. Simulations demonstrate orders of magnitude improvement in estimation accuracy and online computation costs of/ver the state-of-the-art solutions.
{"title":"Learning the Unobservable: High-Resolution State Estimation via Deep Learning","authors":"Kursat Rasim Mestav, L. Tong","doi":"10.1109/ALLERTON.2019.8919782","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919782","url":null,"abstract":"The problem of achieving the fast-timescale state estimation for a power system with limited deployment of phasor measurement units is considered. A deep neural-network architecture that integrates bad-data detection, data cleansing, and the minimum mean squared error state estimation is developed. It includes a universal bad-data detection and a Bayesian state estimation subnetworks. A novel universal bad-data detection technique is proposed that requires no knowledge about data distributions under regular and irregular operating conditions. The subnetwork for universal bad-data detection consists of an inverse generative model and a coincidence test. It is implemented through the training of a generative adversary network and an auto-encoder using slow-timescale historical data. The Bayesian state estimation subnetwork is trained through a generative adversary network with embedded physical models of the power system. Comparing with the conventional weighted least squares approach to state estimation, the proposed minimum mean-squared error state estimator does not require observability. Simulations demonstrate orders of magnitude improvement in estimation accuracy and online computation costs of/ver the state-of-the-art solutions.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123964812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919901
B. Song, A. Trachtenberg
We consider the problem of reconciling similar, but remote, strings with minimum communication complexity. This “string reconciliation” problem is a fundamental building block for a variety of networking applications, including those that maintain large-scale distributed networks and perform remote file synchronization. We present the novel Recursive Content-Dependent Shingling (RCDS) protocol that is computationally practical for large strings and scales linearly with the edit distance between the remote strings. We provide comparisons to the performance of rsync, one of the most popular file synchronization tools in active use. Our experiments show that, with minimal engineering, RCDS outperforms the heavily optimized rsync in reconciling release revisions for about 51% of the 5000 top starred git repositories on GitHub. The improvement is particularly evident for repositories that see frequent, but small, updates.
{"title":"Scalable String Reconciliation by Recursive Content-Dependent Shingling","authors":"B. Song, A. Trachtenberg","doi":"10.1109/ALLERTON.2019.8919901","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919901","url":null,"abstract":"We consider the problem of reconciling similar, but remote, strings with minimum communication complexity. This “string reconciliation” problem is a fundamental building block for a variety of networking applications, including those that maintain large-scale distributed networks and perform remote file synchronization. We present the novel Recursive Content-Dependent Shingling (RCDS) protocol that is computationally practical for large strings and scales linearly with the edit distance between the remote strings. We provide comparisons to the performance of rsync, one of the most popular file synchronization tools in active use. Our experiments show that, with minimal engineering, RCDS outperforms the heavily optimized rsync in reconciling release revisions for about 51% of the 5000 top starred git repositories on GitHub. The improvement is particularly evident for repositories that see frequent, but small, updates.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117129531","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919769
Tolga Ergen, Mert Pilanci
We consider non-convex training of shallow neural networks and introduce a convex relaxation approach with theoretical guarantees. For the single neuron case, we prove that the relaxation preserves the location of the global minimum under a planted model assumption. Therefore, a globally optimal solution can be efficiently found via a gradient method. We show that gradient descent applied on the relaxation always outperforms gradient descent on the original non-convex loss with no additional computational cost. We then characterize this relaxation as a regularizer and further introduce extensions to multineuron single hidden layer networks.
{"title":"Convex Optimization for Shallow Neural Networks","authors":"Tolga Ergen, Mert Pilanci","doi":"10.1109/ALLERTON.2019.8919769","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919769","url":null,"abstract":"We consider non-convex training of shallow neural networks and introduce a convex relaxation approach with theoretical guarantees. For the single neuron case, we prove that the relaxation preserves the location of the global minimum under a planted model assumption. Therefore, a globally optimal solution can be efficiently found via a gradient method. We show that gradient descent applied on the relaxation always outperforms gradient descent on the original non-convex loss with no additional computational cost. We then characterize this relaxation as a regularizer and further introduce extensions to multineuron single hidden layer networks.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121494842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-09-01DOI: 10.1109/ALLERTON.2019.8919726
Yousef AlHassoun, Faisal Alotaibi, A. E. Gamal, H. E. Gamal
Recently, coded caching techniques have received tremendous attention due to its significant gain in reducing the cost of delivery rate. However, this gain was only considered with the assumption of free placement phase. Motivated by our recent result of coded caching, we focus here on minimizing the overall rate of the caching network by capturing the transmission cost of the placement and delivery phases under limited storage memory at the end user. We model the dynamic nature of the network through a cost structure that allows for varying the network architecture and cost per transmission across the two phases of caching. The optimal caching decision for the worst case scenario with memory constraint is provided. Moreover, analysis of the delivery phase is proposed where trade-offs between system parameters, memory, and delivery rate are considered. Interestingly, we show that there are regions where the uncoded caching scheme outperforms the coded caching scheme. Finally, we provide numerical results to support and demonstrate our findings.
{"title":"Efficient Coded Caching with Limited Memory","authors":"Yousef AlHassoun, Faisal Alotaibi, A. E. Gamal, H. E. Gamal","doi":"10.1109/ALLERTON.2019.8919726","DOIUrl":"https://doi.org/10.1109/ALLERTON.2019.8919726","url":null,"abstract":"Recently, coded caching techniques have received tremendous attention due to its significant gain in reducing the cost of delivery rate. However, this gain was only considered with the assumption of free placement phase. Motivated by our recent result of coded caching, we focus here on minimizing the overall rate of the caching network by capturing the transmission cost of the placement and delivery phases under limited storage memory at the end user. We model the dynamic nature of the network through a cost structure that allows for varying the network architecture and cost per transmission across the two phases of caching. The optimal caching decision for the worst case scenario with memory constraint is provided. Moreover, analysis of the delivery phase is proposed where trade-offs between system parameters, memory, and delivery rate are considered. Interestingly, we show that there are regions where the uncoded caching scheme outperforms the coded caching scheme. Finally, we provide numerical results to support and demonstrate our findings.","PeriodicalId":120479,"journal":{"name":"2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121032831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：