Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174136
Heasung Kim, Taehyun Cho, Jungwoo Lee, W. Shin, H. Poor
This paper deals with the power allocation problem for achieving the upper bound of sum-rate region in energy harvesting downlink channels. We prove that the optimal power allocation policy that maximizes the sum-rate is an increasing function for harvested energy, channel gains, and remaining battery, regardless of the number of users in the downlink channels. We use this proof as a mathematical basis for the construction of a shallow neural network that can fully reflect the increasing property of the optimal policy. This scheme helps us to avoid using big neural networks which requires huge computational resources and causes overfitting. Through experiments, we reveal the inefficiencies and risks of deep neural network that are not optimized enough for the desired policy, and shows that our approach learns a robust policy even with the severe randomness of environments.
{"title":"An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels","authors":"Heasung Kim, Taehyun Cho, Jungwoo Lee, W. Shin, H. Poor","doi":"10.1109/ISIT44484.2020.9174136","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174136","url":null,"abstract":"This paper deals with the power allocation problem for achieving the upper bound of sum-rate region in energy harvesting downlink channels. We prove that the optimal power allocation policy that maximizes the sum-rate is an increasing function for harvested energy, channel gains, and remaining battery, regardless of the number of users in the downlink channels. We use this proof as a mathematical basis for the construction of a shallow neural network that can fully reflect the increasing property of the optimal policy. This scheme helps us to avoid using big neural networks which requires huge computational resources and causes overfitting. Through experiments, we reveal the inefficiencies and risks of deep neural network that are not optimized enough for the desired policy, and shows that our approach learns a robust policy even with the severe randomness of environments.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131006392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Secret sharing scheme is a method in which the secret is divided among finitely many participants by a dealer such that only the legitimate set of participants can recover the secret. The collection of sets of legitimate participants is called the access structure of the secret sharing scheme. There are various ways of constructing secret sharing schemes. Determination of the access structure for a secret sharing scheme is an important problem. Most of the known secret sharing schemes are based on linear codes. A major drawback of secret sharing schemes based on linear codes is that these schemes are susceptible to Tompa- Woll attack. In this paper, we use nonlinear codes to construct secret sharing schemes. These secret sharing schemes perform better than the secret sharing schemes based on linear codes with respect to Tompa-Woll attack.
{"title":"Secret sharing schemes based on Nonlinear codes","authors":"Deepak Agrawal, Smarajit Das, Srinivasan Krishanaswamy","doi":"10.1109/ISIT44484.2020.9174044","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174044","url":null,"abstract":"Secret sharing scheme is a method in which the secret is divided among finitely many participants by a dealer such that only the legitimate set of participants can recover the secret. The collection of sets of legitimate participants is called the access structure of the secret sharing scheme. There are various ways of constructing secret sharing schemes. Determination of the access structure for a secret sharing scheme is an important problem. Most of the known secret sharing schemes are based on linear codes. A major drawback of secret sharing schemes based on linear codes is that these schemes are susceptible to Tompa- Woll attack. In this paper, we use nonlinear codes to construct secret sharing schemes. These secret sharing schemes perform better than the secret sharing schemes based on linear codes with respect to Tompa-Woll attack.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129205504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174156
Xiaowei Wu, Min Qiu, Jinhong Yuan
Partially information coupled turbo codes (PIC-TCs) is a class of spatially coupled turbo codes that can approach the BEC capacity while keeping the encoding and decoding architectures of the underlying component codes unchanged. However, PIC-TCs have significant rate loss compared to its component rate-$frac{1}{3}$ turbo code, and the rate loss increases with the coupling ratio. To absorb the rate loss, in this paper, we propose the partially information coupled duo-binary turbo codes (PIC-dTCs). Given a rate-$frac{1}{3}$ turbo code as the benchmark, we construct a duo-binary turbo code by introducing one extra input to the benchmark code. Then, parts of the information sequence from the original input are coupled to the extra input of the succeeding code blocks. By looking into the graph model of PIC-dTC ensembles, we derive the exact density evolution equations of the PIC-dTC ensembles, and compute their belief propagation decoding thresholds on the binary erasure channel. Simulation results verify the correctness of our theoretical analysis, and also show significant error performance improvement over the uncoupled rate-$frac{1}{3}$ turbo codes and existing designs of spatially coupled turbo codes.
{"title":"Partially Information Coupled Duo-Binary Turbo Codes","authors":"Xiaowei Wu, Min Qiu, Jinhong Yuan","doi":"10.1109/ISIT44484.2020.9174156","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174156","url":null,"abstract":"Partially information coupled turbo codes (PIC-TCs) is a class of spatially coupled turbo codes that can approach the BEC capacity while keeping the encoding and decoding architectures of the underlying component codes unchanged. However, PIC-TCs have significant rate loss compared to its component rate-$frac{1}{3}$ turbo code, and the rate loss increases with the coupling ratio. To absorb the rate loss, in this paper, we propose the partially information coupled duo-binary turbo codes (PIC-dTCs). Given a rate-$frac{1}{3}$ turbo code as the benchmark, we construct a duo-binary turbo code by introducing one extra input to the benchmark code. Then, parts of the information sequence from the original input are coupled to the extra input of the succeeding code blocks. By looking into the graph model of PIC-dTC ensembles, we derive the exact density evolution equations of the PIC-dTC ensembles, and compute their belief propagation decoding thresholds on the binary erasure channel. Simulation results verify the correctness of our theoretical analysis, and also show significant error performance improvement over the uncoupled rate-$frac{1}{3}$ turbo codes and existing designs of spatially coupled turbo codes.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125481568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9173972
A. Mahesh, Nujoom Sageer Karat, B. Rajan
For the problem of embedded index coding, a matrix representation, called a side-information matrix and a metric called min-rank are defined to characterize the length of an optimal embedded index code. An optimal embedded index code for a given embedded index coding problem is shown to be obtainable from the columns of its side information matrix. Further, for a class of embedded index coding problems, called one-sided neighboring side information problems, the min-rank is derived and a transmission scheme which has length equal to this min-rank is presented.
{"title":"Min-rank of Embedded Index Coding Problems","authors":"A. Mahesh, Nujoom Sageer Karat, B. Rajan","doi":"10.1109/ISIT44484.2020.9173972","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9173972","url":null,"abstract":"For the problem of embedded index coding, a matrix representation, called a side-information matrix and a metric called min-rank are defined to characterize the length of an optimal embedded index code. An optimal embedded index code for a given embedded index coding problem is shown to be obtainable from the columns of its side information matrix. Further, for a class of embedded index coding problems, called one-sided neighboring side information problems, the min-rank is derived and a transmission scheme which has length equal to this min-rank is presented.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125499582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9173951
Orhan Ocal, K. Ramchandran
Matrix-matrix multiplication and its derivatives are fundamental linear-algebraic primitives at the core of many modern optimization and machine learning algorithms. We design a new and novel framework for speeding up large-scale matrix-matrix multiplication when the output matrix is known to be sparse, as is true in many applications of interest. Our solution is based on a novel use of product codes which have been studied in the communications literature. In particular, when multiplying two matrices of sizes n × d and d n where the output matrix is (exactly) K-sparse with support× uniformly distributed, our algorithm requires max(O(dK), O(dn)) computations. We also extend our framework to handle the approximately-sparse setting where the output matrix has K-entries that are significantly larger than the rest. In this case, the computational complexity is max(O(dK log2(n)), O(dn log2(n))). We corroborate our findings with numerical simulations that validate our claims.
{"title":"Fast Compressive Large-Scale Matrix-Matrix Multiplication Using Product Codes","authors":"Orhan Ocal, K. Ramchandran","doi":"10.1109/ISIT44484.2020.9173951","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9173951","url":null,"abstract":"Matrix-matrix multiplication and its derivatives are fundamental linear-algebraic primitives at the core of many modern optimization and machine learning algorithms. We design a new and novel framework for speeding up large-scale matrix-matrix multiplication when the output matrix is known to be sparse, as is true in many applications of interest. Our solution is based on a novel use of product codes which have been studied in the communications literature. In particular, when multiplying two matrices of sizes n × d and d n where the output matrix is (exactly) K-sparse with support× uniformly distributed, our algorithm requires max(O(dK), O(dn)) computations. We also extend our framework to handle the approximately-sparse setting where the output matrix has K-entries that are significantly larger than the rest. In this case, the computational complexity is max(O(dK log2(n)), O(dn log2(n))). We corroborate our findings with numerical simulations that validate our claims.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126251234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174242
S. Shekhar, M. Ghavamzadeh, T. Javidi
We consider the problem of binary classification with the caveat that the learner can abstain from declaring a label incurring a cost λ ∈ [0,1/2] in the process. This is referred to as the problem of binary classification with a fixed-cost of abstention. For this problem, we propose an active learning strategy that constructs a non-uniform partition of the input space and focuses sampling in the regions near the decision boundaries. Our proposed algorithm can work in all the commonly used active learning query models, namely membership-query, pool-based and stream-based. We obtain an upper bound on the excess risk of our proposed algorithm under standard smoothness and margin assumptions and demonstrate its minimax near-optimality by deriving a matching (modulo poly-logarithmic factors) lower bound. The achieved minimax rates are always faster than the corresponding rates in the passive setting, and furthermore the improvement increases with larger values of the smoothness and margin parameters.
{"title":"Active Learning for Classification with Abstention","authors":"S. Shekhar, M. Ghavamzadeh, T. Javidi","doi":"10.1109/ISIT44484.2020.9174242","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174242","url":null,"abstract":"We consider the problem of binary classification with the caveat that the learner can abstain from declaring a label incurring a cost λ ∈ [0,1/2] in the process. This is referred to as the problem of binary classification with a fixed-cost of abstention. For this problem, we propose an active learning strategy that constructs a non-uniform partition of the input space and focuses sampling in the regions near the decision boundaries. Our proposed algorithm can work in all the commonly used active learning query models, namely membership-query, pool-based and stream-based. We obtain an upper bound on the excess risk of our proposed algorithm under standard smoothness and margin assumptions and demonstrate its minimax near-optimality by deriving a matching (modulo poly-logarithmic factors) lower bound. The achieved minimax rates are always faster than the corresponding rates in the passive setting, and furthermore the improvement increases with larger values of the smoothness and margin parameters.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122217759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174124
Kuan-Yun Lee, T. Courtade
We establish a nonasymptotic lower bound on the L2 minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).
{"title":"Linear Models are Most Favorable among Generalized Linear Models","authors":"Kuan-Yun Lee, T. Courtade","doi":"10.1109/ISIT44484.2020.9174124","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174124","url":null,"abstract":"We establish a nonasymptotic lower bound on the L2 minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126879060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174440
Adarsh M. Subramaniam, A. Heidarzadeh, A. K. Pradhan, K. Narayanan
This work considers the distributed multivariate polynomial evaluation (DMPE) problem using a master-worker framework, which was originally considered by Yu et al., where Lagrange Coded Computing (LCC) was proposed as a coded computation scheme to provide resilience against stragglers for the DMPE problem. In this work, we propose a variant of the LCC scheme, termed Product Lagrange Coded Computing (PLCC), by combining ideas from classical product codes and LCC. The main advantage of PLCC is that they are more numerically stable than LCC; however, their resilience to stragglers is sub-optimal.
{"title":"Product Lagrange Coded Computing","authors":"Adarsh M. Subramaniam, A. Heidarzadeh, A. K. Pradhan, K. Narayanan","doi":"10.1109/ISIT44484.2020.9174440","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174440","url":null,"abstract":"This work considers the distributed multivariate polynomial evaluation (DMPE) problem using a master-worker framework, which was originally considered by Yu et al., where Lagrange Coded Computing (LCC) was proposed as a coded computation scheme to provide resilience against stragglers for the DMPE problem. In this work, we propose a variant of the LCC scheme, termed Product Lagrange Coded Computing (PLCC), by combining ideas from classical product codes and LCC. The main advantage of PLCC is that they are more numerically stable than LCC; however, their resilience to stragglers is sub-optimal.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120973786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174465
J. Melbourne, T. Tkocz
We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log2e of the usual Shannon entropy. With the additional assumption that the variable is monotone we obtain a sharp bound of loge.
{"title":"On the Rényi Entropy of Log-Concave Sequences","authors":"J. Melbourne, T. Tkocz","doi":"10.1109/ISIT44484.2020.9174465","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174465","url":null,"abstract":"We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log2e of the usual Shannon entropy. With the additional assumption that the variable is monotone we obtain a sharp bound of loge.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"110 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116115999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1109/ISIT44484.2020.9174245
Nir Shlezinger, S. Rini, Yonina C. Eldar
Federated learning (FL) refers to the adaptation of a central model based on data sets available at multiple remote users. Two of the common challenges encountered in FL are the fact that training sets obtained by different users are commonly heterogeneous, i.e., arise from different sample distributions, and the need to communicate large amounts of data between the users and the central server over the typically expensive up-link channel. In this work we formulate the problem of FL in which different clusters of users observe labeled samples drawn from different distributions, while operating under constraints on the communication overhead. For such settings, we identify that the combination of statistical heterogeneity and communication constraints induces a tradeoff between the ability of the users of each cluster to learn a proper model and the accuracy in aggregating these models into a global inference rule. We propose an algorithm based on multi-source adaptation methods for such communication-aware clustered FL scenarios which allows to balance these performance measures, and demonstrate its ability to achieve improved inference over conventional federated averaging without inducing additional communication overhead.
{"title":"The Communication-Aware Clustered Federated Learning Problem","authors":"Nir Shlezinger, S. Rini, Yonina C. Eldar","doi":"10.1109/ISIT44484.2020.9174245","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174245","url":null,"abstract":"Federated learning (FL) refers to the adaptation of a central model based on data sets available at multiple remote users. Two of the common challenges encountered in FL are the fact that training sets obtained by different users are commonly heterogeneous, i.e., arise from different sample distributions, and the need to communicate large amounts of data between the users and the central server over the typically expensive up-link channel. In this work we formulate the problem of FL in which different clusters of users observe labeled samples drawn from different distributions, while operating under constraints on the communication overhead. For such settings, we identify that the combination of statistical heterogeneity and communication constraints induces a tradeoff between the ability of the users of each cluster to learn a proper model and the accuracy in aggregating these models into a global inference rule. We propose an algorithm based on multi-source adaptation methods for such communication-aware clustered FL scenarios which allows to balance these performance measures, and demonstrate its ability to achieve improved inference over conventional federated averaging without inducing additional communication overhead.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115275471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：