首页 > 最新文献

2020 IEEE International Symposium on Information Theory (ISIT)最新文献

英文 中文
An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels 能量采集下行信道中速率最大化的高效神经网络结构
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174136
Heasung Kim, Taehyun Cho, Jungwoo Lee, W. Shin, H. Poor
This paper deals with the power allocation problem for achieving the upper bound of sum-rate region in energy harvesting downlink channels. We prove that the optimal power allocation policy that maximizes the sum-rate is an increasing function for harvested energy, channel gains, and remaining battery, regardless of the number of users in the downlink channels. We use this proof as a mathematical basis for the construction of a shallow neural network that can fully reflect the increasing property of the optimal policy. This scheme helps us to avoid using big neural networks which requires huge computational resources and causes overfitting. Through experiments, we reveal the inefficiencies and risks of deep neural network that are not optimized enough for the desired policy, and shows that our approach learns a robust policy even with the severe randomness of environments.
本文研究了能量收集下行信道中实现和速率区域上界的功率分配问题。我们证明,无论下行信道中的用户数量如何,使和率最大化的最优功率分配策略是收获能量、信道增益和剩余电池的递增函数。我们将这一证明作为构建一个能充分反映最优策略增长特性的浅神经网络的数学基础。该方案帮助我们避免使用需要大量计算资源和导致过拟合的大型神经网络。通过实验,我们揭示了深度神经网络的低效率和风险,没有针对期望的策略进行足够的优化,并表明我们的方法即使在环境的严重随机性下也能学习到稳健的策略。
{"title":"An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels","authors":"Heasung Kim, Taehyun Cho, Jungwoo Lee, W. Shin, H. Poor","doi":"10.1109/ISIT44484.2020.9174136","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174136","url":null,"abstract":"This paper deals with the power allocation problem for achieving the upper bound of sum-rate region in energy harvesting downlink channels. We prove that the optimal power allocation policy that maximizes the sum-rate is an increasing function for harvested energy, channel gains, and remaining battery, regardless of the number of users in the downlink channels. We use this proof as a mathematical basis for the construction of a shallow neural network that can fully reflect the increasing property of the optimal policy. This scheme helps us to avoid using big neural networks which requires huge computational resources and causes overfitting. Through experiments, we reveal the inefficiencies and risks of deep neural network that are not optimized enough for the desired policy, and shows that our approach learns a robust policy even with the severe randomness of environments.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131006392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Secret sharing schemes based on Nonlinear codes 基于非线性码的秘密共享方案
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174044
Deepak Agrawal, Smarajit Das, Srinivasan Krishanaswamy
Secret sharing scheme is a method in which the secret is divided among finitely many participants by a dealer such that only the legitimate set of participants can recover the secret. The collection of sets of legitimate participants is called the access structure of the secret sharing scheme. There are various ways of constructing secret sharing schemes. Determination of the access structure for a secret sharing scheme is an important problem. Most of the known secret sharing schemes are based on linear codes. A major drawback of secret sharing schemes based on linear codes is that these schemes are susceptible to Tompa- Woll attack. In this paper, we use nonlinear codes to construct secret sharing schemes. These secret sharing schemes perform better than the secret sharing schemes based on linear codes with respect to Tompa-Woll attack.
秘密共享方案是一种由交易商将秘密分配给有限多个参与者的方法,只有合法的参与者才能获得秘密。合法参与者集合的集合称为秘密共享方案的访问结构。构建秘密共享方案的方法多种多样。秘密共享方案访问结构的确定是一个重要的问题。大多数已知的秘密共享方案是基于线性码的。基于线性码的秘密共享方案的一个主要缺点是这些方案容易受到Tompa- Woll攻击。在本文中,我们使用非线性码来构造秘密共享方案。对于Tompa-Woll攻击,这些秘密共享方案的性能优于基于线性码的秘密共享方案。
{"title":"Secret sharing schemes based on Nonlinear codes","authors":"Deepak Agrawal, Smarajit Das, Srinivasan Krishanaswamy","doi":"10.1109/ISIT44484.2020.9174044","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174044","url":null,"abstract":"Secret sharing scheme is a method in which the secret is divided among finitely many participants by a dealer such that only the legitimate set of participants can recover the secret. The collection of sets of legitimate participants is called the access structure of the secret sharing scheme. There are various ways of constructing secret sharing schemes. Determination of the access structure for a secret sharing scheme is an important problem. Most of the known secret sharing schemes are based on linear codes. A major drawback of secret sharing schemes based on linear codes is that these schemes are susceptible to Tompa- Woll attack. In this paper, we use nonlinear codes to construct secret sharing schemes. These secret sharing schemes perform better than the secret sharing schemes based on linear codes with respect to Tompa-Woll attack.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129205504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Partially Information Coupled Duo-Binary Turbo Codes 部分信息耦合的双二进制Turbo码
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174156
Xiaowei Wu, Min Qiu, Jinhong Yuan
Partially information coupled turbo codes (PIC-TCs) is a class of spatially coupled turbo codes that can approach the BEC capacity while keeping the encoding and decoding architectures of the underlying component codes unchanged. However, PIC-TCs have significant rate loss compared to its component rate-$frac{1}{3}$ turbo code, and the rate loss increases with the coupling ratio. To absorb the rate loss, in this paper, we propose the partially information coupled duo-binary turbo codes (PIC-dTCs). Given a rate-$frac{1}{3}$ turbo code as the benchmark, we construct a duo-binary turbo code by introducing one extra input to the benchmark code. Then, parts of the information sequence from the original input are coupled to the extra input of the succeeding code blocks. By looking into the graph model of PIC-dTC ensembles, we derive the exact density evolution equations of the PIC-dTC ensembles, and compute their belief propagation decoding thresholds on the binary erasure channel. Simulation results verify the correctness of our theoretical analysis, and also show significant error performance improvement over the uncoupled rate-$frac{1}{3}$ turbo codes and existing designs of spatially coupled turbo codes.
部分信息耦合turbo码(pic - tc)是一类空间耦合的turbo码,它可以在保持底层组件码的编码和解码结构不变的情况下接近BEC容量。但pic - tc相对于其组件速率$frac{1}{3}$ turbo码具有显著的速率损耗,且速率损耗随耦合比的增大而增大。为了吸收速率损失,本文提出了部分信息耦合的双二进制turbo码(pic - dtc)。给定一个rate-$frac{1}{3}$ turbo代码作为基准,我们通过在基准代码中引入一个额外的输入来构造一个双二进制turbo代码。然后,来自原始输入的部分信息序列耦合到后续代码块的额外输入。通过研究PIC-dTC集成的图模型,导出了PIC-dTC集成的精确密度演化方程,并计算了它们在二进制擦除信道上的信念传播译码阈值。仿真结果验证了理论分析的正确性,并且与不耦合率-$frac{1}{3}$ turbo码和现有空间耦合turbo码相比,误差性能得到了显著改善。
{"title":"Partially Information Coupled Duo-Binary Turbo Codes","authors":"Xiaowei Wu, Min Qiu, Jinhong Yuan","doi":"10.1109/ISIT44484.2020.9174156","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174156","url":null,"abstract":"Partially information coupled turbo codes (PIC-TCs) is a class of spatially coupled turbo codes that can approach the BEC capacity while keeping the encoding and decoding architectures of the underlying component codes unchanged. However, PIC-TCs have significant rate loss compared to its component rate-$frac{1}{3}$ turbo code, and the rate loss increases with the coupling ratio. To absorb the rate loss, in this paper, we propose the partially information coupled duo-binary turbo codes (PIC-dTCs). Given a rate-$frac{1}{3}$ turbo code as the benchmark, we construct a duo-binary turbo code by introducing one extra input to the benchmark code. Then, parts of the information sequence from the original input are coupled to the extra input of the succeeding code blocks. By looking into the graph model of PIC-dTC ensembles, we derive the exact density evolution equations of the PIC-dTC ensembles, and compute their belief propagation decoding thresholds on the binary erasure channel. Simulation results verify the correctness of our theoretical analysis, and also show significant error performance improvement over the uncoupled rate-$frac{1}{3}$ turbo codes and existing designs of spatially coupled turbo codes.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125481568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Min-rank of Embedded Index Coding Problems 嵌入式索引编码问题的最小秩
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9173972
A. Mahesh, Nujoom Sageer Karat, B. Rajan
For the problem of embedded index coding, a matrix representation, called a side-information matrix and a metric called min-rank are defined to characterize the length of an optimal embedded index code. An optimal embedded index code for a given embedded index coding problem is shown to be obtainable from the columns of its side information matrix. Further, for a class of embedded index coding problems, called one-sided neighboring side information problems, the min-rank is derived and a transmission scheme which has length equal to this min-rank is presented.
对于嵌入式索引编码问题,定义了一种称为侧信息矩阵的矩阵表示和一种称为min-rank的度量来表征最优嵌入式索引编码的长度。对于给定的嵌入式索引编码问题,可以从其侧信息矩阵的列中获得最优嵌入式索引编码。进一步,针对一类嵌入式索引编码问题——单侧邻边信息问题,导出了最小秩,并给出了长度等于该最小秩的传输方案。
{"title":"Min-rank of Embedded Index Coding Problems","authors":"A. Mahesh, Nujoom Sageer Karat, B. Rajan","doi":"10.1109/ISIT44484.2020.9173972","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9173972","url":null,"abstract":"For the problem of embedded index coding, a matrix representation, called a side-information matrix and a metric called min-rank are defined to characterize the length of an optimal embedded index code. An optimal embedded index code for a given embedded index coding problem is shown to be obtainable from the columns of its side information matrix. Further, for a class of embedded index coding problems, called one-sided neighboring side information problems, the min-rank is derived and a transmission scheme which has length equal to this min-rank is presented.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125499582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Fast Compressive Large-Scale Matrix-Matrix Multiplication Using Product Codes 使用乘积码的快速压缩大规模矩阵-矩阵乘法
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9173951
Orhan Ocal, K. Ramchandran
Matrix-matrix multiplication and its derivatives are fundamental linear-algebraic primitives at the core of many modern optimization and machine learning algorithms. We design a new and novel framework for speeding up large-scale matrix-matrix multiplication when the output matrix is known to be sparse, as is true in many applications of interest. Our solution is based on a novel use of product codes which have been studied in the communications literature. In particular, when multiplying two matrices of sizes n × d and d n where the output matrix is (exactly) K-sparse with support× uniformly distributed, our algorithm requires max(O(dK), O(dn)) computations. We also extend our framework to handle the approximately-sparse setting where the output matrix has K-entries that are significantly larger than the rest. In this case, the computational complexity is max(O(dK log2(n)), O(dn log2(n))). We corroborate our findings with numerical simulations that validate our claims.
矩阵-矩阵乘法及其导数是基本的线性代数原语,是许多现代优化和机器学习算法的核心。当输出矩阵已知为稀疏时,我们设计了一个新的和新颖的框架来加速大规模矩阵-矩阵乘法,正如在许多感兴趣的应用中一样。我们的解决方案是基于在通信文献中研究过的产品代码的一种新用法。特别是,当两个大小为n × d和dn的矩阵相乘时,其中输出矩阵(恰好)为k -稀疏且supportx均匀分布时,我们的算法需要最大(O(dK), O(dn))次计算。我们还扩展了我们的框架来处理近似稀疏的设置,其中输出矩阵有k个条目,这些条目明显大于其他条目。在这种情况下,计算复杂度为max(O(dK log2(n)), O(dn log2(n)))。我们用数值模拟证实了我们的发现。
{"title":"Fast Compressive Large-Scale Matrix-Matrix Multiplication Using Product Codes","authors":"Orhan Ocal, K. Ramchandran","doi":"10.1109/ISIT44484.2020.9173951","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9173951","url":null,"abstract":"Matrix-matrix multiplication and its derivatives are fundamental linear-algebraic primitives at the core of many modern optimization and machine learning algorithms. We design a new and novel framework for speeding up large-scale matrix-matrix multiplication when the output matrix is known to be sparse, as is true in many applications of interest. Our solution is based on a novel use of product codes which have been studied in the communications literature. In particular, when multiplying two matrices of sizes n × d and d n where the output matrix is (exactly) K-sparse with support× uniformly distributed, our algorithm requires max(O(dK), O(dn)) computations. We also extend our framework to handle the approximately-sparse setting where the output matrix has K-entries that are significantly larger than the rest. In this case, the computational complexity is max(O(dK log2(n)), O(dn log2(n))). We corroborate our findings with numerical simulations that validate our claims.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126251234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Active Learning for Classification with Abstention 基于弃权的分类主动学习
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174242
S. Shekhar, M. Ghavamzadeh, T. Javidi
We consider the problem of binary classification with the caveat that the learner can abstain from declaring a label incurring a cost λ ∈ [0,1/2] in the process. This is referred to as the problem of binary classification with a fixed-cost of abstention. For this problem, we propose an active learning strategy that constructs a non-uniform partition of the input space and focuses sampling in the regions near the decision boundaries. Our proposed algorithm can work in all the commonly used active learning query models, namely membership-query, pool-based and stream-based. We obtain an upper bound on the excess risk of our proposed algorithm under standard smoothness and margin assumptions and demonstrate its minimax near-optimality by deriving a matching (modulo poly-logarithmic factors) lower bound. The achieved minimax rates are always faster than the corresponding rates in the passive setting, and furthermore the improvement increases with larger values of the smoothness and margin parameters.
我们考虑二元分类问题的前提是,学习器可以避免声明一个标签,在这个过程中会产生代价λ∈[0,1/2]。这被称为具有固定弃权成本的二元分类问题。针对这一问题,我们提出了一种主动学习策略,该策略构建了输入空间的非均匀划分,并将采样集中在决策边界附近的区域。我们提出的算法适用于所有常用的主动学习查询模型,即成员查询模型、基于池的模型和基于流的模型。在标准平滑和边际假设下,我们得到了该算法的超额风险的上界,并通过推导匹配的(模多对数因子)下界证明了其极小极大近最优性。在被动条件下,所获得的极大极小速率总是快于相应的速率,并且随着平滑度和裕度参数值的增大,所获得的极大极小速率也随之增大。
{"title":"Active Learning for Classification with Abstention","authors":"S. Shekhar, M. Ghavamzadeh, T. Javidi","doi":"10.1109/ISIT44484.2020.9174242","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174242","url":null,"abstract":"We consider the problem of binary classification with the caveat that the learner can abstain from declaring a label incurring a cost λ ∈ [0,1/2] in the process. This is referred to as the problem of binary classification with a fixed-cost of abstention. For this problem, we propose an active learning strategy that constructs a non-uniform partition of the input space and focuses sampling in the regions near the decision boundaries. Our proposed algorithm can work in all the commonly used active learning query models, namely membership-query, pool-based and stream-based. We obtain an upper bound on the excess risk of our proposed algorithm under standard smoothness and margin assumptions and demonstrate its minimax near-optimality by deriving a matching (modulo poly-logarithmic factors) lower bound. The achieved minimax rates are always faster than the corresponding rates in the passive setting, and furthermore the improvement increases with larger values of the smoothness and margin parameters.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122217759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Linear Models are Most Favorable among Generalized Linear Models 在广义线性模型中,线性模型是最有利的
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174124
Kuan-Yun Lee, T. Courtade
We establish a nonasymptotic lower bound on the L2 minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).
建立了一类广义线性模型的L2极大极小风险的非渐近下界。进一步证明了典型线性模型的极大极小风险与这个下界匹配到一个普遍常数。因此,在考虑的一类广义线性模型中,规范线性模型可以被认为是最有利的(就最小最大风险而言)。该证明使用了由Aras等人(2019)建立的log-凹先验的信息论贝叶斯cram r- rao界。
{"title":"Linear Models are Most Favorable among Generalized Linear Models","authors":"Kuan-Yun Lee, T. Courtade","doi":"10.1109/ISIT44484.2020.9174124","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174124","url":null,"abstract":"We establish a nonasymptotic lower bound on the L2 minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126879060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Product Lagrange Coded Computing 产品拉格朗日编码计算
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174440
Adarsh M. Subramaniam, A. Heidarzadeh, A. K. Pradhan, K. Narayanan
This work considers the distributed multivariate polynomial evaluation (DMPE) problem using a master-worker framework, which was originally considered by Yu et al., where Lagrange Coded Computing (LCC) was proposed as a coded computation scheme to provide resilience against stragglers for the DMPE problem. In this work, we propose a variant of the LCC scheme, termed Product Lagrange Coded Computing (PLCC), by combining ideas from classical product codes and LCC. The main advantage of PLCC is that they are more numerically stable than LCC; however, their resilience to stragglers is sub-optimal.
这项工作考虑了使用主工框架的分布式多元多项式评估(DMPE)问题,该框架最初由Yu等人考虑,其中拉格朗日编码计算(LCC)被提出作为一种编码计算方案,为DMPE问题提供针对离散者的弹性。在这项工作中,我们提出了LCC方案的一种变体,称为产品拉格朗日编码计算(PLCC),通过结合经典产品代码和LCC的思想。PLCC的主要优点是它们在数值上比LCC更稳定;然而,他们对掉队者的适应能力不是最优的。
{"title":"Product Lagrange Coded Computing","authors":"Adarsh M. Subramaniam, A. Heidarzadeh, A. K. Pradhan, K. Narayanan","doi":"10.1109/ISIT44484.2020.9174440","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174440","url":null,"abstract":"This work considers the distributed multivariate polynomial evaluation (DMPE) problem using a master-worker framework, which was originally considered by Yu et al., where Lagrange Coded Computing (LCC) was proposed as a coded computation scheme to provide resilience against stragglers for the DMPE problem. In this work, we propose a variant of the LCC scheme, termed Product Lagrange Coded Computing (PLCC), by combining ideas from classical product codes and LCC. The main advantage of PLCC is that they are more numerically stable than LCC; however, their resilience to stragglers is sub-optimal.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120973786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
On the Rényi Entropy of Log-Concave Sequences log -凹序列的r<s:1>熵
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174465
J. Melbourne, T. Tkocz
We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log2e of the usual Shannon entropy. With the additional assumption that the variable is monotone we obtain a sharp bound of loge.
我们建立了一个离散的模拟由于Bobkov和maddiman的r熵比较。对于整数上的对数凹变量,最小熵在通常香农熵的log2e以内。在附加假设变量是单调的情况下,我们得到了loge的一个锐界。
{"title":"On the Rényi Entropy of Log-Concave Sequences","authors":"J. Melbourne, T. Tkocz","doi":"10.1109/ISIT44484.2020.9174465","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174465","url":null,"abstract":"We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log2e of the usual Shannon entropy. With the additional assumption that the variable is monotone we obtain a sharp bound of loge.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"110 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116115999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
The Communication-Aware Clustered Federated Learning Problem 感知通信的聚类联邦学习问题
Pub Date : 2020-06-01 DOI: 10.1109/ISIT44484.2020.9174245
Nir Shlezinger, S. Rini, Yonina C. Eldar
Federated learning (FL) refers to the adaptation of a central model based on data sets available at multiple remote users. Two of the common challenges encountered in FL are the fact that training sets obtained by different users are commonly heterogeneous, i.e., arise from different sample distributions, and the need to communicate large amounts of data between the users and the central server over the typically expensive up-link channel. In this work we formulate the problem of FL in which different clusters of users observe labeled samples drawn from different distributions, while operating under constraints on the communication overhead. For such settings, we identify that the combination of statistical heterogeneity and communication constraints induces a tradeoff between the ability of the users of each cluster to learn a proper model and the accuracy in aggregating these models into a global inference rule. We propose an algorithm based on multi-source adaptation methods for such communication-aware clustered FL scenarios which allows to balance these performance measures, and demonstrate its ability to achieve improved inference over conventional federated averaging without inducing additional communication overhead.
联邦学习(FL)指的是基于多个远程用户可用的数据集对中心模型进行调整。FL中遇到的两个常见挑战是,不同用户获得的训练集通常是异构的,即来自不同的样本分布,并且需要在用户和中央服务器之间通过通常昂贵的上行链路通道进行大量数据通信。在这项工作中,我们制定了FL问题,其中不同的用户群观察从不同分布中抽取的标记样本,同时在通信开销的约束下运行。对于这样的设置,我们发现统计异质性和通信约束的组合导致每个集群的用户学习适当模型的能力和将这些模型聚合到全局推理规则中的准确性之间的权衡。我们提出了一种基于多源自适应方法的算法,用于这种通信感知的集群FL场景,该算法允许平衡这些性能度量,并证明其能够在不引起额外通信开销的情况下实现优于传统联邦平均的改进推理。
{"title":"The Communication-Aware Clustered Federated Learning Problem","authors":"Nir Shlezinger, S. Rini, Yonina C. Eldar","doi":"10.1109/ISIT44484.2020.9174245","DOIUrl":"https://doi.org/10.1109/ISIT44484.2020.9174245","url":null,"abstract":"Federated learning (FL) refers to the adaptation of a central model based on data sets available at multiple remote users. Two of the common challenges encountered in FL are the fact that training sets obtained by different users are commonly heterogeneous, i.e., arise from different sample distributions, and the need to communicate large amounts of data between the users and the central server over the typically expensive up-link channel. In this work we formulate the problem of FL in which different clusters of users observe labeled samples drawn from different distributions, while operating under constraints on the communication overhead. For such settings, we identify that the combination of statistical heterogeneity and communication constraints induces a tradeoff between the ability of the users of each cluster to learn a proper model and the accuracy in aggregating these models into a global inference rule. We propose an algorithm based on multi-source adaptation methods for such communication-aware clustered FL scenarios which allows to balance these performance measures, and demonstrate its ability to achieve improved inference over conventional federated averaging without inducing additional communication overhead.","PeriodicalId":159311,"journal":{"name":"2020 IEEE International Symposium on Information Theory (ISIT)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115275471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 24
期刊
2020 IEEE International Symposium on Information Theory (ISIT)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1