首页 > 最新文献

2020 Information Theory and Applications Workshop (ITA)最新文献

英文 中文
A Brain-Inspired Framework for Evolutionary Artificial General Intelligence 进化人工通用智能的大脑启发框架
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9245000
Mohammad Nadji-Tehrani, A. Eslami
From the medical field to agriculture, from energy to transportation, every industry is going through a revolution by embracing artificial intelligence (AI); nevertheless, AI is still in its infancy. Inspired by the evolution of the human brain, this paper demonstrates a novel method and framework to synthesize an artificial brain with cognitive abilities by taking advantage of the same process responsible for the growth of the biological brain called "neuroembryogenesis." This framework shares some of the key behavioral aspects of the biological brain such as spiking neurons, neuroplasticity, neuronal pruning, and excitatory and inhibitory interactions between neurons, together making it capable of learning and memorizing. One of the highlights of the proposed design is its potential to incrementally improve itself over generations based on system performance, using genetic algorithms. A proof of concept at the end of the paper demonstrates how a simplified implementation of the human visual cortex using the proposed framework is capable of character recognition. Our framework is open-source and the code is shared with the scientific community at www.feagi.org.
从医疗领域到农业,从能源到交通运输,每个行业都在经历一场人工智能(AI)的革命;然而,人工智能仍处于起步阶段。受人类大脑进化的启发,这篇论文展示了一种新的方法和框架,通过利用与生物大脑生长相同的过程,即“神经胚胎发生”,来合成具有认知能力的人工大脑。这个框架分享了生物大脑的一些关键行为方面,如尖峰神经元、神经可塑性、神经元修剪、神经元之间的兴奋性和抑制性相互作用,这些共同使它能够学习和记忆。所提出的设计的亮点之一是它有可能使用遗传算法根据系统性能逐步改进自身。本文最后的概念验证演示了如何使用所提出的框架简化人类视觉皮层的实现,从而能够进行字符识别。我们的框架是开源的,代码在www.feagi.org上与科学界共享。
{"title":"A Brain-Inspired Framework for Evolutionary Artificial General Intelligence","authors":"Mohammad Nadji-Tehrani, A. Eslami","doi":"10.1109/ITA50056.2020.9245000","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9245000","url":null,"abstract":"From the medical field to agriculture, from energy to transportation, every industry is going through a revolution by embracing artificial intelligence (AI); nevertheless, AI is still in its infancy. Inspired by the evolution of the human brain, this paper demonstrates a novel method and framework to synthesize an artificial brain with cognitive abilities by taking advantage of the same process responsible for the growth of the biological brain called \"neuroembryogenesis.\" This framework shares some of the key behavioral aspects of the biological brain such as spiking neurons, neuroplasticity, neuronal pruning, and excitatory and inhibitory interactions between neurons, together making it capable of learning and memorizing. One of the highlights of the proposed design is its potential to incrementally improve itself over generations based on system performance, using genetic algorithms. A proof of concept at the end of the paper demonstrates how a simplified implementation of the human visual cortex using the proposed framework is capable of character recognition. Our framework is open-source and the code is shared with the scientific community at www.feagi.org.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125499340","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Applications of Online Nonnegative Matrix Factorization to Image and Time-Series Data 在线非负矩阵分解在图像和时间序列数据中的应用
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9245004
Hanbaek Lyu, G. Menz, D. Needell, Christopher Strohmeier
Online nonnegative matrix factorization (ONMF) is a matrix factorization technique in the online setting where data are acquired in a streaming fashion and the matrix factors are updated each time. This enables factor analysis to be performed concurrently with the arrival of new data samples. In this article, we demonstrate how one can use online nonnegative matrix factorization algorithms to learn joint dictionary atoms from an ensemble of correlated data sets. We propose a temporal dictionary learning scheme for time-series data sets, based on ONMF algorithms. We demonstrate our dictionary learning technique in the application contexts of historical temperature data, video frames, and color images.
在线非负矩阵分解(ONMF)是一种在线环境下的矩阵分解技术,它以流方式获取数据,每次更新矩阵因子。这使得因子分析可以与新数据样本同时进行。在本文中,我们演示了如何使用在线非负矩阵分解算法从相关数据集的集合中学习联合字典原子。我们提出了一种基于ONMF算法的时间序列数据集时态字典学习方案。我们在历史温度数据、视频帧和彩色图像的应用环境中演示了我们的字典学习技术。
{"title":"Applications of Online Nonnegative Matrix Factorization to Image and Time-Series Data","authors":"Hanbaek Lyu, G. Menz, D. Needell, Christopher Strohmeier","doi":"10.1109/ITA50056.2020.9245004","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9245004","url":null,"abstract":"Online nonnegative matrix factorization (ONMF) is a matrix factorization technique in the online setting where data are acquired in a streaming fashion and the matrix factors are updated each time. This enables factor analysis to be performed concurrently with the arrival of new data samples. In this article, we demonstrate how one can use online nonnegative matrix factorization algorithms to learn joint dictionary atoms from an ensemble of correlated data sets. We propose a temporal dictionary learning scheme for time-series data sets, based on ONMF algorithms. We demonstrate our dictionary learning technique in the application contexts of historical temperature data, video frames, and color images.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120890361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Residual Based Sampling for Online Low Rank Approximation 基于残差的在线低秩逼近抽样
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9244974
Aditya Bhaskara, Silvio Lattanzi, Sergei Vassilvitskii, Morteza Zadimoghaddam
We propose online algorithms for Column Subset Selection (CSS) and Principal Component Analysis (PCA), two methods that are widely employed for data analysis, summarization, and visualization. Given a data matrix A that is revealed one column at a time, the online CSS problems asks to keep a small set of columns, S, that best approximates the space spanned by the columns of A. As each column arrives, the algorithm must irrevocably decide whether to add it to S, or to ignore it. In the online PCA problem, the goal is to output a projection of each column to a low dimensional subspace. In other words, the algorithm must provide an embedding for each column as it arrives, which cannot be changed as new columns arrive.While both of these problems have been studied in the online setting, only additive approximations were known prior to our work. The core of our approach is an adaptive sampling technique that gives a practical and efficient algorithm for both of these problems. We prove that by sampling columns using their "residual norm" (i.e. their norm orthogonal to directions sampled so far), we end up with a significantly better dependence between the number of columns sampled, and the desired error in the approximation.We further show how to combine our algorithm "in series" with prior algorithms. In particular, using the results of Boutsidis et al. [5] and Frieze et al. [15] that have additive guarantees, we show how to improve the bounds on the error of our algorithm.
我们提出了列子集选择(CSS)和主成分分析(PCA)的在线算法,这两种方法被广泛用于数据分析,汇总和可视化。给定一个每次显示一列的数据矩阵a,在线CSS问题要求保留一小组列S,这最接近a的列所跨越的空间。当每列到达时,算法必须不可撤销地决定是将其添加到S中,还是忽略它。在在线PCA问题中,目标是将每列的投影输出到低维子空间。换句话说,算法必须在每个列到达时为其提供嵌入,不能在新列到达时更改。虽然这两个问题都在在线环境中进行了研究,但在我们的工作之前,只有加法近似是已知的。我们的方法的核心是一种自适应采样技术,它为这两个问题提供了一个实用而有效的算法。我们证明,通过使用它们的“残差范数”(即它们的范数与迄今为止采样的方向正交)对列进行采样,我们最终得到了采样列数与近似中期望的误差之间明显更好的相关性。我们进一步展示了如何将我们的算法与先前的算法“串联”起来。特别地,利用具有可加性保证的boutis等人[5]和Frieze等人[15]的结果,我们展示了如何改进我们算法的误差界。
{"title":"Residual Based Sampling for Online Low Rank Approximation","authors":"Aditya Bhaskara, Silvio Lattanzi, Sergei Vassilvitskii, Morteza Zadimoghaddam","doi":"10.1109/ITA50056.2020.9244974","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9244974","url":null,"abstract":"We propose online algorithms for Column Subset Selection (CSS) and Principal Component Analysis (PCA), two methods that are widely employed for data analysis, summarization, and visualization. Given a data matrix A that is revealed one column at a time, the online CSS problems asks to keep a small set of columns, S, that best approximates the space spanned by the columns of A. As each column arrives, the algorithm must irrevocably decide whether to add it to S, or to ignore it. In the online PCA problem, the goal is to output a projection of each column to a low dimensional subspace. In other words, the algorithm must provide an embedding for each column as it arrives, which cannot be changed as new columns arrive.While both of these problems have been studied in the online setting, only additive approximations were known prior to our work. The core of our approach is an adaptive sampling technique that gives a practical and efficient algorithm for both of these problems. We prove that by sampling columns using their \"residual norm\" (i.e. their norm orthogonal to directions sampled so far), we end up with a significantly better dependence between the number of columns sampled, and the desired error in the approximation.We further show how to combine our algorithm \"in series\" with prior algorithms. In particular, using the results of Boutsidis et al. [5] and Frieze et al. [15] that have additive guarantees, we show how to improve the bounds on the error of our algorithm.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116212465","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An Interference-Resilient Relay Beamforming Scheme Inspired by Back-Propagation Algorithm 一种基于反向传播算法的抗干扰中继波束形成方案
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9245001
Rui Wang, Yinglin Jiang
A relay node can be used to improve the distance and service quality of a communication link, but not when it is being interfered. In this paper, we consider a relay network consisting of one source, one destination, and multiple relay nodes, and draw analogy between the relay network and a three-layer artificial neural network (ANN). Inspired by the classic back-propagation (BP) algorithm for the ANN, we develop an interference-resilient algorithm that can optimize the beamforming-and-forwarding weights of the relay nodes so that the interferences will be canceled at the destination. The proposed algorithm requires no channel state information (CSI), no data exchanges between the relay nodes; it requires that the source transmit training sequences in the forward channel (source-to-relays) and the destination transmit error sequences in the backward channel (destination-to-relays). The simulation results verify the effectiveness of the proposed scheme in the interference environment.
中继节点可用于提高通信链路的距离和服务质量,但不能用于受到干扰的情况。本文考虑一个由一个源、一个目的和多个中继节点组成的中继网络,并将该中继网络与三层人工神经网络(ANN)进行类比。受经典的神经网络反向传播(BP)算法的启发,我们开发了一种抗干扰算法,该算法可以优化中继节点的波束形成和转发权重,从而在目的地消除干扰。该算法不需要信道状态信息(CSI),中继节点之间不需要数据交换;它要求源在前向信道(源到中继)中发送训练序列,目的在后向信道(目的到中继)中发送错误序列。仿真结果验证了该方案在干扰环境下的有效性。
{"title":"An Interference-Resilient Relay Beamforming Scheme Inspired by Back-Propagation Algorithm","authors":"Rui Wang, Yinglin Jiang","doi":"10.1109/ITA50056.2020.9245001","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9245001","url":null,"abstract":"A relay node can be used to improve the distance and service quality of a communication link, but not when it is being interfered. In this paper, we consider a relay network consisting of one source, one destination, and multiple relay nodes, and draw analogy between the relay network and a three-layer artificial neural network (ANN). Inspired by the classic back-propagation (BP) algorithm for the ANN, we develop an interference-resilient algorithm that can optimize the beamforming-and-forwarding weights of the relay nodes so that the interferences will be canceled at the destination. The proposed algorithm requires no channel state information (CSI), no data exchanges between the relay nodes; it requires that the source transmit training sequences in the forward channel (source-to-relays) and the destination transmit error sequences in the backward channel (destination-to-relays). The simulation results verify the effectiveness of the proposed scheme in the interference environment.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116746577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
On-the-fly Uplink Training and Pilot Code Design for Massive MIMO Cellular Networks 大规模MIMO蜂窝网络的动态上行训练与导频码设计
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9244985
Chenwei Wang, Zekun Zhang, H. Papadopoulos
We investigate non-orthogonal uplink pilot designs for improving the area spectral efficiency in the downlink of TDD reciprocity-based massive MIMO cellular networks. In particular, we develop a class of pilot designs that are locally orthogonal within each cell, while maintaining low inner-product properties between codes in different cells. Using channel estimations provided by observations on these codes, each cell independently serves its locally active users with MU-MIMO transmission that is also designed to mitigate interference to a subset of "strongly interfered" out-of-cell users. As our analysis shows, such cellular operation based on the proposed codes yields improvements (with respect to conventional operation) in user-rate CDFs, cell-throughput and cell-edge throughput performance.
为了提高基于TDD互向的大规模MIMO蜂窝网络下行链路的区域频谱效率,我们研究了非正交上行导频设计。特别是,我们开发了一类在每个单元内局部正交的试验设计,同时在不同单元中的代码之间保持低内积性质。利用对这些代码的观察提供的信道估计,每个小区独立地为其本地活跃用户提供MU-MIMO传输,该传输也旨在减轻对“强干扰”小区外用户子集的干扰。正如我们的分析所显示的那样,基于所提议的代码的这种蜂窝操作(相对于传统操作)在用户速率CDFs、蜂窝吞吐量和蜂窝边缘吞吐量性能方面产生了改进。
{"title":"On-the-fly Uplink Training and Pilot Code Design for Massive MIMO Cellular Networks","authors":"Chenwei Wang, Zekun Zhang, H. Papadopoulos","doi":"10.1109/ITA50056.2020.9244985","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9244985","url":null,"abstract":"We investigate non-orthogonal uplink pilot designs for improving the area spectral efficiency in the downlink of TDD reciprocity-based massive MIMO cellular networks. In particular, we develop a class of pilot designs that are locally orthogonal within each cell, while maintaining low inner-product properties between codes in different cells. Using channel estimations provided by observations on these codes, each cell independently serves its locally active users with MU-MIMO transmission that is also designed to mitigate interference to a subset of \"strongly interfered\" out-of-cell users. As our analysis shows, such cellular operation based on the proposed codes yields improvements (with respect to conventional operation) in user-rate CDFs, cell-throughput and cell-edge throughput performance.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131157125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Limits of Detecting Text Generated by Large-Scale Language Models 大规模语言模型生成文本检测的局限性
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9245012
L. Varshney, N. Keskar, R. Socher
Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated. We show that error exponents for particular language models are bounded in terms of their perplexity, a standard measure of language generation performance. Under the assumption that human language is stationary and ergodic, the formulation is ex-tended from considering specific language models to considering maximum likelihood language models, among the class of k-order Markov approximations; error probabilities are characterized. Some discussion of incorporating semantic side information is also given.
一些人认为可以生成长而连贯的文本片段的大规模语言模型是危险的,因为它们可能被用于错误的信息宣传活动。在这里,我们将大规模语言模型输出检测作为假设检验问题来将文本分类为真实文本或生成文本。我们表明,特定语言模型的错误指数在其困惑度方面是有界的,这是语言生成性能的标准度量。在假设人类语言是平稳和遍历的前提下,将该公式从考虑特定的语言模型扩展到考虑k阶马尔可夫近似类中的极大似然语言模型;对误差概率进行了表征。本文还对语义侧信息的融合进行了讨论。
{"title":"Limits of Detecting Text Generated by Large-Scale Language Models","authors":"L. Varshney, N. Keskar, R. Socher","doi":"10.1109/ITA50056.2020.9245012","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9245012","url":null,"abstract":"Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated. We show that error exponents for particular language models are bounded in terms of their perplexity, a standard measure of language generation performance. Under the assumption that human language is stationary and ergodic, the formulation is ex-tended from considering specific language models to considering maximum likelihood language models, among the class of k-order Markov approximations; error probabilities are characterized. Some discussion of incorporating semantic side information is also given.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133092015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 13
Non-Negative Matrix Factorization via Low-Rank Stochastic Manifold Optimization 基于低秩随机流形优化的非负矩阵分解
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9244937
Ahmed Douik, B. Hassibi
Several real-world applications, notably in non-negative matrix factorization, graph-based clustering, and machine learning, require solving a convex optimization problem over the set of stochastic and doubly stochastic matrices. A common feature of these problems is that the optimal solution is generally a low-rank matrix. This paper suggests reformulating the problem by taking advantage of the low-rank factorization X = UVT and develops a Riemannian optimization framework for solving optimization problems on the set of low-rank stochastic and doubly stochastic matrices. In particular, this paper introduces and studies the geometry of the low-rank stochastic multinomial and the doubly stochastic manifold in order to derive first-order optimization algorithms. Being carefully designed and of lower dimension than the original problem, the proposed Riemannian optimization framework presents a clear complexity advantage. The claim is attested through numerical experiments on real-world and synthetic data for Non-negative Matrix Factorization (NFM) applications. The proposed algorithm is shown to outperform, in terms of running time, state-of-the-art methods for NFM.
一些现实世界的应用,特别是在非负矩阵分解、基于图的聚类和机器学习中,需要解决随机和双随机矩阵集合上的凸优化问题。这些问题的一个共同特征是最优解通常是一个低秩矩阵。本文利用低秩分解X = UVT对问题进行了重新表述,并建立了求解低秩随机和双随机矩阵集合上的优化问题的黎曼优化框架。本文特别介绍和研究了低秩随机多项式和双随机流形的几何性质,从而推导出一阶优化算法。所提出的黎曼优化框架经过精心设计,比原问题的维数更低,具有明显的复杂性优势。通过非负矩阵分解(NFM)应用的真实世界和合成数据的数值实验证明了这一说法。就运行时间而言,所提出的算法优于NFM的最先进方法。
{"title":"Non-Negative Matrix Factorization via Low-Rank Stochastic Manifold Optimization","authors":"Ahmed Douik, B. Hassibi","doi":"10.1109/ITA50056.2020.9244937","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9244937","url":null,"abstract":"Several real-world applications, notably in non-negative matrix factorization, graph-based clustering, and machine learning, require solving a convex optimization problem over the set of stochastic and doubly stochastic matrices. A common feature of these problems is that the optimal solution is generally a low-rank matrix. This paper suggests reformulating the problem by taking advantage of the low-rank factorization X = UVT and develops a Riemannian optimization framework for solving optimization problems on the set of low-rank stochastic and doubly stochastic matrices. In particular, this paper introduces and studies the geometry of the low-rank stochastic multinomial and the doubly stochastic manifold in order to derive first-order optimization algorithms. Being carefully designed and of lower dimension than the original problem, the proposed Riemannian optimization framework presents a clear complexity advantage. The claim is attested through numerical experiments on real-world and synthetic data for Non-negative Matrix Factorization (NFM) applications. The proposed algorithm is shown to outperform, in terms of running time, state-of-the-art methods for NFM.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128585975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Efficient Matrix Multiplication: The Sparse Power-of-2 Factorization 高效矩阵乘法:稀疏的2次方分解
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9244952
R. Müller, Bernhard Gäde, Ali Bereyhi
We present an algorithm to reduce the computational effort for the multiplication of a given matrix with an unknown column vector. The algorithm decomposes the given matrix into a product of matrices whose entries are either zero or integer powers of two utilizing the principles of sparse recovery. While classical low resolution quantization achieves an accuracy of 6 dB per bit, our method can achieve many times more than that for large matrices. Numerical and analytical evidence suggests that the improvement actually grows unboundedly with matrix size. Due to sparsity, the algorithm even allows for quantization levels below 1 bit per matrix entry while achieving highly accurate approximations for large matrices. Applications include, but are not limited to, neural networks, as well as fully digital beam-forming for massive MIMO and millimeter wave applications.
我们提出了一种算法,以减少计算的努力为一个给定的矩阵与一个未知的列向量的乘法。该算法利用稀疏恢复原理,将给定矩阵分解为元素为零或2的整数次幂的矩阵的乘积。虽然经典的低分辨率量化可以达到每比特6 dB的精度,但我们的方法可以实现比大型矩阵高许多倍的精度。数值和分析证据表明,这种改进实际上不受矩阵大小的限制。由于稀疏性,该算法甚至允许每个矩阵条目低于1位的量化水平,同时对大型矩阵实现高度精确的近似值。应用包括但不限于神经网络,以及用于大规模MIMO和毫米波应用的全数字波束形成。
{"title":"Efficient Matrix Multiplication: The Sparse Power-of-2 Factorization","authors":"R. Müller, Bernhard Gäde, Ali Bereyhi","doi":"10.1109/ITA50056.2020.9244952","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9244952","url":null,"abstract":"We present an algorithm to reduce the computational effort for the multiplication of a given matrix with an unknown column vector. The algorithm decomposes the given matrix into a product of matrices whose entries are either zero or integer powers of two utilizing the principles of sparse recovery. While classical low resolution quantization achieves an accuracy of 6 dB per bit, our method can achieve many times more than that for large matrices. Numerical and analytical evidence suggests that the improvement actually grows unboundedly with matrix size. Due to sparsity, the algorithm even allows for quantization levels below 1 bit per matrix entry while achieving highly accurate approximations for large matrices. Applications include, but are not limited to, neural networks, as well as fully digital beam-forming for massive MIMO and millimeter wave applications.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129094347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
On Marton's Achievable Region: Local Tensorization for Product Channels with a Binary Component 关于马顿可达区域:具有二元分量的产品通道的局部张化
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9244997
Chandra Nair
We show that Marton's achievable rate region for product broadcast channels with one binary component satisfies a property called local tensorization. If a corresponding global tensorization property held for the same setting, then this would be equivalent to showing the optimality of Marton's achievable region for any two receiver broadcast channel with binary inputs.
我们证明了具有一个二元分量的产品广播信道的马尔顿可实现速率区域满足一个称为局部张化的性质。如果一个对应的全局张量化属性在相同的设置下保持不变,那么这将相当于显示任何两个具有二进制输入的接收广播信道的马尔顿可实现区域的最优性。
{"title":"On Marton's Achievable Region: Local Tensorization for Product Channels with a Binary Component","authors":"Chandra Nair","doi":"10.1109/ITA50056.2020.9244997","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9244997","url":null,"abstract":"We show that Marton's achievable rate region for product broadcast channels with one binary component satisfies a property called local tensorization. If a corresponding global tensorization property held for the same setting, then this would be equivalent to showing the optimality of Marton's achievable region for any two receiver broadcast channel with binary inputs.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114842802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Identifying unpredictable test examples with worst-case guarantees 识别具有最坏情况保证的不可预测的测试示例
Pub Date : 2020-02-02 DOI: 10.1109/ITA50056.2020.9244996
S. Goldwasser, A. Kalai, Y. Kalai, Omar Montasser
Often times, whether it be for adversarial or natural reasons, the distributions of test and training data differ. We give an algorithm that, given sets of training and test examples, identifies regions of test examples that cannot be predicted with low error. These regions are classified as ƒ or equivalently omitted from classification. Assuming only that labels are consistent with a family of classifiers of low VC dimension, the algorithm is shown to make few misclassification errors and few errors of omission in both adversarial and covariate-shift settings. Previous models of learning with different training and test distributions required assumptions connecting the two.
通常情况下,无论是出于对抗还是自然原因,测试和训练数据的分布都是不同的。我们给出了一种算法,在给定训练和测试样例集的情况下,以低误差识别测试样例中无法预测的区域。这些区域被分类为f或从分类中省略。仅假设标签与一组低VC维的分类器一致,该算法在对抗和协变量移位设置下都很少出现误分类错误和遗漏错误。以前使用不同训练和测试分布的学习模型需要将两者连接起来的假设。
{"title":"Identifying unpredictable test examples with worst-case guarantees","authors":"S. Goldwasser, A. Kalai, Y. Kalai, Omar Montasser","doi":"10.1109/ITA50056.2020.9244996","DOIUrl":"https://doi.org/10.1109/ITA50056.2020.9244996","url":null,"abstract":"Often times, whether it be for adversarial or natural reasons, the distributions of test and training data differ. We give an algorithm that, given sets of training and test examples, identifies regions of test examples that cannot be predicted with low error. These regions are classified as ƒ or equivalently omitted from classification. Assuming only that labels are consistent with a family of classifiers of low VC dimension, the algorithm is shown to make few misclassification errors and few errors of omission in both adversarial and covariate-shift settings. Previous models of learning with different training and test distributions required assumptions connecting the two.","PeriodicalId":137257,"journal":{"name":"2020 Information Theory and Applications Workshop (ITA)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121030559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
2020 Information Theory and Applications Workshop (ITA)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1