Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404730
P. Grover
Does approaching fundamental limits on rates of information acquisitionor transmission fundamentally require increased power consumption in the processing circuitry? Our recent work shows that this is the case for channel coding for some simple circuit and channel models. In this paper, we first develop parallel results for source coding. Reinterpreting existing results on complexity of lossless source coding, we first observe that the sum power consumed in computational nodes in the circuitry of the encoder and the decoder diverges to infinity as the target error probability approaches zero and the coding rate approaches the source entropy. Next, focusing on on-chip wires, we show that the power consumed in circuit wiring also diverges to infinity as the error probability approaches zero. For the closely related problem of recovering a sparse signal, we first derive a fundamental bound on the required number of “finite-capacity” (e.g. quantized or noisy) measurements. By extending our bounds on wiring complexity and power consumption to sparse-signal recovery, we observe that there is a tradeoff between measurement power and power required to compute the recovered signal.
{"title":"Fundamental limits on power consumption for lossless signal reconstruction","authors":"P. Grover","doi":"10.1109/ITW.2012.6404730","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404730","url":null,"abstract":"Does approaching fundamental limits on rates of information acquisitionor transmission fundamentally require increased power consumption in the processing circuitry? Our recent work shows that this is the case for channel coding for some simple circuit and channel models. In this paper, we first develop parallel results for source coding. Reinterpreting existing results on complexity of lossless source coding, we first observe that the sum power consumed in computational nodes in the circuitry of the encoder and the decoder diverges to infinity as the target error probability approaches zero and the coding rate approaches the source entropy. Next, focusing on on-chip wires, we show that the power consumed in circuit wiring also diverges to infinity as the error probability approaches zero. For the closely related problem of recovering a sparse signal, we first derive a fundamental bound on the required number of “finite-capacity” (e.g. quantized or noisy) measurements. By extending our bounds on wiring complexity and power consumption to sparse-signal recovery, we observe that there is a tradeoff between measurement power and power required to compute the recovered signal.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130089401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404714
A. Khina, Or Ordentlich, U. Erez, Y. Kochman, G. Wornell
This work considers practical implementation of the decode-and-forward relaying protocol for the full-duplex Gaussian relay channel. Unlike previous works which developed coding techniques tailored to this protocol, it is shown that standard codes which are good for the Gaussian scalar channel of fixed signal-to-noise ratio suffice to approach the theoretical performance promised by this protocol. The proposed technique employs only linear operations and successive interference cancelation in conjunction with fixed signal-to-noise ratio base codes, and the achievable rate is solely dictated by the performance of these base codes. The same approach and results carry over to the multiple-antenna case as well.
{"title":"Decode-and-forward for the Gaussian relay channel via standard AWGN coding and decoding","authors":"A. Khina, Or Ordentlich, U. Erez, Y. Kochman, G. Wornell","doi":"10.1109/ITW.2012.6404714","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404714","url":null,"abstract":"This work considers practical implementation of the decode-and-forward relaying protocol for the full-duplex Gaussian relay channel. Unlike previous works which developed coding techniques tailored to this protocol, it is shown that standard codes which are good for the Gaussian scalar channel of fixed signal-to-noise ratio suffice to approach the theoretical performance promised by this protocol. The proposed technique employs only linear operations and successive interference cancelation in conjunction with fixed signal-to-noise ratio base codes, and the achievable rate is solely dictated by the performance of these base codes. The same approach and results carry over to the multiple-antenna case as well.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121399206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404709
J. Birget, D. Lun, Anthony Wirth, Dawei Hong
We take a theoretical approach to the problem of identification, or “reverse engineering”, of gene regulatory networks. Through a mathematical model of a gene regulatory network, we examine fundamental questions on the limits and achievability of network identification. We apply simplifying assumptions to construct an acyclic binary model, and we assume that the identification strategy is restricted to perturbing the network by gene expression assignments, followed by expression profile measurements at steady-state. Further, we assume the presence of side information, which we call sensitivity, that is likely to be present in actual gene networks. We show that with sensitivity side information and realistic topology assumptions we can identify the topology of acyclic binary networks using O(n) assignments and measurements, n being the number of genes in the network. Our work establishes a theoretical framework for examining an important technological problem where a number of significant questions remain open.
{"title":"A theoretical approach to gene network identification","authors":"J. Birget, D. Lun, Anthony Wirth, Dawei Hong","doi":"10.1109/ITW.2012.6404709","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404709","url":null,"abstract":"We take a theoretical approach to the problem of identification, or “reverse engineering”, of gene regulatory networks. Through a mathematical model of a gene regulatory network, we examine fundamental questions on the limits and achievability of network identification. We apply simplifying assumptions to construct an acyclic binary model, and we assume that the identification strategy is restricted to perturbing the network by gene expression assignments, followed by expression profile measurements at steady-state. Further, we assume the presence of side information, which we call sensitivity, that is likely to be present in actual gene networks. We show that with sensitivity side information and realistic topology assumptions we can identify the topology of acyclic binary networks using O(n) assignments and measurements, n being the number of genes in the network. Our work establishes a theoretical framework for examining an important technological problem where a number of significant questions remain open.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129521746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404711
R. Gohary, H. Yanikomeroglu
This paper considers the ergodic non-coherent capacity of a multiple-input multiple-output frequency-flat block Rayleigh fading relay channel. It is shown that for this channel restricting the input distribution to be isotropic on a compact Grassmann manifold maximizes an upper bound on the cut-set bound at high signal-to-noise ratios (SNRs). Furthermore, Grassmannian signalling at the source achieves this bound within an SNR-independent gap. For moderate-to-high SNRs, a Grassmannian decode-and-forward (DF) relaying scheme is devised, and the optimal signalling dimensionality that minimizes the gap to the upper bound is obtained.
{"title":"Grassmannian signalling achieves the ergodic high SNR capacity of the non-coherent MIMO relay channel within an SNR-independent gap","authors":"R. Gohary, H. Yanikomeroglu","doi":"10.1109/ITW.2012.6404711","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404711","url":null,"abstract":"This paper considers the ergodic non-coherent capacity of a multiple-input multiple-output frequency-flat block Rayleigh fading relay channel. It is shown that for this channel restricting the input distribution to be isotropic on a compact Grassmann manifold maximizes an upper bound on the cut-set bound at high signal-to-noise ratios (SNRs). Furthermore, Grassmannian signalling at the source achieves this bound within an SNR-independent gap. For moderate-to-high SNRs, a Grassmannian decode-and-forward (DF) relaying scheme is devised, and the optimal signalling dimensionality that minimizes the gap to the upper bound is obtained.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130758774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404658
David G. M. Mitchell, R. Smarandache, D. Costello
Quasi-cyclic (QC) low-density parity-check (LDPC) codes are of great interest to code designers because of their implementation advantages and algebraic properties that facilitate their analysis. In this paper, we present some new results on QC-LDPC codes that are constructed using a two-step lifting procedure based on a protograph, and, by implementing this method instead of the usual one-step procedure, we are able to show improved minimum distance and girth properties. We also present two design rules to construct QC-LDPC codes: one uses only circulant permutation matrices at the first (pre-lifting) stage and the other uses a selection of non-commuting permutation matrices. For both techniques, we obtain a demonstrable increase in the minimum distance compared to a one-step circulant-based lifting. The expected performance improvement is verified by simulation results.
{"title":"Constructing good QC-LDPC codes by pre-lifting protographs","authors":"David G. M. Mitchell, R. Smarandache, D. Costello","doi":"10.1109/ITW.2012.6404658","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404658","url":null,"abstract":"Quasi-cyclic (QC) low-density parity-check (LDPC) codes are of great interest to code designers because of their implementation advantages and algebraic properties that facilitate their analysis. In this paper, we present some new results on QC-LDPC codes that are constructed using a two-step lifting procedure based on a protograph, and, by implementing this method instead of the usual one-step procedure, we are able to show improved minimum distance and girth properties. We also present two design rules to construct QC-LDPC codes: one uses only circulant permutation matrices at the first (pre-lifting) stage and the other uses a selection of non-commuting permutation matrices. For both techniques, we obtain a demonstrable increase in the minimum distance compared to a one-step circulant-based lifting. The expected performance improvement is verified by simulation results.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124324430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404736
Hiroki Kanazawa, K. Yamanishi
Kleinberg has proposed an algorithm for detecting bursts from a data sequence, which has turned out to be effective in the scenario of data mining, such as topic detection, change-detection. In this paper we extend Kleinberg's algorithm in an information-theoretic fashion to obtain a new class of algorithms and apply it into learning of piecewise stationary memoryless sources (PSMSs). The keys of the proposed algorithm are; 1) the parameter space is discretized so that discretization scale depends on the Fisher information, and 2) the optimal path over the discretized parameter space is efficiently computed using the dynamic programming method so that the sum of the data and parameter description lengths is minimized on the basis of the MDL principle. We prove that an upper bound on the total code-length for the proposed algorithm asymptotically matches Merhav's lower bound.
{"title":"An MDL-based change-detection algorithm with its applications to learning piecewise stationary memoryless sources","authors":"Hiroki Kanazawa, K. Yamanishi","doi":"10.1109/ITW.2012.6404736","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404736","url":null,"abstract":"Kleinberg has proposed an algorithm for detecting bursts from a data sequence, which has turned out to be effective in the scenario of data mining, such as topic detection, change-detection. In this paper we extend Kleinberg's algorithm in an information-theoretic fashion to obtain a new class of algorithms and apply it into learning of piecewise stationary memoryless sources (PSMSs). The keys of the proposed algorithm are; 1) the parameter space is discretized so that discretization scale depends on the Fisher information, and 2) the optimal path over the discretized parameter space is efficiently computed using the dynamic programming method so that the sum of the data and parameter description lengths is minimized on the basis of the MDL principle. We prove that an upper bound on the total code-length for the proposed algorithm asymptotically matches Merhav's lower bound.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"42 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128772462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404652
Shun Watanabe, Y. Oohama
In this paper, we consider a situation such that legitimate parties, Alice and Bob, share an identical source to generate a secret key, and an eavesdropper, Eve, can access a correlated data that is stored in a storage with bounded size. Then, Alice and Bob want to extract a secret as long as possible. We show a privacy amplification theorem for this problem, i.e., we clarify the rate of key generation for given rate of Eve's storage. The problem can be regarded as a dual randomness generation problem of the Wyner-Ahlswede-Körner type source coding system, and the techniques used in the proof are exchanged, i.e., the so-called Markov lemma is used in the converse part, and the so-called image size characterization is used in the direct part.
{"title":"Privacy amplification theorem for bounded storage eavesdropper","authors":"Shun Watanabe, Y. Oohama","doi":"10.1109/ITW.2012.6404652","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404652","url":null,"abstract":"In this paper, we consider a situation such that legitimate parties, Alice and Bob, share an identical source to generate a secret key, and an eavesdropper, Eve, can access a correlated data that is stored in a storage with bounded size. Then, Alice and Bob want to extract a secret as long as possible. We show a privacy amplification theorem for this problem, i.e., we clarify the rate of key generation for given rate of Eve's storage. The problem can be regarded as a dual randomness generation problem of the Wyner-Ahlswede-Körner type source coding system, and the techniques used in the proof are exchanged, i.e., the so-called Markov lemma is used in the converse part, and the so-called image size characterization is used in the direct part.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128539920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404722
P. Olmos, F. Pérez-Cruz, Luis Salamanca, J. J. Murillo-Fuentes
Spatially-coupled (SC) LDPC codes are constructed from a set of L regular sparse codes of length M. In the asymptotic limit of these parameters, SC codes present an excellent decoding threshold under belief propagation (BP) decoding, close to the maximum a posteriori (MAP) threshold of the underlying regular code. In the finite-length regime, we need both dimensions, L and M, to be sufficiently large, yielding a very large code length and decoding latency. In this paper, and for the erasure channel, we show that the finite-length performance of SC codes is improved if we consider the tree-structured expectation propagation (TEP) algorithm in the decoding stage. When applied to the decoding of SC LDPC codes, it allows using shorter codes to achieve similar error rates. We also propose a window-sliding scheme for the TEP decoder to reduce the decoding latency.
{"title":"Finite-length performance of spatially-coupled LDPC codes under TEP decoding","authors":"P. Olmos, F. Pérez-Cruz, Luis Salamanca, J. J. Murillo-Fuentes","doi":"10.1109/ITW.2012.6404722","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404722","url":null,"abstract":"Spatially-coupled (SC) LDPC codes are constructed from a set of L regular sparse codes of length M. In the asymptotic limit of these parameters, SC codes present an excellent decoding threshold under belief propagation (BP) decoding, close to the maximum a posteriori (MAP) threshold of the underlying regular code. In the finite-length regime, we need both dimensions, L and M, to be sufficiently large, yielding a very large code length and decoding latency. In this paper, and for the erasure channel, we show that the finite-length performance of SC codes is improved if we consider the tree-structured expectation propagation (TEP) algorithm in the decoding stage. When applied to the decoding of SC LDPC codes, it allows using shorter codes to achieve similar error rates. We also propose a window-sliding scheme for the TEP decoder to reduce the decoding latency.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116924412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404765
A. Amini, U. Kamilov, M. Unser
Motivated by the analog nature of real-world signals, we investigate continuous-time random processes. For this purpose, we consider the stochastic processes that can be whitened by linear transformations and we show that the distribution of their samples is necessarily infinitely divisible. As a consequence, such a modeling rules out the Bernoulli-Gaussian distribution since we are able to show in this paper that it is not infinitely divisible. In other words, while the Bernoulli-Gaussian distribution is among the most studied priors for modeling sparse signals, it cannot be associated with any continuous-time stochastic process. Instead, we propose to adapt the priors that correspond to the increments of compound Poisson processes, which are both sparse and infinitely divisible.
{"title":"The analog formulation of sparsity implies infinite divisibility and rules out Bernoulli-Gaussian priors","authors":"A. Amini, U. Kamilov, M. Unser","doi":"10.1109/ITW.2012.6404765","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404765","url":null,"abstract":"Motivated by the analog nature of real-world signals, we investigate continuous-time random processes. For this purpose, we consider the stochastic processes that can be whitened by linear transformations and we show that the distribution of their samples is necessarily infinitely divisible. As a consequence, such a modeling rules out the Bernoulli-Gaussian distribution since we are able to show in this paper that it is not infinitely divisible. In other words, while the Bernoulli-Gaussian distribution is among the most studied priors for modeling sparse signals, it cannot be associated with any continuous-time stochastic process. Instead, we propose to adapt the priors that correspond to the increments of compound Poisson processes, which are both sparse and infinitely divisible.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"148 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130658723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/ITW.2012.6404772
Kaya Tutuncuoglu, A. Yener
This work considers a Gaussian multiple access channel with two energy harvesting transmitters with lossy energy storage. The power allocation policy maximizing the average weighted sum rate given the energy harvesting profiles is found. In particular, it is shown that the optimal policy has a double-threshold structure on each of the transmit powers, while the two transmit powers interact through the multivariate achievable rate function which determines the thresholds. For the special case of sum rate maximization in a Gaussian MAC channel, it is shown that the thresholds apply to the sum power, and the optimal policy consists of three thresholds, rather than four, which enables the user with a more efficient battery to be given priority in energy storage.
{"title":"The energy harvesting multiple access channel with energy storage losses","authors":"Kaya Tutuncuoglu, A. Yener","doi":"10.1109/ITW.2012.6404772","DOIUrl":"https://doi.org/10.1109/ITW.2012.6404772","url":null,"abstract":"This work considers a Gaussian multiple access channel with two energy harvesting transmitters with lossy energy storage. The power allocation policy maximizing the average weighted sum rate given the energy harvesting profiles is found. In particular, it is shown that the optimal policy has a double-threshold structure on each of the transmit powers, while the two transmit powers interact through the multivariate achievable rate function which determines the thresholds. For the special case of sum rate maximization in a Gaussian MAC channel, it is shown that the thresholds apply to the sum power, and the optimal policy consists of three thresholds, rather than four, which enables the user with a more efficient battery to be given priority in energy storage.","PeriodicalId":325771,"journal":{"name":"2012 IEEE Information Theory Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125667365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：