Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541444
M. Kohler, A. Krzyżak
Regression estimation often suffers from the curse of dimensionality. In the present paper we circumvent this problem by introducing a class of models called hierarchical interaction models where the values of a function m : ℝd → ℝ are computed in a feed-forward manner in several layers, where in each layer a function of at most d* inputs produced by the previous layer is computed. We introduce regression estimates based on neural networks with two hidden layers and apply them to estimation of regression functions from a class of hierarchical interaction models. Under smoothness condition imposed on all functions occurring in the model we show that the rate of convergence of these estimates depends on d*, which is typically much smaller than d.
{"title":"The rates of convergence of neural network estimates of hierarchical interaction regression models","authors":"M. Kohler, A. Krzyżak","doi":"10.1109/ISIT.2016.7541444","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541444","url":null,"abstract":"Regression estimation often suffers from the curse of dimensionality. In the present paper we circumvent this problem by introducing a class of models called hierarchical interaction models where the values of a function m : ℝd → ℝ are computed in a feed-forward manner in several layers, where in each layer a function of at most d* inputs produced by the previous layer is computed. We introduce regression estimates based on neural networks with two hidden layers and apply them to estimation of regression functions from a class of hierarchical interaction models. Under smoothness condition imposed on all functions occurring in the model we show that the rate of convergence of these estimates depends on d*, which is typically much smaller than d.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116814538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541289
Y. M. Chee, Johan Chrisnata, H. M. Kiah, S. Ling, T. T. Nguyen, Van Khu Vu
For certain families of substrings F, we provide a closed formula for the maximum size of a q-ary F-avoiding code with a given composition. In addition, we provide numerical procedures to determine the asymptotic information rate for F-avoiding codes with certain composition ratios. Using our procedures, we recover known results and compute the information rates for certain classes of F-avoiding constant-composition codes for 2 ≤ q ≤ 8. For these values of q, we find composition ratios such that the rates of F-avoiding codes with constant composition achieve the capacity of the F-avoiding channel.
{"title":"Rates of constant-composition codes that mitigate intercell interference","authors":"Y. M. Chee, Johan Chrisnata, H. M. Kiah, S. Ling, T. T. Nguyen, Van Khu Vu","doi":"10.1109/ISIT.2016.7541289","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541289","url":null,"abstract":"For certain families of substrings F, we provide a closed formula for the maximum size of a q-ary F-avoiding code with a given composition. In addition, we provide numerical procedures to determine the asymptotic information rate for F-avoiding codes with certain composition ratios. Using our procedures, we recover known results and compute the information rates for certain classes of F-avoiding constant-composition codes for 2 ≤ q ≤ 8. For these values of q, we find composition ratios such that the rates of F-avoiding codes with constant composition achieve the capacity of the F-avoiding channel.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128658710","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541481
Jalal Etesami, N. Kiyavash
In this paper, we introduce a new type of graphical model, interventional dependency graphs, to encode interactions among processes. These type of graphical models are defined using a measure that captures the influence relationships based on the principle of intervention. Principle of intervention discovers an influence relationship by making assignment to certain variables while fixing other variables to see how these changes influence statistics of variables of interest. Furthermore, we derive some properties of the dynamics that can be inferred from these graphs and establish the relationship between this new graphical model and the directed information graphs used for causal inference.
{"title":"Interventional dependency graphs: An approach for discovering influence structure","authors":"Jalal Etesami, N. Kiyavash","doi":"10.1109/ISIT.2016.7541481","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541481","url":null,"abstract":"In this paper, we introduce a new type of graphical model, interventional dependency graphs, to encode interactions among processes. These type of graphical models are defined using a measure that captures the influence relationships based on the principle of intervention. Principle of intervention discovers an influence relationship by making assignment to certain variables while fixing other variables to see how these changes influence statistics of variables of interest. Furthermore, we derive some properties of the dynamics that can be inferred from these graphs and establish the relationship between this new graphical model and the directed information graphs used for causal inference.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"118 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124372303","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541843
Vinay Praneeth Boda, P. Narayan
Consider a discrete memoryless multiple source with m component sources. A subset of k ≤ m sources are sampled at each time instant and jointly compressed in order to reconstruct all the m sources under a given distortion criterion. A sampling rate distortion function is studied for two main sampling schemes. First, for independent random sampling performed without knowledge of the source outputs, it is shown that the sampling rate distortion function is the same regardless of whether the decoder is informed or not of the sequence of sampling sets. Next, memoryless random sampling is considered with the sampler depending on the source outputs and with an informed decoder. It is shown that deterministic sampling, characterized by a conditional point-mass, is optimal and suffices to achieve the sampling rate distortion function. For memoryless random sampling with an uninformed decoder, an upper bound for the sampling rate distortion function is seen to possess a similar property of conditional point-mass optimality. It is shown by example that memoryless sampling with an informed decoder can outperform strictly any independent random sampler, and that memoryless sampling can do strictly better with an informed decoder than without.
{"title":"Independent and memoryless sampling rate distortion","authors":"Vinay Praneeth Boda, P. Narayan","doi":"10.1109/ISIT.2016.7541843","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541843","url":null,"abstract":"Consider a discrete memoryless multiple source with m component sources. A subset of k ≤ m sources are sampled at each time instant and jointly compressed in order to reconstruct all the m sources under a given distortion criterion. A sampling rate distortion function is studied for two main sampling schemes. First, for independent random sampling performed without knowledge of the source outputs, it is shown that the sampling rate distortion function is the same regardless of whether the decoder is informed or not of the sequence of sampling sets. Next, memoryless random sampling is considered with the sampler depending on the source outputs and with an informed decoder. It is shown that deterministic sampling, characterized by a conditional point-mass, is optimal and suffices to achieve the sampling rate distortion function. For memoryless random sampling with an uninformed decoder, an upper bound for the sampling rate distortion function is seen to possess a similar property of conditional point-mass optimality. It is shown by example that memoryless sampling with an informed decoder can outperform strictly any independent random sampler, and that memoryless sampling can do strictly better with an informed decoder than without.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"83 23","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131873642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541604
Xunrui Yin, Zongpeng Li, Xin Wang
The multiple-unicast conjecture in network coding states that for multiple unicast sessions in an undirected network, network coding has no advantage over routing in improving the throughput or saving bandwidth. In this work, we propose a reduction method to study the multiple-unicast conjecture, and prove the conjecture for a new class of networks that are characterized by relations between cut-sets and source-receiver paths. This class subsumes the two known types of networks with non-zero max-flow min-cut gaps. Further combing this result with a computer-aided search, we derive as a corollary that network coding is unnecessary in networks with up to 6 nodes. We also prove the multiple-unicast conjecture for almost all unit-link-length networks with up to 3 sessions and 7 nodes.
{"title":"A reduction approach to the multiple-unicast conjecture in network coding","authors":"Xunrui Yin, Zongpeng Li, Xin Wang","doi":"10.1109/ISIT.2016.7541604","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541604","url":null,"abstract":"The multiple-unicast conjecture in network coding states that for multiple unicast sessions in an undirected network, network coding has no advantage over routing in improving the throughput or saving bandwidth. In this work, we propose a reduction method to study the multiple-unicast conjecture, and prove the conjecture for a new class of networks that are characterized by relations between cut-sets and source-receiver paths. This class subsumes the two known types of networks with non-zero max-flow min-cut gaps. Further combing this result with a computer-aided search, we derive as a corollary that network coding is unnecessary in networks with up to 6 nodes. We also prove the multiple-unicast conjecture for almost all unit-link-length networks with up to 3 sessions and 7 nodes.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123872775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541709
Jun Chen, Hao Ye, C. Tian, Tie Liu, Zhiqing Xiao
A cyclically symmetric entropy inequality is of the form hO≥chO', where hO and hO' are two cyclic orbit entropy terms. A computational approach is formulated for bounding the extremal value of c̅, which is denoted by c̅O,O'. For two non-empty orbits O and O' of a cyclic group, it is said that O dominates O' if c̅O,O' = 1. Special attention is paid to characterizing such dominance relationship, and a graphical method is developed for that purpose.
{"title":"Cyclically symmetric entropy inequalities","authors":"Jun Chen, Hao Ye, C. Tian, Tie Liu, Zhiqing Xiao","doi":"10.1109/ISIT.2016.7541709","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541709","url":null,"abstract":"A cyclically symmetric entropy inequality is of the form h<sub>O</sub>≥ch<sub>O'</sub>, where h<sub>O</sub> and h<sub>O'</sub> are two cyclic orbit entropy terms. A computational approach is formulated for bounding the extremal value of c̅, which is denoted by c̅<sub>O,O'</sub>. For two non-empty orbits O and O' of a cyclic group, it is said that O dominates O' if c̅<sub>O,O'</sub> = 1. Special attention is paid to characterizing such dominance relationship, and a graphical method is developed for that purpose.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"15 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114020299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541673
Rafah El-Khatib, N. Macris
We consider the dynamics of belief propagation decoding of spatially coupled Low-Density Parity-Check codes. It has been conjectured that after a short transient phase, the profile of “error probabilities” along the spatial direction of a spatially coupled code develops a uniquely-shaped wavelike solution that propagates with constant velocity ν. Under this assumption and for transmission over general Binary Memoryless Symmetric channels, we derive a formula for ν. We also propose approximations that are simpler to compute and support our findings using numerical data.
{"title":"The velocity of the decoding wave for spatially coupled codes on BMS channels","authors":"Rafah El-Khatib, N. Macris","doi":"10.1109/ISIT.2016.7541673","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541673","url":null,"abstract":"We consider the dynamics of belief propagation decoding of spatially coupled Low-Density Parity-Check codes. It has been conjectured that after a short transient phase, the profile of “error probabilities” along the spatial direction of a spatially coupled code develops a uniquely-shaped wavelike solution that propagates with constant velocity ν. Under this assumption and for transmission over general Binary Memoryless Symmetric channels, we derive a formula for ν. We also propose approximations that are simpler to compute and support our findings using numerical data.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131629544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541651
Basak Guler, Deniz Gündüz, A. Yener
We study lossy communication of correlated sources over a multiple access channel. In particular, we provide a joint source-channel coding scheme for transmitting correlated sources with decoder side information, and study the conditions under which separate source and channel coding is optimal. For the latter, the encoders and/or the decoder have access to a common observation conditioned on which the two sources are independent. By establishing necessary and sufficient conditions, we show the optimality of separation when the encoders and the decoder both have access to the common observation. We also demonstrate that separation is optimal when only the encoders have access to the common observation whose lossless recovery is required at the decoder. As a special case, we study separation for sources with a common part. Our results indicate that side information can have significant impact on the optimality of source-channel separation in lossy transmission.
{"title":"On lossy transmission of correlated sources over a multiple access channel","authors":"Basak Guler, Deniz Gündüz, A. Yener","doi":"10.1109/ISIT.2016.7541651","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541651","url":null,"abstract":"We study lossy communication of correlated sources over a multiple access channel. In particular, we provide a joint source-channel coding scheme for transmitting correlated sources with decoder side information, and study the conditions under which separate source and channel coding is optimal. For the latter, the encoders and/or the decoder have access to a common observation conditioned on which the two sources are independent. By establishing necessary and sufficient conditions, we show the optimality of separation when the encoders and the decoder both have access to the common observation. We also demonstrate that separation is optimal when only the encoders have access to the common observation whose lossless recovery is required at the decoder. As a special case, we study separation for sources with a common part. Our results indicate that side information can have significant impact on the optimality of source-channel separation in lossy transmission.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"198 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121070815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541496
Kristo Visk, Ago-Erik Riet
We develop a general framework to prove Kraft-type inequalities for prefix-free permutation codes for source coding with various notions of permutation code and prefix. We also show that the McMillan-type converse theorem in most of these cases fails, and give a general form of a counterexample. Our approach is more general and works for other structures besides permutation codes. The classical Kraft inequality for prefix-free codes and results about permutation codes follow as corollaries of our main theorem and main counterexample.
{"title":"Generalisation of Kraft inequality for source coding into permutations","authors":"Kristo Visk, Ago-Erik Riet","doi":"10.1109/ISIT.2016.7541496","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541496","url":null,"abstract":"We develop a general framework to prove Kraft-type inequalities for prefix-free permutation codes for source coding with various notions of permutation code and prefix. We also show that the McMillan-type converse theorem in most of these cases fails, and give a general form of a counterexample. Our approach is more general and works for other structures besides permutation codes. The classical Kraft inequality for prefix-free codes and results about permutation codes follow as corollaries of our main theorem and main counterexample.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128434362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-08-11DOI: 10.1109/ISIT.2016.7541503
A. Bracher, A. Lapidoth
The zero-error feedback capacity of the Gelfand-Pinsker channel is established. It can be positive even if the channel's zero-error capacity is zero in the absence of feedback. Moreover, the error-free transmission of a single bit may require more than one channel use.
{"title":"The zero-error capacity of the Gelfand-Pinsker channel with a feedback link","authors":"A. Bracher, A. Lapidoth","doi":"10.1109/ISIT.2016.7541503","DOIUrl":"https://doi.org/10.1109/ISIT.2016.7541503","url":null,"abstract":"The zero-error feedback capacity of the Gelfand-Pinsker channel is established. It can be positive even if the channel's zero-error capacity is zero in the absence of feedback. Moreover, the error-free transmission of a single bit may require more than one channel use.","PeriodicalId":198767,"journal":{"name":"2016 IEEE International Symposium on Information Theory (ISIT)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122309608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：