Pub Date : 2021-08-01DOI: 10.1109/mbits.2021.3097878
R. Gallager
One of Claude Shannon’s best remembered “toys” was his maze-solving machine, created by partitions on a rectangular grid. A mechanical mouse was started at one point in the maze with the task of finding cheese at another point. Relays under the board guided successive moves, each of which were taken in the first open counterclockwise direction from the previous move. In belated honor of Shannon’s centenary and of amnesia in the mouse at age 70, we compare this deterministic search strategy with a random search requiring no memory. For simplicity, the rectangular grid with partitions is replaced by a finite connected graph. A maze is then a graph with some given destination node. The worst case required number of steps to find the cheese for deterministic searches and the expected number for random searches are remarkably similar, each being, for example, $|mathcal {E}^2|$|E2| taken over all graphs of $|mathcal {E}|$|E| edges. Finally, we demonstrate a simple improvement to the above algorithms that generates an Eulerian cycle on the directed edges of $G$G, i.e., a walk on $G$G of $2|mathcal {E}|$2|E| steps that traverses each edge in $G$G exactly once in each direction before returning to the starting point.
{"title":"Shannon, Euler, and Mazes","authors":"R. Gallager","doi":"10.1109/mbits.2021.3097878","DOIUrl":"https://doi.org/10.1109/mbits.2021.3097878","url":null,"abstract":"One of Claude Shannon’s best remembered “toys” was his maze-solving machine, created by partitions on a rectangular grid. A mechanical mouse was started at one point in the maze with the task of finding cheese at another point. Relays under the board guided successive moves, each of which were taken in the first open counterclockwise direction from the previous move. In belated honor of Shannon’s centenary and of amnesia in the mouse at age 70, we compare this deterministic search strategy with a random search requiring no memory. For simplicity, the rectangular grid with partitions is replaced by a finite connected graph. A maze is then a graph with some given destination node. The worst case required number of steps to find the cheese for deterministic searches and the expected number for random searches are remarkably similar, each being, for example, $|mathcal {E}^2|$|E2| taken over all graphs of $|mathcal {E}|$|E| edges. Finally, we demonstrate a simple improvement to the above algorithms that generates an Eulerian cycle on the directed edges of $G$G, i.e., a walk on $G$G of $2|mathcal {E}|$2|E| steps that traverses each edge in $G$G exactly once in each direction before returning to the starting point.","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128784340","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-01DOI: 10.1109/MBITS.2021.3108124
Hsiang Hsu, Natalia Martínez, M. Bertrán, Guillermo Sapiro, F. Calmon
Privacy has become an emerging challenge in both information theory and computer science due to massive (centralized) collection of user data. In this article, we overview privacy-preserving mechanisms and metrics from the lenses of information theory, and unify different privacy metrics, including f-divergences, Rényi divergences, and differential privacy (DP), in terms of the probability likelihood ratio (and its logarithm). We review recent progress on the design of privacy-preserving mechanisms according to the privacy metrics in computer science, where DP is the standard privacy notion which controls the output shift given small input perturbation, and information theory, where the privacy is guaranteed by minimizing information leakage. In particular, for DP, we include its important variants (e.g., Rényi DP, Pufferfish privacy) and properties, discuss its connections with information-theoretic quantities, and provide the operational interpretations of its additive noise mechanisms. For information-theoretic privacy, we cover notable frameworks, including the privacy funnel, originated from rate-distortion theory and information bottleneck, to privacy guarantee against statistical inference/guessing, and information obfuscation on samples and features. Finally, we discuss the implementations of these privacy-preserving mechanisms in current data-driven machine learning scenarios, including deep learning, information obfuscation, federated learning, and dataset sharing.
{"title":"A Survey on Statistical, Information, and Estimation—Theoretic Views on Privacy","authors":"Hsiang Hsu, Natalia Martínez, M. Bertrán, Guillermo Sapiro, F. Calmon","doi":"10.1109/MBITS.2021.3108124","DOIUrl":"https://doi.org/10.1109/MBITS.2021.3108124","url":null,"abstract":"Privacy has become an emerging challenge in both information theory and computer science due to massive (centralized) collection of user data. In this article, we overview privacy-preserving mechanisms and metrics from the lenses of information theory, and unify different privacy metrics, including f-divergences, Rényi divergences, and differential privacy (DP), in terms of the probability likelihood ratio (and its logarithm). We review recent progress on the design of privacy-preserving mechanisms according to the privacy metrics in computer science, where DP is the standard privacy notion which controls the output shift given small input perturbation, and information theory, where the privacy is guaranteed by minimizing information leakage. In particular, for DP, we include its important variants (e.g., Rényi DP, Pufferfish privacy) and properties, discuss its connections with information-theoretic quantities, and provide the operational interpretations of its additive noise mechanisms. For information-theoretic privacy, we cover notable frameworks, including the privacy funnel, originated from rate-distortion theory and information bottleneck, to privacy guarantee against statistical inference/guessing, and information obfuscation on samples and features. Finally, we discuss the implementations of these privacy-preserving mechanisms in current data-driven machine learning scenarios, including deep learning, information obfuscation, federated learning, and dataset sharing.","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115434439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-01DOI: 10.1109/mbits.2021.3116479
SAR has become a valuable tool for civilian remote sensing applications as well as for military surveillance and reconnaissance. SAR operations can take place in all weather and times. SAR data can provide key information about the scene which can be extracted e.g. from the polarimetric features, the phase variation over time, and the reflectivity dependency on frequency. A wide variety of airand space based sensors for long and short range operation has been realized, operating at frequencies extending from VHF to the upper millimeter wave region. Spectacular missions like the Shuttle Radar Topographic Mission, the TanDEM-X satellite pair and the COSMO/SkyMed constellation have underscored the unique and important role of SAR. In addition to the basic SAR modes, acrossand along track interferometry have been established. Multi-band operations and the full polarimetric scattering matrix have been effectively utilized.
{"title":"IEEE Signal Processing Magazine","authors":"","doi":"10.1109/mbits.2021.3116479","DOIUrl":"https://doi.org/10.1109/mbits.2021.3116479","url":null,"abstract":"SAR has become a valuable tool for civilian remote sensing applications as well as for military surveillance and reconnaissance. SAR operations can take place in all weather and times. SAR data can provide key information about the scene which can be extracted e.g. from the polarimetric features, the phase variation over time, and the reflectivity dependency on frequency. A wide variety of airand space based sensors for long and short range operation has been realized, operating at frequencies extending from VHF to the upper millimeter wave region. Spectacular missions like the Shuttle Radar Topographic Mission, the TanDEM-X satellite pair and the COSMO/SkyMed constellation have underscored the unique and important role of SAR. In addition to the basic SAR modes, acrossand along track interferometry have been established. Multi-band operations and the full polarimetric scattering matrix have been effectively utilized.","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"108 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122881102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-01DOI: 10.1109/mbits.2021.3123197
R. Yeung, Cheuk Ting Li
The entropy function plays a central role in information theory. Constraints on the entropy function in the form of inequalities, viz. entropy inequalities (often conditional on certain Markov conditions imposed by the problem under consideration), are indispensable tools for proving converse coding theorems. In this expository article, we give an overview of this fundamental subject. After presenting a geometrical framework for the entropy function, we explain how an entropy inequality can be formulated, with or without constraints on the entropy function. Among all entropy inequalities, Shannon-type inequalities, namely those implied by the nonnegativity of Shannon’s information measures, are best understood. The main focus of this article is the verification of Shannon-type inequalities, which in fact can be formulated as a linear programming problem. ITIP, a software package developed for this purpose, as well as two of its variants, AITIP and PSITIP, are discussed. This article ends with a discussion on the hardness of verifying entropy inequalities in general.
{"title":"Machine-Proving of Entropy Inequalities","authors":"R. Yeung, Cheuk Ting Li","doi":"10.1109/mbits.2021.3123197","DOIUrl":"https://doi.org/10.1109/mbits.2021.3123197","url":null,"abstract":"The entropy function plays a central role in information theory. Constraints on the entropy function in the form of inequalities, viz. entropy inequalities (often conditional on certain Markov conditions imposed by the problem under consideration), are indispensable tools for proving converse coding theorems. In this expository article, we give an overview of this fundamental subject. After presenting a geometrical framework for the entropy function, we explain how an entropy inequality can be formulated, with or without constraints on the entropy function. Among all entropy inequalities, Shannon-type inequalities, namely those implied by the nonnegativity of Shannon’s information measures, are best understood. The main focus of this article is the verification of Shannon-type inequalities, which in fact can be formulated as a linear programming problem. ITIP, a software package developed for this purpose, as well as two of its variants, AITIP and PSITIP, are discussed. This article ends with a discussion on the hardness of verifying entropy inequalities in general.","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122569693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-01DOI: 10.1109/mbits.2021.3118662
G. Fettweis, H. Boche
The initial vision of cellular communications was to deliver ubiquitous voice communications to anyone anywhere. In a simplified view, 1G delivered voice services for business customers, and only 2G for consumers. Next, this also initiated the appetite for cellular data, for which 3G was designed. However, Blackberry delivered business smartphones, and 4G made smartphones a consumer device. The promise of 5G is to start the Tactile Internet, to control real and virtual objects in real-time via cellular. However, the hype around 5G is, again, focusing on business customers, in particular in the context of campus networks. Consequently, 6G must provide an infrastructure to enable remote-controlled mobile robotic solutions for everyone—the Personal Tactile Internet. Which role can information and communication theory play in this context, and what are the big challenges ahead?
{"title":"6G: The Personal Tactile Internet—And Open Questions for Information Theory","authors":"G. Fettweis, H. Boche","doi":"10.1109/mbits.2021.3118662","DOIUrl":"https://doi.org/10.1109/mbits.2021.3118662","url":null,"abstract":"The initial vision of cellular communications was to deliver ubiquitous voice communications to anyone anywhere. In a simplified view, 1G delivered voice services for business customers, and only 2G for consumers. Next, this also initiated the appetite for cellular data, for which 3G was designed. However, Blackberry delivered business smartphones, and 4G made smartphones a consumer device. The promise of 5G is to start the Tactile Internet, to control real and virtual objects in real-time via cellular. However, the hype around 5G is, again, focusing on business customers, in particular in the context of campus networks. Consequently, 6G must provide an infrastructure to enable remote-controlled mobile robotic solutions for everyone—the Personal Tactile Internet. Which role can information and communication theory play in this context, and what are the big challenges ahead?","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115515423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-01DOI: 10.1109/mbits.2021.3108946
S. Diggavi, A. Gamal, I. Kontoyiannis, A. Paranjpe, Murali Subbarao
In this article, we commemorate Joy Thomas, who sadly passed away in 2020. Joy is best known for his seminal textbook Elements of Information Theory, coauthored with his Ph.D. advisor the late Thomas Cover. We also catalog some of Joy’s technical contributions to information theory and to its practice. Joy also made an impact on industry through several successful startups. In recognition of Joy’s contribution to the exposition of information theory, the IEEE Information Theory Society Board of Governors has agreed to establish the Joy Thomas IEEE Information Theory Society Tutorial Paper Award. The Joy Thomas Foundation, established to honor Joy’s legacy of excellence and giving back, will sponsor this award.
在这篇文章中，我们纪念乔伊·托马斯，他在2020年不幸去世。乔伊最为人所知的是他的开创性教科书《信息论要素》(Elements of Information Theory)，这本书是他与已故的博士导师托马斯·盖伊(Thomas Cover)合著的。我们还列出了乔伊对信息理论及其实践的一些技术贡献。乔伊还通过几家成功的创业公司对行业产生了影响。为了表彰Joy在阐述信息理论方面的贡献，IEEE信息理论学会理事会同意设立Joy Thomas IEEE信息理论学会指导论文奖。乔伊·托马斯基金会(Joy Thomas Foundation)将赞助这一奖项，该基金会的成立是为了纪念乔伊的卓越遗产和回馈社会。
{"title":"Joy Thomas: Legacy, Foundation and the IT Society","authors":"S. Diggavi, A. Gamal, I. Kontoyiannis, A. Paranjpe, Murali Subbarao","doi":"10.1109/mbits.2021.3108946","DOIUrl":"https://doi.org/10.1109/mbits.2021.3108946","url":null,"abstract":"In this article, we commemorate Joy Thomas, who sadly passed away in 2020. Joy is best known for his seminal textbook Elements of Information Theory, coauthored with his Ph.D. advisor the late Thomas Cover. We also catalog some of Joy’s technical contributions to information theory and to its practice. Joy also made an impact on industry through several successful startups. In recognition of Joy’s contribution to the exposition of information theory, the IEEE Information Theory Society Board of Governors has agreed to establish the Joy Thomas IEEE Information Theory Society Tutorial Paper Award. The Joy Thomas Foundation, established to honor Joy’s legacy of excellence and giving back, will sponsor this award.","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128035557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-01DOI: 10.1109/mbits.2021.3102337
{"title":"THE INFORMATION THEORY SOCIETY PRESENTS","authors":"","doi":"10.1109/mbits.2021.3102337","DOIUrl":"https://doi.org/10.1109/mbits.2021.3102337","url":null,"abstract":"","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128240304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-23DOI: 10.1109/mbits.2021.3103144
O. Drori, Alon Mamistvalov, Oren Solomon, Yonina C. Eldar
Medical ultrasound imaging is an ongoing research field for digital signal processing. Following decades of developement in the analogue domain, the introduction of high power computation has led to increased activity and research in the fields of digital signal processing, and, most recently, in machine learning, for the sake of delivering higher quality imaging, while reducing the size of the data required to acquire, process, display and, recently, wirelessly transmit the data to remote devices. An overview of the basics of ultrasound acquisition and imaging is given, followed by a presentation of its limitations and challenges. An in-depth explanation about the development and algorithmics of the methods designed to address each of these challenges follows, with an introduction of the outstanding tasks and issues posed in each field, and their potential applications and benefits to modern healthcare.
{"title":"Compressed Ultrasound Imaging: From Sub-Nyquist Rates to Super Resolution","authors":"O. Drori, Alon Mamistvalov, Oren Solomon, Yonina C. Eldar","doi":"10.1109/mbits.2021.3103144","DOIUrl":"https://doi.org/10.1109/mbits.2021.3103144","url":null,"abstract":"Medical ultrasound imaging is an ongoing research field for digital signal processing. Following decades of developement in the analogue domain, the introduction of high power computation has led to increased activity and research in the fields of digital signal processing, and, most recently, in machine learning, for the sake of delivering higher quality imaging, while reducing the size of the data required to acquire, process, display and, recently, wirelessly transmit the data to remote devices. An overview of the basics of ultrasound acquisition and imaging is given, followed by a presentation of its limitations and challenges. An in-depth explanation about the development and algorithmics of the methods designed to address each of these challenges follows, with an introduction of the outstanding tasks and issues posed in each field, and their potential applications and benefits to modern healthcare.","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"201 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133673095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1900-01-01DOI: 10.1109/mbits.2021.3111537
Wei-ZhouLuandShun-Zheng Yu
{"title":"Publications of Information Theory Society","authors":"Wei-ZhouLuandShun-Zheng Yu","doi":"10.1109/mbits.2021.3111537","DOIUrl":"https://doi.org/10.1109/mbits.2021.3111537","url":null,"abstract":"","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115669298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1900-01-01DOI: 10.1109/mbits.2023.3263813
U. Thomas, M. Feder
{"title":"Q&A: Meir Feder on Using Information Theory's Concepts to Solve Complex Problems Across Disciplines","authors":"U. Thomas, M. Feder","doi":"10.1109/mbits.2023.3263813","DOIUrl":"https://doi.org/10.1109/mbits.2023.3263813","url":null,"abstract":"","PeriodicalId":448036,"journal":{"name":"IEEE BITS the Information Theory Magazine","volume":"28 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120860827","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：