Pub Date : 2023-12-12DOI: 10.1109/JPROC.2023.3338272
Jinli Suo;Weihang Zhang;Jin Gong;Xin Yuan;David J. Brady;Qionghai Dai
Signal capture is at the forefront of perceiving and understanding the environment; thus, imaging plays a pivotal role in mobile vision. Recent unprecedented progress in artificial intelligence (AI) has shown great potential in the development of advanced mobile platforms with new imaging devices. Traditional imaging systems based on the “capturing images first and processing afterward” mechanism cannot meet this explosive demand. On the other hand, computational imaging (CI) systems are designed to capture high-dimensional data in an encoded manner to provide more information for mobile vision systems. Thanks to AI, CI can now be used in real-life systems by integrating deep learning algorithms into the mobile vision platform to achieve a closed loop of intelligent acquisition, processing, and decision-making, thus leading to the next revolution of mobile vision. Starting from the history of mobile vision using digital cameras, this work first introduces the advancement of CI in diverse applications and then conducts a comprehensive review of current research topics combining CI and AI. Although new-generation mobile platforms, represented by smart mobile phones, have deeply integrated CI and AI for better image acquisition and processing, most mobile vision platforms, such as self-driving cars and drones only loosely connect CI and AI, and are calling for a closer integration. Motivated by this fact, at the end of this work, we propose some potential technologies and disciplines that aid the deep integration of CI and AI and shed light on new directions in the future generation of mobile vision platforms.
信号捕捉是感知和理解环境的最前沿；因此，成像在移动视觉中起着举足轻重的作用。最近，人工智能（AI）取得了前所未有的进展，这为开发配备新型成像设备的先进移动平台提供了巨大的潜力。基于 "先捕捉图像、后处理 "机制的传统成像系统无法满足这一爆炸性需求。另一方面，计算成像（CI）系统旨在以编码方式捕捉高维数据，为移动视觉系统提供更多信息。得益于人工智能的发展，CI 现在可以通过将深度学习算法集成到移动视觉平台中，实现智能采集、处理和决策的闭环，从而应用于现实系统中，从而引发移动视觉的下一次革命。本著作从使用数码相机的移动视觉的历史出发，首先介绍了 CI 在各种应用中的进展，然后对当前 CI 与 AI 结合的研究课题进行了全面回顾。尽管以智能手机为代表的新一代移动平台已将 CI 与 AI 深度结合，以实现更好的图像采集和处理，但大多数移动视觉平台（如自动驾驶汽车和无人机）只是将 CI 与 AI 松散地联系在一起，因此需要更紧密的结合。在这一事实的推动下，我们在本作品的最后提出了一些有助于 CI 和 AI 深度融合的潜在技术和学科，并阐明了未来新一代移动视觉平台的新方向。
{"title":"Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision","authors":"Jinli Suo;Weihang Zhang;Jin Gong;Xin Yuan;David J. Brady;Qionghai Dai","doi":"10.1109/JPROC.2023.3338272","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3338272","url":null,"abstract":"Signal capture is at the forefront of perceiving and understanding the environment; thus, imaging plays a pivotal role in mobile vision. Recent unprecedented progress in artificial intelligence (AI) has shown great potential in the development of advanced mobile platforms with new imaging devices. Traditional imaging systems based on the “capturing images first and processing afterward” mechanism cannot meet this explosive demand. On the other hand, computational imaging (CI) systems are designed to capture high-dimensional data in an encoded manner to provide more information for mobile vision systems. Thanks to AI, CI can now be used in real-life systems by integrating deep learning algorithms into the mobile vision platform to achieve a closed loop of intelligent acquisition, processing, and decision-making, thus leading to the next revolution of mobile vision. Starting from the history of mobile vision using digital cameras, this work first introduces the advancement of CI in diverse applications and then conducts a comprehensive review of current research topics combining CI and AI. Although new-generation mobile platforms, represented by smart mobile phones, have deeply integrated CI and AI for better image acquisition and processing, most mobile vision platforms, such as self-driving cars and drones only loosely connect CI and AI, and are calling for a closer integration. Motivated by this fact, at the end of this work, we propose some potential technologies and disciplines that aid the deep integration of CI and AI and shed light on new directions in the future generation of mobile vision platforms.","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 12","pages":"1607-1639"},"PeriodicalIF":20.6,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138739398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Graph neural networks (GNNs) have been demonstrated to be a powerful algorithmic model in broad application fields for their effectiveness in learning over graphs. To scale GNN training up for large-scale and ever-growing graphs, the most promising solution is distributed training that distributes the workload of training across multiple computing nodes. At present, the volume of related research on distributed GNN training is exceptionally vast, accompanied by an extraordinarily rapid pace of publication. Moreover, the approaches reported in these studies exhibit significant divergence. This situation poses a considerable challenge for newcomers, hindering their ability to grasp a comprehensive understanding of the workflows, computational patterns, communication strategies, and optimization techniques employed in distributed GNN training. As a result, there is a pressing need for a survey to provide correct recognition, analysis, and comparisons in this field. In this article, we provide a comprehensive survey of distributed GNN training by investigating various optimization techniques used in distributed GNN training. First, distributed GNN training is classified into several categories according to their workflows. In addition, their computational patterns and communication patterns, as well as the optimization techniques proposed by recent work, are introduced. Second, the software frameworks and hardware platforms of distributed GNN training are also introduced for a deeper understanding. Third, distributed GNN training is compared with distributed training of deep neural networks (DNNs), emphasizing the uniqueness of distributed GNN training. Finally, interesting issues and opportunities in this field are discussed.
{"title":"A Comprehensive Survey on Distributed Training of Graph Neural Networks","authors":"Haiyang Lin;Mingyu Yan;Xiaochun Ye;Dongrui Fan;Shirui Pan;Wenguang Chen;Yuan Xie","doi":"10.1109/JPROC.2023.3337442","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3337442","url":null,"abstract":"Graph neural networks (GNNs) have been demonstrated to be a powerful algorithmic model in broad application fields for their effectiveness in learning over graphs. To scale GNN training up for large-scale and ever-growing graphs, the most promising solution is distributed training that distributes the workload of training across multiple computing nodes. At present, the volume of related research on distributed GNN training is exceptionally vast, accompanied by an extraordinarily rapid pace of publication. Moreover, the approaches reported in these studies exhibit significant divergence. This situation poses a considerable challenge for newcomers, hindering their ability to grasp a comprehensive understanding of the workflows, computational patterns, communication strategies, and optimization techniques employed in distributed GNN training. As a result, there is a pressing need for a survey to provide correct recognition, analysis, and comparisons in this field. In this article, we provide a comprehensive survey of distributed GNN training by investigating various optimization techniques used in distributed GNN training. First, distributed GNN training is classified into several categories according to their workflows. In addition, their computational patterns and communication patterns, as well as the optimization techniques proposed by recent work, are introduced. Second, the software frameworks and hardware platforms of distributed GNN training are also introduced for a deeper understanding. Third, distributed GNN training is compared with distributed training of deep neural networks (DNNs), emphasizing the uniqueness of distributed GNN training. Finally, interesting issues and opportunities in this field are discussed.","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 12","pages":"1572-1606"},"PeriodicalIF":20.6,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138739494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-21DOI: 10.1109/JPROC.2023.3330729
Mirjana Stojilović;Kasper Rasmussen;Francesco Regazzoni;Mehdi B. Tahoori;Russell Tessier
Field-programmable gate arrays (FPGAs) have become critical components in many cloud computing platforms. These devices possess the fine-grained parallelism and specialization needed to accelerate applications ranging from machine learning to networking and signal processing, among many others. Unfortunately, fine-grained programmability also makes FPGAs a security risk. Here, we review the current scope of attacks on cloud FPGAs and their remediation. Many of the FPGA security limitations are enabled by the shared power distribution network in FPGA devices. The simultaneous sharing of FPGAs is a particular concern. Other attacks on the memory, host microprocessor, and input/output channels are also possible. After examining current attacks, we describe trends in cloud architecture and how they are likely to impact possible future attacks. FPGA integration into cloud hypervisors and system software will provide extensive computing opportunities but invite new avenues of attack. We identify a series of system, software, and FPGA architectural changes that will facilitate improved security for cloud FPGAs and the overall systems in which they are located.
{"title":"A Visionary Look at the Security of Reconfigurable Cloud Computing","authors":"Mirjana Stojilović;Kasper Rasmussen;Francesco Regazzoni;Mehdi B. Tahoori;Russell Tessier","doi":"10.1109/JPROC.2023.3330729","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3330729","url":null,"abstract":"Field-programmable gate arrays (FPGAs) have become critical components in many cloud computing platforms. These devices possess the fine-grained parallelism and specialization needed to accelerate applications ranging from machine learning to networking and signal processing, among many others. Unfortunately, fine-grained programmability also makes FPGAs a security risk. Here, we review the current scope of attacks on cloud FPGAs and their remediation. Many of the FPGA security limitations are enabled by the shared power distribution network in FPGA devices. The simultaneous sharing of FPGAs is a particular concern. Other attacks on the memory, host microprocessor, and input/output channels are also possible. After examining current attacks, we describe trends in cloud architecture and how they are likely to impact possible future attacks. FPGA integration into cloud hypervisors and system software will provide extensive computing opportunities but invite new avenues of attack. We identify a series of system, software, and FPGA architectural changes that will facilitate improved security for cloud FPGAs and the overall systems in which they are located.","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 12","pages":"1548-1571"},"PeriodicalIF":20.6,"publicationDate":"2023-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138739520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Scanning the Issue","authors":"","doi":"10.1109/JPROC.2023.3328660","DOIUrl":"10.1109/JPROC.2023.3328660","url":null,"abstract":"Deep-Learning-Based 3-D Surface Reconstruction—A Survey","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"1462-1463"},"PeriodicalIF":20.6,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10323288","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138293580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1109/JPROC.2023.3328675
{"title":"IEEE Membership","authors":"","doi":"10.1109/JPROC.2023.3328675","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3328675","url":null,"abstract":"","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"C3-C3"},"PeriodicalIF":20.6,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10323246","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138431109","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1109/JPROC.2023.3328669
{"title":"Proceedings of the IEEE Publication Information","authors":"","doi":"10.1109/JPROC.2023.3328669","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3328669","url":null,"abstract":"","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"C2-C2"},"PeriodicalIF":20.6,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10323287","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138431086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1109/JPROC.2023.3328920
Onel L. A. López;Nurul H. Mahmood;Mohammad Shehab;Hirley Alves;Osmel Martínez Rosabal;Leatile Marata;Matti Latva-Aho
Ultrareliable low-latency communication (URLLC) constitutes a key service class of the fifth generation (5G) and beyond cellular networks. Notably, designing and supporting URLLC pose a herculean task due to the fundamental need to identify and accurately characterize the underlying statistical models in which the system operates, e.g., interference statistics, channel conditions, and the behavior of protocols. In general, multilayer end-to-end approaches considering all the potential delay and error sources and proper statistical tools and methodologies are inevitably required for providing strong reliability and latency guarantees. This article contributes to the body of knowledge in the latter aspect by providing a tutorial on several statistical tools and methodologies that are useful for designing and analyzing URLLC systems. Specifically, we overview the frameworks related to the following: 1) reliability theory; 2) short packet communications; 3) inequalities, distribution bounds, and tail approximations; 4) rare-events simulation; 5) queuing theory and information freshness; and 6) large-scale tools, such as stochastic geometry, clustering, compressed sensing, and mean-field (MF) games. Moreover, we often refer to prominent data-driven algorithms within the scope of the discussed tools/methodologies. Throughout this article, we briefly review the state-of-the-art works using the addressed tools and methodologies, and their link to URLLC systems. Moreover, we discuss novel application examples focused on physical and medium access control layers. Finally, key research challenges and directions are highlighted to elucidate how URLLC analysis/design research may evolve in the coming years.
{"title":"Statistical Tools and Methodologies for Ultrareliable Low-Latency Communication—A Tutorial","authors":"Onel L. A. López;Nurul H. Mahmood;Mohammad Shehab;Hirley Alves;Osmel Martínez Rosabal;Leatile Marata;Matti Latva-Aho","doi":"10.1109/JPROC.2023.3328920","DOIUrl":"10.1109/JPROC.2023.3328920","url":null,"abstract":"Ultrareliable low-latency communication (URLLC) constitutes a key service class of the fifth generation (5G) and beyond cellular networks. Notably, designing and supporting URLLC pose a herculean task due to the fundamental need to identify and accurately characterize the underlying statistical models in which the system operates, e.g., interference statistics, channel conditions, and the behavior of protocols. In general, multilayer end-to-end approaches considering all the potential delay and error sources and proper statistical tools and methodologies are inevitably required for providing strong reliability and latency guarantees. This article contributes to the body of knowledge in the latter aspect by providing a tutorial on several statistical tools and methodologies that are useful for designing and analyzing URLLC systems. Specifically, we overview the frameworks related to the following: 1) reliability theory; 2) short packet communications; 3) inequalities, distribution bounds, and tail approximations; 4) rare-events simulation; 5) queuing theory and information freshness; and 6) large-scale tools, such as stochastic geometry, clustering, compressed sensing, and mean-field (MF) games. Moreover, we often refer to prominent data-driven algorithms within the scope of the discussed tools/methodologies. Throughout this article, we briefly review the state-of-the-art works using the addressed tools and methodologies, and their link to URLLC systems. Moreover, we discuss novel application examples focused on physical and medium access control layers. Finally, key research challenges and directions are highlighted to elucidate how URLLC analysis/design research may evolve in the coming years.","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"1502-1543"},"PeriodicalIF":20.6,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10323296","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138293579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1109/JPROC.2023.3328673
{"title":"Future Special Issues/Special Sections of the Proceedings","authors":"","doi":"10.1109/JPROC.2023.3328673","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3328673","url":null,"abstract":"","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"1544-1544"},"PeriodicalIF":20.6,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10323289","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138431082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1109/JPROC.2023.3328677
{"title":"Proceedings of the IEEE: Stay Informed. Become Inspired.","authors":"","doi":"10.1109/JPROC.2023.3328677","DOIUrl":"https://doi.org/10.1109/JPROC.2023.3328677","url":null,"abstract":"","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"C4-C4"},"PeriodicalIF":20.6,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10323243","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138431108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In the last decade, deep learning (DL) has significantly impacted industry and science. Initially largely motivated by computer vision tasks in 2-D imagery, the focus has shifted toward 3-D data analysis. In particular, 3-D surface reconstruction, i.e., reconstructing a 3-D shape from sparse input, is of great interest to a large variety of application fields. DL-based approaches show promising quantitative and qualitative surface reconstruction performance compared to traditional computer vision and geometric algorithms. This survey provides a comprehensive overview of these DL-based methods for 3-D surface reconstruction. To this end, we will first discuss input data modalities, such as volumetric data, point clouds, and RGB, single-view, multiview, and depth images, along with corresponding acquisition technologies and common benchmark datasets. For practical purposes, we also discuss evaluation metrics enabling us to judge the reconstructive performance of different methods. The main part of the document will introduce a methodological taxonomy ranging from point- and mesh-based techniques to volumetric and implicit neural approaches. Recent research trends, both methodological and for applications, are highlighted, pointing toward future developments.
{"title":"Deep-Learning-Based 3-D Surface Reconstruction—A Survey","authors":"Anis Farshian;Markus Götz;Gabriele Cavallaro;Charlotte Debus;Matthias Nießner;Jón Atli Benediktsson;Achim Streit","doi":"10.1109/JPROC.2023.3321433","DOIUrl":"10.1109/JPROC.2023.3321433","url":null,"abstract":"In the last decade, deep learning (DL) has significantly impacted industry and science. Initially largely motivated by computer vision tasks in 2-D imagery, the focus has shifted toward 3-D data analysis. In particular, 3-D surface reconstruction, i.e., reconstructing a 3-D shape from sparse input, is of great interest to a large variety of application fields. DL-based approaches show promising quantitative and qualitative surface reconstruction performance compared to traditional computer vision and geometric algorithms. This survey provides a comprehensive overview of these DL-based methods for 3-D surface reconstruction. To this end, we will first discuss input data modalities, such as volumetric data, point clouds, and RGB, single-view, multiview, and depth images, along with corresponding acquisition technologies and common benchmark datasets. For practical purposes, we also discuss evaluation metrics enabling us to judge the reconstructive performance of different methods. The main part of the document will introduce a methodological taxonomy ranging from point- and mesh-based techniques to volumetric and implicit neural approaches. Recent research trends, both methodological and for applications, are highlighted, pointing toward future developments.","PeriodicalId":20556,"journal":{"name":"Proceedings of the IEEE","volume":"111 11","pages":"1464-1501"},"PeriodicalIF":20.6,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10301359","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135260924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：