Pub Date : 2019-10-23DOI: 10.1147/JRD.2019.2948824
R. Nair;B. S. Madsen;H. Lassen;S. Baduk;S. Nagarajan;L. H. Mogensen;R. Novack;R. Curzon;J. Paraszczak;S. Urbak
The development of MM4SIGHT, a machine learning system that enables annual forecasts of mixed-migration flows, is presented. Mixed migration refers to cross-border movements of people that are motivated by a multiplicity of factors to move including refugees fleeing persecution and conflict, victims of trafficking, and people seeking better lives and opportunity. Such populations have a range of legal status, some of which are not reflected in official government statistics. The system combines institutional estimates of migration along with in-person monitoring surveys to establish a migration volume baseline. The surveys reveal clusters of migratory drivers of populations on the move. Given macrolevel indicators that reflect migratory drivers found in the surveys, we develop an ensemble model to determine the volume of migration between source and host country along with uncertainty bounds. Using more than 80 macroindicators, we present results from a case study of migratory flows from Ethiopia to six countries. Our evaluations show error rates for annual forecasts to be within a few thousand persons per year for most destinations.
{"title":"A machine learning approach to scenario analysis and forecasting of mixed migration","authors":"R. Nair;B. S. Madsen;H. Lassen;S. Baduk;S. Nagarajan;L. H. Mogensen;R. Novack;R. Curzon;J. Paraszczak;S. Urbak","doi":"10.1147/JRD.2019.2948824","DOIUrl":"https://doi.org/10.1147/JRD.2019.2948824","url":null,"abstract":"The development of MM4SIGHT, a machine learning system that enables annual forecasts of mixed-migration flows, is presented. Mixed migration refers to cross-border movements of people that are motivated by a multiplicity of factors to move including refugees fleeing persecution and conflict, victims of trafficking, and people seeking better lives and opportunity. Such populations have a range of legal status, some of which are not reflected in official government statistics. The system combines institutional estimates of migration along with in-person monitoring surveys to establish a migration volume baseline. The surveys reveal clusters of migratory drivers of populations on the move. Given macrolevel indicators that reflect migratory drivers found in the surveys, we develop an ensemble model to determine the volume of migration between source and host country along with uncertainty bounds. Using more than 80 macroindicators, we present results from a case study of migratory flows from Ethiopia to six countries. Our evaluations show error rates for annual forecasts to be within a few thousand persons per year for most destinations.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2948824","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49980045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-16DOI: 10.1147/JRD.2019.2947373
S. Mukhopadhyay;Y. Long;B. Mudassar;C. S. Nair;B. H. DeProspo;H. M. Torun;M. Kathaperumal;V. Smet;D. Kim;S. Yalamanchili;M. Swaminathan
The recent progress in artificial intelligence (AI) and machine learning (ML) has enabled computing platforms to solve highly complex difficult problems in computer vision, robotics, finance, security, and science. The algorithmic progress in AI/ML have motivated new research in hardware accelerators. The dedicated accelerators promise high energy efficiency compared to software solutions using CPU. However, as AI/ML models become complex, the increasing memory demands and, hence, high energy/time cost of communication between logic and memory possess a major challenge to energy efficiency. We review the potential of heterogeneous integration in addressing the preceding challenge and present different approaches to leverage heterogeneous integration for energy-efficient AI platforms. First, we discuss packaging technologies for efficient chip-to-chip communication. Second, we present near-memory-processing architecture for AI accelerations that leverages 3D die-stacking. Third, processing-in-memory architectures using heterogeneous integration of CMOS and embedded non-volatile memory are presented. Finally, the article presents case studies that integrate preceding concepts to advance AI/ML hardware platform for different application domains.
{"title":"Heterogeneous integration for artificial intelligence: Challenges and opportunities","authors":"S. Mukhopadhyay;Y. Long;B. Mudassar;C. S. Nair;B. H. DeProspo;H. M. Torun;M. Kathaperumal;V. Smet;D. Kim;S. Yalamanchili;M. Swaminathan","doi":"10.1147/JRD.2019.2947373","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947373","url":null,"abstract":"The recent progress in artificial intelligence (AI) and machine learning (ML) has enabled computing platforms to solve highly complex difficult problems in computer vision, robotics, finance, security, and science. The algorithmic progress in AI/ML have motivated new research in hardware accelerators. The dedicated accelerators promise high energy efficiency compared to software solutions using CPU. However, as AI/ML models become complex, the increasing memory demands and, hence, high energy/time cost of communication between logic and memory possess a major challenge to energy efficiency. We review the potential of heterogeneous integration in addressing the preceding challenge and present different approaches to leverage heterogeneous integration for energy-efficient AI platforms. First, we discuss packaging technologies for efficient chip-to-chip communication. Second, we present near-memory-processing architecture for AI accelerations that leverages 3D die-stacking. Third, processing-in-memory architectures using heterogeneous integration of CMOS and embedded non-volatile memory are presented. Finally, the article presents case studies that integrate preceding concepts to advance AI/ML hardware platform for different application domains.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947373","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49993118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-16DOI: 10.1147/JRD.2019.2945519
P. Sattigeri;S. C. Hoffman;V. Chenthamarakshan;K. R. Varshney
We introduce the Fairness GAN (generative adversarial network), an approach for generating a dataset that is plausibly similar to a given multimedia dataset, but is more fair with respect to protected attributes in decision making. We propose a novel auxiliary classifier GAN that strives for demographic parity or equality of opportunity and show empirical results on several datasets, including the CelebFaces Attributes (CelebA) dataset, the Quick, Draw! dataset, and a dataset of soccer player images and the offenses for which they were called. The proposed formulation is well suited to absorbing unlabeled data; we leverage this to augment the soccer dataset with the much larger CelebA dataset. The methodology tends to improve demographic parity and equality of opportunity while generating plausible images.
{"title":"Fairness GAN: Generating datasets with fairness properties using a generative adversarial network","authors":"P. Sattigeri;S. C. Hoffman;V. Chenthamarakshan;K. R. Varshney","doi":"10.1147/JRD.2019.2945519","DOIUrl":"https://doi.org/10.1147/JRD.2019.2945519","url":null,"abstract":"We introduce the Fairness GAN (generative adversarial network), an approach for generating a dataset that is plausibly similar to a given multimedia dataset, but is more fair with respect to protected attributes in decision making. We propose a novel auxiliary classifier GAN that strives for demographic parity or equality of opportunity and show empirical results on several datasets, including the CelebFaces Attributes (CelebA) dataset, the Quick, Draw! dataset, and a dataset of soccer player images and the offenses for which they were called. The proposed formulation is well suited to absorbing unlabeled data; we leverage this to augment the soccer dataset with the much larger CelebA dataset. The methodology tends to improve demographic parity and equality of opportunity while generating plausible images.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2945519","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49946299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2945693
S. Dalal;D. Bassu
We discuss dynamic real-time analysis from multimodal data fusion for contextual risk identification to generate “risk maps” for the workplace, resulting in timely identification of hazards and associated risk mitigation. It includes new machine/deep learning, analytics, methods, and its applications that deal with the unconventional data collected from pictures, videos, documents, mobile apps, sensors/Internet of Things, Occupational Safety and Health Administration (OSHA) rules, and Building Information Model (BIM) Models. Specifically, we describe a number of advances and challenges in this field with applications of computer vision, natural language processing, and sensor data analysis. Applications include automated cause identification, damage prevention, and disaster recovery using current and historical claims data and other public data. The methods developed can be applied to any given situation with different groups of people, including first responders. Finally, we discuss some of the important nontechnical challenges related to business practicality, privacy, and industry regulations.
{"title":"Deep analytics for workplace risk and disaster management","authors":"S. Dalal;D. Bassu","doi":"10.1147/JRD.2019.2945693","DOIUrl":"https://doi.org/10.1147/JRD.2019.2945693","url":null,"abstract":"We discuss dynamic real-time analysis from multimodal data fusion for contextual risk identification to generate “risk maps” for the workplace, resulting in timely identification of hazards and associated risk mitigation. It includes new machine/deep learning, analytics, methods, and its applications that deal with the unconventional data collected from pictures, videos, documents, mobile apps, sensors/Internet of Things, Occupational Safety and Health Administration (OSHA) rules, and Building Information Model (BIM) Models. Specifically, we describe a number of advances and challenges in this field with applications of computer vision, natural language processing, and sensor data analysis. Applications include automated cause identification, damage prevention, and disaster recovery using current and historical claims data and other public data. The methods developed can be applied to any given situation with different groups of people, including first responders. Finally, we discuss some of the important nontechnical challenges related to business practicality, privacy, and industry regulations.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2945693","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49986746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2947018
P. Pandey;R. Litoriya
This article attempts to provide a minimal disaster management framework for the elderly who are living alone. Elderly people are generally vulnerable to hazards and emergency situations. The proposed framework aims at developing an Internet of Things (IoT)-based intelligent, protective ecosystem for elderly that calls for help in emergencies such as floods, earthquakes, home fires, volcanic eruptions, and storms. This disaster system makes use of a range of calamity sensors in conjunction with in-house activities of the elderly subject. In the case of mishappening, the disaster relief authorities, community members, and other stakeholders will be instantly informed. All these sensors are powered through an uninterrupted electricity supply system, which will continue to work even in the case of power outages. The work carried out in this article is deeply inspired by the need to have holistic platforms that ensure a low-cost, robust, and responsive disaster alert system for the elderly (DASE) in place. Our objective is to overcome many of the shortcomings in the existing systems and offer a reactive disaster recovery technique. Additionally, this article also incorporates the need to take care of numerous important factors, for instance, the elderly individual's limited physical and cognitive limitations, ergonomic requirements, spending capacity, etc.
{"title":"Elderly care through unusual behavior detection: A disaster management approach using IoT and intelligence","authors":"P. Pandey;R. Litoriya","doi":"10.1147/JRD.2019.2947018","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947018","url":null,"abstract":"This article attempts to provide a minimal disaster management framework for the elderly who are living alone. Elderly people are generally vulnerable to hazards and emergency situations. The proposed framework aims at developing an Internet of Things (IoT)-based intelligent, protective ecosystem for elderly that calls for help in emergencies such as floods, earthquakes, home fires, volcanic eruptions, and storms. This disaster system makes use of a range of calamity sensors in conjunction with in-house activities of the elderly subject. In the case of mishappening, the disaster relief authorities, community members, and other stakeholders will be instantly informed. All these sensors are powered through an uninterrupted electricity supply system, which will continue to work even in the case of power outages. The work carried out in this article is deeply inspired by the need to have holistic platforms that ensure a low-cost, robust, and responsive disaster alert system for the elderly (DASE) in place. Our objective is to overcome many of the shortcomings in the existing systems and offer a reactive disaster recovery technique. Additionally, this article also incorporates the need to take care of numerous important factors, for instance, the elderly individual's limited physical and cognitive limitations, ergonomic requirements, spending capacity, etc.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947018","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49980051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2947002
D. Shiri;F. S. Salman
After a disaster, first responders should reach critical locations in the disaster-affected region in the shortest time. However, road network edges can be damaged or blocked by debris. Since response time is crucial, relief operations may start before knowing which edges are blocked. A blocked edge is revealed online when it is visited at one of its end-nodes. Multiple first-responder teams, who can communicate the blockage information, gather initially at an origin node and are assigned to target destinations (nodes) in the disaster-affected area. We consider multiple teams assigned to one destination. The objective is to find an online travel plan such that at least one of the teams finds a route from the origin to the destination in minimum time. This problem is known as the online multi-agent Canadian traveler problem. We develop an effective online heuristic policy and test it on real city road networks as well as randomly generated networks leading to instances with multiple blockages. We compare the performance of the online strategy with the offline optimum and obtain an average competitive ratio of 1.164 over 70,100 instances with varying parameter values.
{"title":"Online optimization of first-responder routes in disaster response logistics","authors":"D. Shiri;F. S. Salman","doi":"10.1147/JRD.2019.2947002","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947002","url":null,"abstract":"After a disaster, first responders should reach critical locations in the disaster-affected region in the shortest time. However, road network edges can be damaged or blocked by debris. Since response time is crucial, relief operations may start before knowing which edges are blocked. A blocked edge is revealed online when it is visited at one of its end-nodes. Multiple first-responder teams, who can communicate the blockage information, gather initially at an origin node and are assigned to target destinations (nodes) in the disaster-affected area. We consider multiple teams assigned to one destination. The objective is to find an online travel plan such that at least one of the teams finds a route from the origin to the destination in minimum time. This problem is known as the online multi-agent Canadian traveler problem. We develop an effective online heuristic policy and test it on real city road networks as well as randomly generated networks leading to instances with multiple blockages. We compare the performance of the online strategy with the offline optimum and obtain an average competitive ratio of 1.164 over 70,100 instances with varying parameter values.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947002","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49986747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2947008
E. Eleftheriou;M. Le Gallo;S. R. Nandakumar;C. Piveteau;I. Boybat;V. Joshi;R. Khaddam-Aljameh;M. Dazzi;I. Giannopoulos;G. Karunaratne;B. Kersting;M. Stanisavljevic;V. P. Jonnalagadda;N. Ioannou;K. Kourtis;P. A. Francese;A. Sebastian
Performing computations on conventional von Neumann computing systems results in a significant amount of data being moved back and forth between the physically separated memory and processing units. This costs time and energy, and constitutes an inherent performance bottleneck. In-memory computing is a novel non-von Neumann approach, where certain computational tasks are performed in the memory itself. This is enabled by the physical attributes and state dynamics of memory devices, in particular, resistance-based nonvolatile memory technology. Several computational tasks such as logical operations, arithmetic operations, and even certain machine learning tasks can be implemented in such a computational memory unit. In this article, we first introduce the general notion of in-memory computing and then focus on mixed-precision deep learning training with in-memory computing. The efficacy of this new approach will be demonstrated by training the MNIST multilayer perceptron network achieving high accuracy. Moreover, we show how the precision of in-memory computing can be further improved through architectural and device-level innovations. Finally, we present system aspects, such as high-level system architecture, including core-to-core interconnect technologies, and high-level ideas and concepts of the software stack .
{"title":"Deep learning acceleration based on in-memory computing","authors":"E. Eleftheriou;M. Le Gallo;S. R. Nandakumar;C. Piveteau;I. Boybat;V. Joshi;R. Khaddam-Aljameh;M. Dazzi;I. Giannopoulos;G. Karunaratne;B. Kersting;M. Stanisavljevic;V. P. Jonnalagadda;N. Ioannou;K. Kourtis;P. A. Francese;A. Sebastian","doi":"10.1147/JRD.2019.2947008","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947008","url":null,"abstract":"Performing computations on conventional von Neumann computing systems results in a significant amount of data being moved back and forth between the physically separated memory and processing units. This costs time and energy, and constitutes an inherent performance bottleneck. In-memory computing is a novel non-von Neumann approach, where certain computational tasks are performed in the memory itself. This is enabled by the physical attributes and state dynamics of memory devices, in particular, resistance-based nonvolatile memory technology. Several computational tasks such as logical operations, arithmetic operations, and even certain machine learning tasks can be implemented in such a computational memory unit. In this article, we first introduce the general notion of in-memory computing and then focus on mixed-precision deep learning training with in-memory computing. The efficacy of this new approach will be demonstrated by training the MNIST multilayer perceptron network achieving high accuracy. Moreover, we show how the precision of in-memory computing can be further improved through architectural and device-level innovations. Finally, we present system aspects, such as high-level system architecture, including core-to-core interconnect technologies, and high-level ideas and concepts of the software stack\u0000<italic>.</i>","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947008","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49993113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2947014
T. Kreutzer;P. Vinck;P. N. Pham;A. An;L. Appel;E. DeLuca;G. Tang;M. Alzghool;K. Hachhethu;B. Morris;S. L. Walton-Ellery;J. Crowley;J. Orbinski
An effective response to humanitarian crises relies on detailed information about the needs of the affected population. Current assessment approaches often require interviewers to convert complex, open-ended responses into simplified quantitative data. More nuanced insights require the use of qualitative methods, but proper transcription and manual coding are hard to conduct rapidly and at scale during a crisis. Natural language processing (NLP), a type of artificial intelligence, may provide potentially important new opportunities to capture qualitative data from voice responses and analyze it for relevant content to better inform more effective and rapid humanitarian assistance operational decisions. This article provides an overview of how NLP can be used to transcribe, translate, and analyze large sets of qualitative responses with a view to improving the quality and effectiveness of humanitarian assistance. We describe the practical and ethical challenges of building on the diffusion of digital data collection platforms and introducing this new technology to the humanitarian context. Finally, we provide an overview of the principles that should be used to anticipate and mitigate risks.
{"title":"Improving humanitarian needs assessments through natural language processing","authors":"T. Kreutzer;P. Vinck;P. N. Pham;A. An;L. Appel;E. DeLuca;G. Tang;M. Alzghool;K. Hachhethu;B. Morris;S. L. Walton-Ellery;J. Crowley;J. Orbinski","doi":"10.1147/JRD.2019.2947014","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947014","url":null,"abstract":"An effective response to humanitarian crises relies on detailed information about the needs of the affected population. Current assessment approaches often require interviewers to convert complex, open-ended responses into simplified quantitative data. More nuanced insights require the use of qualitative methods, but proper transcription and manual coding are hard to conduct rapidly and at scale during a crisis. Natural language processing (NLP), a type of artificial intelligence, may provide potentially important new opportunities to capture qualitative data from voice responses and analyze it for relevant content to better inform more effective and rapid humanitarian assistance operational decisions. This article provides an overview of how NLP can be used to transcribe, translate, and analyze large sets of qualitative responses with a view to improving the quality and effectiveness of humanitarian assistance. We describe the practical and ethical challenges of building on the diffusion of digital data collection platforms and introducing this new technology to the humanitarian context. Finally, we provide an overview of the principles that should be used to anticipate and mitigate risks.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947014","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49953417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2947011
S. Jain;A. Ankit;I. Chakraborty;T. Gokmen;M. Rasch;W. Haensch;K. Roy;A. Raghunathan
Deep neural networks (DNNs) achieve best-known accuracies in many machine learning tasks involved in image, voice, and natural language processing and are being used in an ever-increasing range of applications. However, their algorithmic benefits are accompanied by extremely high computation and storage costs, sparking intense efforts in optimizing the design of computing platforms for DNNs. Today, graphics processing units (GPUs) and specialized digital CMOS accelerators represent the state-of-the-art in DNN hardware, with near-term efforts focusing on approximate computing through reduced precision. However, the ever-increasing complexities of DNNs and the data they process have fueled an active interest in alternative hardware fabrics that can deliver the next leap in efficiency. Resistive crossbars designed using emerging nonvolatile memory technologies have emerged as a promising candidate building block for future DNN hardware fabrics since they can natively execute massively parallel vector-matrix multiplications (the dominant compute kernel in DNNs) in the analog domain within the memory arrays. Leveraging in-memory computing and dense storage, resistive-crossbar-based systems cater to both the high computation and storage demands of complex DNNs and promise energy efficiency beyond current DNN accelerators by mitigating data transfer and memory bottlenecks. However, several design challenges need to be addressed to enable their adoption. For example, the overheads of peripheral circuits (analog-to-digital converters and digital-to-analog converters) and other components (scratchpad memories and on-chip interconnect) may significantly diminish the efficiency benefits at the system level. Additionally, the analog crossbar computations are intrinsically subject to noise due to a range of device- and circuit-level nonidealities, potentially leading to lower accuracy at the application level. In this article, we highlight the prospects for designing hardware accelerators for neural networks using resistive crossbars. We also underscore the key open challenges and some possible approaches to address them.
{"title":"Neural network accelerator design with resistive crossbars: Opportunities and challenges","authors":"S. Jain;A. Ankit;I. Chakraborty;T. Gokmen;M. Rasch;W. Haensch;K. Roy;A. Raghunathan","doi":"10.1147/JRD.2019.2947011","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947011","url":null,"abstract":"Deep neural networks (DNNs) achieve best-known accuracies in many machine learning tasks involved in image, voice, and natural language processing and are being used in an ever-increasing range of applications. However, their algorithmic benefits are accompanied by extremely high computation and storage costs, sparking intense efforts in optimizing the design of computing platforms for DNNs. Today, graphics processing units (GPUs) and specialized digital CMOS accelerators represent the state-of-the-art in DNN hardware, with near-term efforts focusing on approximate computing through reduced precision. However, the ever-increasing complexities of DNNs and the data they process have fueled an active interest in alternative hardware fabrics that can deliver the next leap in efficiency. Resistive crossbars designed using emerging nonvolatile memory technologies have emerged as a promising candidate building block for future DNN hardware fabrics since they can natively execute massively parallel vector-matrix multiplications (the dominant compute kernel in DNNs) in the analog domain within the memory arrays. Leveraging in-memory computing and dense storage, resistive-crossbar-based systems cater to both the high computation and storage demands of complex DNNs and promise energy efficiency beyond current DNN accelerators by mitigating data transfer and memory bottlenecks. However, several design challenges need to be addressed to enable their adoption. For example, the overheads of peripheral circuits (analog-to-digital converters and digital-to-analog converters) and other components (scratchpad memories and on-chip interconnect) may significantly diminish the efficiency benefits at the system level. Additionally, the analog crossbar computations are intrinsically subject to noise due to a range of device- and circuit-level nonidealities, potentially leading to lower accuracy at the application level. In this article, we highlight the prospects for designing hardware accelerators for neural networks using resistive crossbars. We also underscore the key open challenges and some possible approaches to address them.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947011","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49993116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-14DOI: 10.1147/JRD.2019.2947013
M. Cho;U. Finkler;M. Serrano;D. Kung;H. Hunter
As deep neural networks get more complex and input datasets get larger, it can take days or even weeks to train a deep neural network to the desired accuracy. Therefore, enabling distributed deep learning at a massive scale is critical since it offers the potential to reduce the training time from weeks to hours. In this article, we present BlueConnect, an efficient communication library for distributed deep learning that is highly optimized for popular GPU-based platforms. BlueConnect decomposes a single all-reduce operation into a large number of parallelizable reduce–scatter and all-gather operations to exploit the tradeoff between latency and bandwidth and adapt to a variety of network configurations. Therefore, each individual operation can be mapped to a different network fabric and take advantage of the best performing implementation for the corresponding fabric. According to our experimental results on two system configurations, BlueConnect can outperform the leading industrial communication library by a wide margin, and the BlueConnect-integrated Caffe2 can significantly reduce synchronization overhead by 87% on 192 GPUs for Resnet-50 training over prior schemes.
{"title":"BlueConnect: Decomposing all-reduce for deep learning on heterogeneous network hierarchy","authors":"M. Cho;U. Finkler;M. Serrano;D. Kung;H. Hunter","doi":"10.1147/JRD.2019.2947013","DOIUrl":"https://doi.org/10.1147/JRD.2019.2947013","url":null,"abstract":"As deep neural networks get more complex and input datasets get larger, it can take days or even weeks to train a deep neural network to the desired accuracy. Therefore, enabling distributed deep learning at a massive scale is critical since it offers the potential to reduce the training time from weeks to hours. In this article, we present BlueConnect, an efficient communication library for distributed deep learning that is highly optimized for popular GPU-based platforms. BlueConnect decomposes a single all-reduce operation into a large number of parallelizable reduce–scatter and all-gather operations to exploit the tradeoff between latency and bandwidth and adapt to a variety of network configurations. Therefore, each individual operation can be mapped to a different network fabric and take advantage of the best performing implementation for the corresponding fabric. According to our experimental results on two system configurations, BlueConnect can outperform the leading industrial communication library by a wide margin, and the BlueConnect-integrated Caffe2 can significantly reduce synchronization overhead by 87% on 192 GPUs for Resnet-50 training over prior schemes.","PeriodicalId":55034,"journal":{"name":"IBM Journal of Research and Development","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2019-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1147/JRD.2019.2947013","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49993119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：