Junxian He , Min Tian , Ying Jiang , Haibing Wang , Tengxiao Wang , Xichuan Zhou , Liyuan Liu , Nanjian Wu , Ying Wang , Cong Shi
{"title":"A visual cortex-inspired edge neuromorphic hardware architecture with on-chip multi-layer STDP learning","authors":"Junxian He , Min Tian , Ying Jiang , Haibing Wang , Tengxiao Wang , Xichuan Zhou , Liyuan Liu , Nanjian Wu , Ying Wang , Cong Shi","doi":"10.1016/j.compeleceng.2024.109806","DOIUrl":null,"url":null,"abstract":"<div><div>The era of artificial intelligence of things (AIoT) brings huge challenges on edge visual processing systems under strict processing latency, cost and energy budgets. The emergence of computationally efficient biological spiking neural networks (SNN) and event-driven neuromorphic architecture in recent years have fostered a computing paradigm shift to address these challenges. In this paper, we propose a neuromorphic processor architecture for a multi-layer convolutional SNN (codenamed HMAX SNN model) inspired by human visual cortex hierarchy. The main contributions of this work include: 1) It proposes a fully event-driven, modular, configurable and scalable neuromorphic architecture allowing for flexible tradeoffs among implementation cost, processing speed and visual recognition accuracy with multi-layer convolutional SNNs. 2) It proposes a run-time reconfigurable learning engine enabling fast on-chip unsupervised spike-timing dependent plasticity (STDP) learning for the feature-extraction convolutional layers and also supervised STDP learning for the feature-classification FC layer, in a time-multiplexing way. These techniques greatly improve on-chip learning accuracies beyond 97 % on the Modified National Institute of Standards and Technology database (MNIST) images for the first time among existing edge neuromorphic systems, at reasonable computational and memory costs. Our hardware processor architecture was prototyped on a low-cost Zedboard Zynq-7020 Field-Programmable Gate Array (FPGA) device, and validated on the MNIST, Fashion-MNIST, Olivetti Research Laboratory (ORL) human faces and ETH-80 image datasets. The experimental results demonstrate that the proposed neuromorphic architecture can achieve comparably high on-chip learning accuracy, high inference throughput and high energy efficiency using relatively fewer hardware resource consumptions. We anticipate that the HMAX SNN processor can potentially enhance deployments of edge neuromorphic processors in more practical edge applications.</div></div>","PeriodicalId":50630,"journal":{"name":"Computers & Electrical Engineering","volume":"120 ","pages":"Article 109806"},"PeriodicalIF":4.0000,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers & Electrical Engineering","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S004579062400733X","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}
引用次数: 0
Abstract
The era of artificial intelligence of things (AIoT) brings huge challenges on edge visual processing systems under strict processing latency, cost and energy budgets. The emergence of computationally efficient biological spiking neural networks (SNN) and event-driven neuromorphic architecture in recent years have fostered a computing paradigm shift to address these challenges. In this paper, we propose a neuromorphic processor architecture for a multi-layer convolutional SNN (codenamed HMAX SNN model) inspired by human visual cortex hierarchy. The main contributions of this work include: 1) It proposes a fully event-driven, modular, configurable and scalable neuromorphic architecture allowing for flexible tradeoffs among implementation cost, processing speed and visual recognition accuracy with multi-layer convolutional SNNs. 2) It proposes a run-time reconfigurable learning engine enabling fast on-chip unsupervised spike-timing dependent plasticity (STDP) learning for the feature-extraction convolutional layers and also supervised STDP learning for the feature-classification FC layer, in a time-multiplexing way. These techniques greatly improve on-chip learning accuracies beyond 97 % on the Modified National Institute of Standards and Technology database (MNIST) images for the first time among existing edge neuromorphic systems, at reasonable computational and memory costs. Our hardware processor architecture was prototyped on a low-cost Zedboard Zynq-7020 Field-Programmable Gate Array (FPGA) device, and validated on the MNIST, Fashion-MNIST, Olivetti Research Laboratory (ORL) human faces and ETH-80 image datasets. The experimental results demonstrate that the proposed neuromorphic architecture can achieve comparably high on-chip learning accuracy, high inference throughput and high energy efficiency using relatively fewer hardware resource consumptions. We anticipate that the HMAX SNN processor can potentially enhance deployments of edge neuromorphic processors in more practical edge applications.
期刊介绍:
The impact of computers has nowhere been more revolutionary than in electrical engineering. The design, analysis, and operation of electrical and electronic systems are now dominated by computers, a transformation that has been motivated by the natural ease of interface between computers and electrical systems, and the promise of spectacular improvements in speed and efficiency.
Published since 1973, Computers & Electrical Engineering provides rapid publication of topical research into the integration of computer technology and computational techniques with electrical and electronic systems. The journal publishes papers featuring novel implementations of computers and computational techniques in areas like signal and image processing, high-performance computing, parallel processing, and communications. Special attention will be paid to papers describing innovative architectures, algorithms, and software tools.