Danian Dong, Woyu Zhang, Yuanlu Xie, Jinshan Yue, Kuan Ren, Hongjian Huang, Xu Zheng, Wen Xuan Sun, Jin Ru Lai, Shaoyang Fan, Hongzhou Wang, Zhaoan Yu, Zhihong Yao, Xiaoxin Xu, Dashan Shang, Ming Liu
{"title":"利用基于 RRAM 的数模混合系统实现下一代储层计算的硬件实现","authors":"Danian Dong, Woyu Zhang, Yuanlu Xie, Jinshan Yue, Kuan Ren, Hongjian Huang, Xu Zheng, Wen Xuan Sun, Jin Ru Lai, Shaoyang Fan, Hongzhou Wang, Zhaoan Yu, Zhihong Yao, Xiaoxin Xu, Dashan Shang, Ming Liu","doi":"10.1002/aisy.202400098","DOIUrl":null,"url":null,"abstract":"<p>Reservoir computing (RC) possesses a simple architecture and high energy efficiency for time-series data analysis through machine learning algorithms. To date, RC has evolved into several innovative variants. The next generation reservoir computing (NGRC) variant, founded on nonlinear vector autoregression (NVAR) distinguishes itself due to its fewer hyperparameters and independence from physical random connection matrices, while yielding comparable results. However, NGRC networks struggle with massive Kronecker product calculations and matrix-vector multiplications within the read out layer, leading to substantial efficiency challenges for traditional von Neumann architectures. In this work, a hybrid digital-analog hardware system tailored for NGRC is developed. The digital part is a Kronecker product calculation unit with data filtering, which realizes transformation of nonlinear vector of the input linear vector. For matrix-vector multiplication, a computing-in-memory architecture based on resistive random access memory array offers an energy-efficient hardware solution, which markedly reduces data transfer and greatly improve computational parallelism and energy efficiency. The predictive capabilities of this hybrid NGRC system are validated through the Lorenz63 model, achieving a normalized root mean square error (NRMSE) of 0.00098 and an energy efficiency of 19.42TOPS W<sup>−1</sup>.</p>","PeriodicalId":93858,"journal":{"name":"Advanced intelligent systems (Weinheim an der Bergstrasse, Germany)","volume":"6 10","pages":""},"PeriodicalIF":6.8000,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202400098","citationCount":"0","resultStr":"{\"title\":\"Hardware Implementation of Next Generation Reservoir Computing with RRAM-Based Hybrid Digital-Analog System\",\"authors\":\"Danian Dong, Woyu Zhang, Yuanlu Xie, Jinshan Yue, Kuan Ren, Hongjian Huang, Xu Zheng, Wen Xuan Sun, Jin Ru Lai, Shaoyang Fan, Hongzhou Wang, Zhaoan Yu, Zhihong Yao, Xiaoxin Xu, Dashan Shang, Ming Liu\",\"doi\":\"10.1002/aisy.202400098\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Reservoir computing (RC) possesses a simple architecture and high energy efficiency for time-series data analysis through machine learning algorithms. To date, RC has evolved into several innovative variants. The next generation reservoir computing (NGRC) variant, founded on nonlinear vector autoregression (NVAR) distinguishes itself due to its fewer hyperparameters and independence from physical random connection matrices, while yielding comparable results. However, NGRC networks struggle with massive Kronecker product calculations and matrix-vector multiplications within the read out layer, leading to substantial efficiency challenges for traditional von Neumann architectures. In this work, a hybrid digital-analog hardware system tailored for NGRC is developed. The digital part is a Kronecker product calculation unit with data filtering, which realizes transformation of nonlinear vector of the input linear vector. For matrix-vector multiplication, a computing-in-memory architecture based on resistive random access memory array offers an energy-efficient hardware solution, which markedly reduces data transfer and greatly improve computational parallelism and energy efficiency. The predictive capabilities of this hybrid NGRC system are validated through the Lorenz63 model, achieving a normalized root mean square error (NRMSE) of 0.00098 and an energy efficiency of 19.42TOPS W<sup>−1</sup>.</p>\",\"PeriodicalId\":93858,\"journal\":{\"name\":\"Advanced intelligent systems (Weinheim an der Bergstrasse, Germany)\",\"volume\":\"6 10\",\"pages\":\"\"},\"PeriodicalIF\":6.8000,\"publicationDate\":\"2024-07-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202400098\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Advanced intelligent systems (Weinheim an der Bergstrasse, Germany)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400098\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advanced intelligent systems (Weinheim an der Bergstrasse, Germany)","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400098","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
Hardware Implementation of Next Generation Reservoir Computing with RRAM-Based Hybrid Digital-Analog System
Reservoir computing (RC) possesses a simple architecture and high energy efficiency for time-series data analysis through machine learning algorithms. To date, RC has evolved into several innovative variants. The next generation reservoir computing (NGRC) variant, founded on nonlinear vector autoregression (NVAR) distinguishes itself due to its fewer hyperparameters and independence from physical random connection matrices, while yielding comparable results. However, NGRC networks struggle with massive Kronecker product calculations and matrix-vector multiplications within the read out layer, leading to substantial efficiency challenges for traditional von Neumann architectures. In this work, a hybrid digital-analog hardware system tailored for NGRC is developed. The digital part is a Kronecker product calculation unit with data filtering, which realizes transformation of nonlinear vector of the input linear vector. For matrix-vector multiplication, a computing-in-memory architecture based on resistive random access memory array offers an energy-efficient hardware solution, which markedly reduces data transfer and greatly improve computational parallelism and energy efficiency. The predictive capabilities of this hybrid NGRC system are validated through the Lorenz63 model, achieving a normalized root mean square error (NRMSE) of 0.00098 and an energy efficiency of 19.42TOPS W−1.