Mitigating the Impact of ReRAM I-V Nonlinearity and IR Drop via Fast Offline Network Training

Abstract

ReRAM crossbar arrays (RCAs) have the potential to provide extremely high efficiency for accelerating deep neural networks (DNNs). However, one crucial challenge for RCA-based DNN accelerators is functional inaccuracy due to nonidealities present in RCA hardware. While nonideality-aware training (NAT) could be used to mitigate the effect of nonidealities, with currently available methods it would take months to train even a medium size convolutional neural network (CNN). In this article we propose a nonideality prediction method that enables very fast training of RCA-based neural networks, and show its feasibility through NAT of DNNs. Our key ideas include 1) weight-centric nonideality modeling and 2) data-dependence elimination by tailored input randomization. Our experimental results using a multilayer perceptron and CNNs demonstrate that our method is very fast ( $100\sim 15$ $000\times $ faster training speed) while achieving much better-crossbar-level accuracy ( $2 \sim 90\times $ lower-RMS error) and post-retraining validated accuracy than previous methods.