LTNN: An energy-efficient machine learning accelerator on 3D CMOS-RRAM for layer-wise tensorized neural network

Abstract

An energy efficient machine learning requires an effective construction of neural network during training. This paper introduces a tensorized formulation of neural network during training such that weight matrix can be significantly compressed. The tensorized neural network can be further naturally mapped to a 3D CMOS-RRAM based accelerator with significant bandwidth boosting from vertical I/O connections. As such, high throughput and low power can be achieved simultaneously. Simulation results using the benchmark MNIST show that the proposed accelerator has 1.294x speed-up, 2.393x energy-efficiency and 7.59 x area saving compared to 3D CMOS-ASIC implementation. Moreover, our proposed accelerator can achieve 370.64 GOPS throughput and 1055.95 GOPS/W energy efficiency, which is equivalent to 7.661 TOPS/W for uncompressed neural network. In addition, 142x model compression can be achieved by tensorization with acceptable accuracy loss.