Hardware-efficient learning with feedforward inhibition

Abstract

On-chip learning and classification have a broad impact on many applications. Yet their hardware implementation is still limited by the scale of computation, as well as practical issues of device fabrication, variability and reliability. Inspired by micro neural-circuits in the cortical system, this work develops a novel solution that efficiently reduces the network size and improves the learning accuracy. The building block is the motif of feedforward inhibition that effectively separates main features and the residual in sparse feature extraction. Other learning rules follow the spike-rate-dependent-plasticity (SRDP). As demonstrated in handwriting recognition, such a bio-plausible solution is able to achieve >95% accuracy, comparable to the sparse coding algorithms; in addition, SRDP, instead of gradient based back propagation, is able to save the computation time by >50X. The utilization of the inhibition motif reduces the network size by >3X at the same accuracy, illustrating its potential in hardware efficiency.