The C-CNN model: Do we really need multiplicative synapses in convolutional neural networks?

Abstract

Comparative synapses are proposed and investigated in the context of convolutional neural networks as replacements for the traditional, multiplier-based synapses. A comparative synapse is an operator inspired from the min() operator used in fuzzy-logic a replacement for product to implement AND function. Its implementation complexity is linear in the number of bits unlike multipliers, requiring quadratic complexity. In effect, using a typical resolution of 8 bits the use of comparative synapse would reduce 8 times the number of hardware resources allocated for the operator. A C-CNN model was constructed to support comparative synapses and their update and error propagation rules. GPU acceleration of the C-CNN model was achieved using CUPY. The model was trained with several widely known image recognition datasets including MNIST, CIFAR and USPS. It turns out that functional performance (accuracy) is not dramatically affected in C-CNN against a similar traditional CNN model with multiplicative operators, thus opening an interesting implementation perspective, particularly for the TinyML and HW-oriented solutions with significant reduction in energy, silicon area and costs. The approach is scalable to more sophisticated CNN models providing adequate optimized operators adapted to this new synaptic model.