µ-FF: On-Device Forward-Forward Training Algorithm for Microcontrollers

Abstract

Deliver intelligence into low-cost hardware e.g., Microcontroller Units (MCUs) for the realization of low-power tailored applications nowadays is an emerging research area. However, the training of deep learning models on embedded systems is still challenging mainly due to their low amount of memory, available energy, and computing power which significantly limit the complexity of the tasks that can be executed, thus making impossible use of traditional training algorithms such as backpropagation (BP). During these years techniques such as weights compression and quantization have emerged as solutions, but they only address the inference phase. Forward-Forward (FF) is a novel training algorithm that has been recently proposed as a possible alternative to BP when the available resources are limited. This is achieved by training the layers of a neural network separately, thus reducing the required energy and memory. In this paper, we propose µ-FF, a variation of the original FF which tackles the training process with a multivariate Ridge regression approach and allows to find closed-form solution by using the Mean Squared Error (MSE) as loss function. Such an approach does not use BP and does not need to compute gradients, thus saving memory and computing resources to enable the on-device training directly on MCUs of the STM32 family. Experimental results conducted on the Fashion-MNIST dataset demonstrate the effectiveness of the proposed approach in terms of memory and accuracy.