Food recognition using neural network on mobile device

Abstract

This paper presents a solution to the problem of real-time food detection on mobile devices. The solution compares a YOLOv4 tiny model and a multilayer classification neural network ensemble as the convolutional neural network responsible for food detection, using MobilenetV3 for classification and several YOLOv4 tiny models capable of detecting less food in the overall problem. The model was trained with 40 different food categories using an RTX 3070 video card. Results showed that the multilayer network ensemble performed better than a single YOLOv4 tiny model. The TP value was 258 higher, the FP value was 163 lower and the FN value was 773 lower. The accuracy increased from 0,56 to 0,61, the recall increased from 0,52 to 0,70 and the F1-score also increased from 0,54 to 0,65. Finally, the finished models were implemented on an Android mobile device, and the application allows users to log meals and generate statistics from the saved data.