Food categories classification and Ingredients estimation using CNNs on Raspberry Pi 3

Abstract

Foods are important things to human lives, especially for elderly or diabetics. Tradition nutrition book is not the effective way for people to use and not cover all kind of foods. Most of the food nutrition in the book focused on Western dishes not Asian dishes. This research proposed the new way to categorized Thai fast food dishes, classified and localized the ingredients in each dish. Convolutional Neural Networks (CNNs) are used to achieve these tasks. MobileNet is used as food categorizer while You Only Look Once (YOLO) network works as the ingredients classifier and localizer. Then, ingredients in the pictures are cropped and passed through traditional image processing to calculate area and compared with real ingredient’s dimension. Non-uniform shape ingredients are segmented, then, the nutrition of the dish can be calculated. Finally, the networks are transferred in to Raspberry Pi 3 platform to simulate limited resources and calculation power platform likes in a mobile phone. The networks in Raspberry Pi 3 produce good prediction accuracy but slow speed. PeachPy is introduced to speed up the network and it can run at 3.3 seconds per food image.