Unobtrusive intake actions monitoring through RGB and depth information fusion

Abstract

This paper presents a solution, based on the data fusion approach, to monitor the food and drink intake actions of elderly people during their activities of daily living. The system is non-intrusive and completely transparent to the user. The developed monitor technique is able to overcome the need of relying on direct assistance or diary-based self-monitoring. The proposed solution exploits a depth and RGB camera placed on the ceiling, in top-down view. Starting from the depth information, an adapted version of the Self-Organized Map algorithm is applied to a defined skeleton model, to track the person's movements. The RGB stream is used to recognize specific elements located on the table during eating-related activities, such as glasses. The fusion of these processed data leads to the identification of specific intake behaviours. The system performances have been successfully tested with healthy volunteers of different age and height; the results are promising and confirm the system capacity to recognize the intake activity.