Analyzing Human Behavioral Data to Interact with Restaurant Server Agents

In this paper, we consider a problem of analyzing human behavioral data to predict the human cognitive states and generate corresponding actions of sever-agent. Specifically, we aim at predicting human cognitive states during meal time and generating relevant dining services for the human. For this study, we collect behavioral data using 2 kinds of wearable devices, which are an eye tracker and a watch type EDA device, during meal time. We focus on the characteristics of the behavioral data, which are heterogeneous, noisy and temporal, and suggest a novel machine learning algorithm which can analyze the data integrally. Suggested model has hierarchical structure: the bottom layer combines the multi-modal behavioral data based on causal structure of the data and extracts the feature vector. Using the extracted feature vectors, the upper layer predicts the cognitive states based on temporal correlation between feature vectors. Experimental results show that the suggested model can analyze the behavioral data efficiently and predict the human cognitive states correctly.