Fusion of facial expression and eye-gaze for behavior profiling and decision support

Abstract

This paper describes a method and system for monitoring and correlating digitized non-verbal data (like emotions and eye-gaze) with other kinds of data for decision support and information personalization. This is illustrated through two applications in this paper. The first one describes detection, monitoring and correlation of emotional state changes for behavior profiling of sales candidates. The second one outlines design of eye gaze tracking system using standard web camera (instead of more expensive infra-red camera) for holiday destination selection. The paper makes unique contributions in terms of modelling as well as correlation of non-verbal data in natural environments for improving digital information quality and decision making in business areas like recruitment and tourism.