A framework of personal assistant for computer users by analyzing video stream

Abstract

The engagement time on the computer is increasing steadily with the rapid development of the Internet. During the long period in front of the computer, bad postures and habits will result in some health risks, and the unawareness of fatigue will impair the work efficiency. We investigate how users behave in front of the computer with a camera. Face pose, eye gaze, eye blinking, and yawn frequency are considered. These visual cues are then used to give suggestions to users for correcting wrong posture and indicating the need for a break. We propose a novel framework of personal assistant for a user when he uses computer for a long time. The camera produces the video stream which records the user behavior. The automatically assistant system will analyze the visual inputs and give suggestions at the right time. Our experiment shows that it achieves high accuracy of detecting visual cues, and makes reasonable suggestions to users. The work initializes the area of assistant system for individuals who use computer frequently.