EEyePaint: A Gaze Estimation-Based System for Hands-Free Painting

This paper presents EyePaint, a system that enables hands-free painting on canvas via webcam-based gaze estimation software, a custom graphical user interface (GUI), and a computer numerical control (CNC) modified for painting. The proposed system’s primary use is to assist a user with a physical disability in painting on canvas. Gaze estimation is administered using only a webcam through a unique combination of object detection algorithms, like Haar cascades, to localize facial features and learned linear regression models to translate facial features to gaze location. The custom GUI allows a user to draw a line or a circle in four different paint colors on a virtual canvas before committing their shape to a physical canvas solely through using the lowaccuracy webcam-based gaze estimation. The system and its prototype are found to have utility through basic acceptance and system testing, but the chosen gaze estimation solution is generally too low accuracy for a positive user experience. Keywords— Gaze estimation; Eye tracking; Computer numerical control; Assistive technology; Painting; Graphical user interface.