Occlusion-aware Hand Posture Based Interaction on Tabletop Projector

Abstract

Conventional camera-based hand interaction technique suffered from self-occlusion among fingers, which lowers the detection accuracy of fingertip positions, leading to uncomfortable UI controls. Based on observations, self-occlusion depends on hand postures. We design an interaction framework in which interaction is decided in response to a recognized hand posture. Using a tabletop projection system that has a projector and a depth sensor, we implement the framework by integrating five touch and in-air interactions that balance its stability and usability.