Motion Capture of Hand Movements Using Stereo Vision for Minimally Invasive Vascular Interventions

Abstract

A virtual reality (VR) based training system for Minimally Invasive Vascular Surgery can be a very useful training tool for improving skills and reducing errors in operation. Computer vision techniques have the potential to be incorporated into a VR based training system for developing low cost, high accuracy and flexible systems in this area. In this paper, we present an interactive 3D training system that uses stereo vision to capture hand movements as the input operations for the system. The standard operations i.e. pushing, pulling and twisting are captured with stereo vision based on the improved Camshift tracking algorithm and parallel alignment model theory to acquire hand gestures information. We present a new approach to calculate virtual pushing/pulling force and turning angle as extra inputs for understanding these essential operations. In addition, an algorithm that enables the simulator to model guide wire and catheter insertions realistically is presented through these basic actions. The experiment results demonstrate that stereo vision based training system is useful and effective for simulating guide wire insertion procedures with low system cost and flexible operations.