Augmented Visual Instruction for Surgical Practice and Training

This paper presents two positions about the use of augmented reality (AR) in healthcare scenarios, informed by the authors’ experience as an interdisciplinary team of academics and medical practicioners who have been researching, implementing, and validating an AR surgical telementoring system. First, AR has the potential to greatly improve the areas of surgical telementoring and of medical training on patient simulators. In austere environments, surgical telementoring that connects surgeons with remote experts can be enhanced with the use of AR annotations visualized directly in the surgeon’s field of view. Patient simulators can gain additional value for medical training by overlaying the current and future steps of procedures as AR imagery onto a physical simulator. Second, AR annotations for telementoring and for simulator-based training can be delivered either by video see-through tablet displays or by AR head-mounted displays (HMDs). The paper discusses the two AR approaches by looking at accuracy, depth perception, visualization continuity, visualization latency, and user encumbrance. Specific advantages and disadvantages to each approach mean that the choice of one display method or another must be carefully tailored to the healthcare application in which it is being used.