Data-Driven Digital Twins in Surgery utilizing Augmented Reality and Machine Learning

Abstract

On the one hand, laparoscopic surgery as medical state-of-the-art method is minimal invasive, and thus less stressful for patients. On the other hand, laparoscopy implies higher demands on physicians, such as mental load or preparation time, hence appropriate technical support is essential for quality and suc-cess. Medical Digital Twins provide an integrated and virtual representation of patients' and organs' data, and thus a generic concept to make complex information accessible by surgeons. In this way, minimal invasive surgery could be improved significantly, but requires also a much more complex software system to achieve the various resulting requirements. The biggest challenges for these systems are the safe and precise mapping of the digital twin to reality, i.e. dealing with deformations, movement and distortions, as well as balance out the competing requirement for intuitive and immersive user access and security. The case study ARAILIS is presented as a proof in concept for such a system and provides a starting point for further research. Based on the insights delivered by this prototype, a vision for future Medical Digital Twins in surgery is derived and discussed.