Learning Soft-Tissue Simulation from Models and Observation

Abstract

Accurate soft-tissue simulation using biomechanical models is computationally expensive. This is unfortunate because accurate biomechanical models could model tool-tissue interaction during surgical procedures, thereby providing intra-operative guidance to surgeons. In this work, we present steps toward interactive soft-tissue simulation for specific models using a learning-based framework that learns from finite element method (FEM) simulations. We train a graph neural network that takes the position and velocity of a tracked tool as input and estimates the deformations of a base mesh at each time step. By using data augmentation, the network learns to self-correct for errors in estimation to maintain the stability of the simulation over time. This approach estimates soft tissue deformation with less than 1 mm mean error with respect to FEM simulation over an interaction sequence of 80 s. This error magnitude is within the accuracy of FEM in comparing to the in situ camera observations of the interaction. While the FEM took 15 h to simulate 80 s of interaction, the network-based simulator took 47 s. Despite several open challenges that will be the subject of our future work, this learning-based framework constitutes a step towards real-time biomechanics-based simulation for intraoperative surgical guidance.