Differentiable simulation

Abstract

Differentiable simulation is emerging as a fundamental building block for many cutting-edge applications in computer graphics, vision and robotics, among others. This course provides an introduction to this topic and an overview of state-of-the-art methods in this context. Starting with the basics of dynamic mechanical systems, we will present a general theoretical framework for differentiable simulation, which we will specialize to rigid bodies, deformable solids, and fluids. A particular focus will be on the different alternatives for computing simulation derivatives, ranging from analytical expressions via sensitivity analysis to reverse-mode automatic differentiation. As an important step towards real-world applications, we also present extensions to non-smooth phenomena such as frictional contact. Finally, we will discuss different ways of integrating differentiable simulation into machine learning frameworks. The material covered in this course is based on the author's own works and experience, complemented by a state-of-the-art review of this young but rapidly evolving field. It will be richly illustrated, annotated, and supported by examples ranging from robotic manipulation of deformable materials to simulation-based capture of dynamic fluids. The theoretical parts will be accompanied by source code examples that will be made available to participants prior to this course.