Enhanced robot state estimation using physics-informed neural networks and multimodal proprioceptive data

Abstract

In this study, we introduce an innovative Robot State Estimation (RSE) methodology incorporating a learning-based contact estimation framework for legged robots, which obviates the need for external physical contact sensors. This approach integrates multimodal proprioceptive sensory data, employing a Physics-Informed Neural Network (PINN) in conjunction with an Unscented Kalman Filter (UKF) to enhance the state estimation process. The primary objective of this RSE technique is to calibrate the Inertial Measurement Unit (IMU) effectively and furnish a detailed representation of the robot’s dynamic state. Our methodology exploits the PINN to mitigate IMU drift issues by imposing constraints on the loss function via Ordinary Differential Equations (ODEs). The advantages of utilizing a contact estimator based on proprioceptive sensory data are multifold. Unlike vision-based state estimators, our proprioceptive approach is immune to visual impairments such as obscured or ambiguous environments. Moreover, it circumvents the necessity for dedicated contact sensors—components not universally present on robotic platforms and challenging to integrate without substantial hardware modifications. The contact estimator within our network is trained to discern contact events across various terrains, thereby facilitating resilient proprioceptive odometry. This enables the contact-aided invariant Kalman Filter to produce precise odometric trajectories. Subsequently, the UKF algorithm estimates the robot’s three-dimensional attitude, velocity, and position. Experimental validation of our proposed PINN-based method illustrates its capacity to assimilate data from multiple sensors, effectively reducing the influence of sensor biases by enforcing ODE constraints, all while preserving intrinsic sensor characteristics. When juxtaposed with the employment of the UKF algorithm in isolation, our integrated RSE model demonstrates superior performance in state estimation. This enhanced capability automatically reduces sensor drift impacts during operational deployment, rendering our proposed solution applicable to real-world scenarios.