A Spatio-Temporal-Semantic Environment Representation for Autonomous Mobile Robots equipped with various Sensor Systems

Abstract

The large amount of high resolution sensor data, both temporal and spatial, that autonomous mobile robots collect in today’s systems requires structured and efficient management and storage during the robot mission. In response, we present SEEREP: A Spatio-Temporal-Semantic Environment Representation for Autonomous Mobile Robots. SEEREP handles various types of data at once and provides an efficient query interface for all three modalities that can be combined for high-level analyses. It supports common robotic sensor data types such as images and point clouds, as well as sensor and robot coordinate frames changing over time. Furthermore, SEEREP provides an efficient HDF5-based storage system running on the robot during operation, compatible with ROS and the corresponding sensor message definitions. The compressed HDF5 data backend can be transferred efficiently to an application server with a running SEEREP query server providing gRPC interfaces with Protobuf and Flattbuffer message types. The query server can support high-level planning and reasoning systems in e.g. agricultural environments, or other partially unstructured environments that change over time. In this paper we show that SEEREP is much better suited for these tasks than a traditional GIS, which cannot handle the different types of robotic sensor data.