Autonomous narration of humanoid robot kitchen task experience

Abstract

The progress in humanoid robotics research has led to robots that are able to perform complex tasks with a certain level of autonomy by integrating perception, action, planning, and learning capabilities. However, robot capabilities are still limited in regard to how they externalize their internal state and world state, i.e. their sensorimotor experience, and how they explain which tasks they performed and how they performed these tasks. In other words, their capability in conveying information to the user in a way similar to what humans do is limited. To this end, we present a verbalization system that generates natural language explanations of the robot's past navigation and manipulation experience. We propose a threelayered model to represent robot experience which doubles as a retrievable episodic memory. Through the memory system, the robot can select a matching experience given a user query. In order to generate flexible narrations, we use verbalization parameters to capture user preferences. We show that our verbalization algorithm is capable of producing appropriate results based on these verbalization parameters. The proposed verbalization system is able to generate explanations for navigation as well as grasping and manipulation tasks. The resulting system is evaluated in a pick-and-place kitchen scenario.