Robot Plan Model Generation and Execution with Natural Language Interface*

Verbal interaction between a human and a robot may play a key role in conveying suitable directions for a robot to achieve the goal of a user’s request. However, a robot may need to correct task plans or make new decisions with human help, which would make the interaction inconvenient and also increase the interaction time. In this paper, we propose a new verbal interaction-based method that can generate plan models and execute proper actions without human involvement in the middle of performing a task by a robot. To understand the verbal behaviors of humans when giving instructions to a robot, we first conducted a brief user study and found that a human user does not explicitly express the required task. To handle such unclear instructions by a human, we propose two different algorithms that can generate a component of new plan models based on intents and entities parsed from natural language and can resolve the unclear entities existed in human instructions. An experimental scenario with a robot, Cozmo, was tried in the lab environment to test whether or not the proposed method could generate an appropriate plan model. As a result, we found that the robot could successfully accomplish the task following human instructions and also found that the number of interactions and components in the plan model could be reduced as opposed to the general reactive plan model. In the future, we are going to improve the automated process of generating plan models and apply various scenarios under different service environments and robots.