Use of Spatial Reference Systems in Interactive Object Recognition

We are developing a helper robot that carries out tasks ordered by users through speech. The robot needs a vision system to recognize the objects appearing in the orders. It is, however, difficult to realize vision systems that can work in various conditions. They may find many objects and cannot determine which is the target. We have proposed a method of using a conversation with the user to solve this problem. The robot asks questions which the user can easily answer and whose answer can efficiently reduce the number of candidate objects. In our previous system, however, we assumed that there was no occlusion among objects. This paper presents an extended system that can detect target objects in occlusion cases. It is difficult to obtain some features in occlusion cases. To compensate the system for this shortage of features, we propose to use reference systems to express the positional relationships of objects. Experimental results show that the robot can efficiently detect objects through user-friendly conversation.