Learning new concepts, such as object models, from humanrobot interactions entails different recognition capabilities on a robotic platform. This work proposes a hierarchical approach to address the extra challenges from natural interaction scenarios by exploiting multimodal data. First, a speech-guided recognition of the type of interaction happening is presented. This first step facilitates the following segmentation of relevant visual information to learn the target object model. Our approach includes three complementary strategies to find Regions of Interest (RoI) depending on the interaction type: Point, Show or Speak. We run an exhaustive validation of the proposed strategies using the recently published Multimodal Human-Robot Interaction dataset [1]. The currently presented pipeline is built on the pipeline proposed with the dataset and provides a more complete baseline for target object segmentation on all its recordings.
从人机交互中学习新概念，如对象模型，需要在机器人平台上具有不同的识别能力。这项工作提出了一种分层方法，通过利用多模态数据来解决自然交互场景带来的额外挑战。首先，提出了一种语音引导的交互类型识别方法。这第一步便于后续分割相关的视觉信息来学习目标对象模型。我们的方法包括三种互补的策略来根据交互类型找到感兴趣的区域(RoI): Point, Show或Speak。我们使用最近发布的多模式人机交互数据集[1]对所提出的策略进行了详尽的验证。目前提出的管道是建立在数据集提出的管道基础上的，并为其所有记录的目标对象分割提供了更完整的基线。
{"title":"Finding Regions of Interest from Multimodal Human-Robot Interactions","authors":"P. Azagra, Javier Civera, A. C. Murillo","doi":"10.21437/GLU.2017-15","DOIUrl":"https://doi.org/10.21437/GLU.2017-15","url":null,"abstract":"Learning new concepts, such as object models, from humanrobot interactions entails different recognition capabilities on a robotic platform. This work proposes a hierarchical approach to address the extra challenges from natural interaction scenarios by exploiting multimodal data. First, a speech-guided recognition of the type of interaction happening is presented. This first step facilitates the following segmentation of relevant visual information to learn the target object model. Our approach includes three complementary strategies to find Regions of Interest (RoI) depending on the interaction type: Point, Show or Speak. We run an exhaustive validation of the proposed strategies using the recently published Multimodal Human-Robot Interaction dataset [1]. The currently presented pipeline is built on the pipeline proposed with the dataset and provides a more complete baseline for target object segmentation on all its recordings.","PeriodicalId":273075,"journal":{"name":"GLU 2017 International Workshop on Grounding Language Understanding","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126321789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：