DASH: Modularized Human Manipulation Simulation with Vision and Language for Embodied AI

Yifeng Jiang, Michelle Guo, Jiangshan Li, Ioannis Exarchos, Jiajun Wu, C. Liu
{"title":"DASH: Modularized Human Manipulation Simulation with Vision and Language for Embodied AI","authors":"Yifeng Jiang, Michelle Guo, Jiangshan Li, Ioannis Exarchos, Jiajun Wu, C. Liu","doi":"10.1145/3475946.3480950","DOIUrl":null,"url":null,"abstract":"Creating virtual humans with embodied, human-like perceptual and actuation constraints has the promise to provide an integrated simulation platform for many scientific and engineering applications. We present Dynamic and Autonomous Simulated Human (DASH), an embodied virtual human that, given natural language commands, performs grasp-and-stack tasks in a physically-simulated cluttered environment solely using its own visual perception, proprioception, and touch, without requiring human motion data. By factoring the DASH system into a vision module, a language module, and manipulation modules of two skill categories, we can mix and match analytical and machine learning techniques for different modules so that DASH is able to not only perform randomly arranged tasks with a high success rate, but also do so under anthropomorphic constraints and with fluid and diverse motions. The modular design also favors analysis and extensibility to more complex manipulation skills.","PeriodicalId":300353,"journal":{"name":"The ACM SIGGRAPH / Eurographics Symposium on Computer Animation","volume":"40 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"The ACM SIGGRAPH / Eurographics Symposium on Computer Animation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3475946.3480950","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2

Abstract

Creating virtual humans with embodied, human-like perceptual and actuation constraints has the promise to provide an integrated simulation platform for many scientific and engineering applications. We present Dynamic and Autonomous Simulated Human (DASH), an embodied virtual human that, given natural language commands, performs grasp-and-stack tasks in a physically-simulated cluttered environment solely using its own visual perception, proprioception, and touch, without requiring human motion data. By factoring the DASH system into a vision module, a language module, and manipulation modules of two skill categories, we can mix and match analytical and machine learning techniques for different modules so that DASH is able to not only perform randomly arranged tasks with a high success rate, but also do so under anthropomorphic constraints and with fluid and diverse motions. The modular design also favors analysis and extensibility to more complex manipulation skills.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
DASH:具有视觉和语言的模块化人体操作仿真,用于嵌入式AI
创建具有具体的、类人的感知和驱动约束的虚拟人有望为许多科学和工程应用提供一个集成的仿真平台。我们介绍了动态和自主模拟人类(DASH),这是一种具体的虚拟人类,在给定自然语言命令的情况下,仅使用自己的视觉感知、本体感觉和触觉,在物理模拟的混乱环境中执行抓取和堆叠任务,而不需要人类运动数据。通过将DASH系统分解为视觉模块、语言模块和操作两个技能类别的模块,我们可以混合匹配不同模块的分析和机器学习技术,使DASH不仅能够以高成功率执行随机安排的任务,而且能够在拟人化约束下以流体和多样化的动作完成任务。模块化设计还有利于分析和扩展到更复杂的操作技能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Two-Stage Motion Editing Interface for Character Animation Facial Expression Transfer from Video Via Deep Learning DASH: Modularized Human Manipulation Simulation with Vision and Language for Embodied AI
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1