具有自主学习功能的形位感知融合电子皮肤,用于手势交互

IF 7.3 1区 工程技术 Q1 INSTRUMENTS & INSTRUMENTATION Microsystems & Nanoengineering Pub Date : 2024-07-22 DOI:10.1038/s41378-024-00739-9
Qian Wang, Mingming Li, Pingping Guo, Liang Gao, Ling Weng, Wenmei Huang
{"title":"具有自主学习功能的形位感知融合电子皮肤,用于手势交互","authors":"Qian Wang, Mingming Li, Pingping Guo, Liang Gao, Ling Weng, Wenmei Huang","doi":"10.1038/s41378-024-00739-9","DOIUrl":null,"url":null,"abstract":"<p>Wearable devices, such as data gloves and electronic skins, can perceive human instructions, behaviors and even emotions by tracking a hand's motion, with the help of knowledge learning. The shape or position single-mode sensor in such devices often lacks comprehensive information to perceive interactive gestures. Meanwhile, the limited computing power of wearable applications restricts the multimode fusion of different sensing data and the deployment of deep learning networks. We propose a perceptive fusion electronic skin (PFES) with a bioinspired hierarchical structure that utilizes the magnetization state of a magnetostrictive alloy film to be sensitive to external strain or magnetic field. Installed at the joints of a hand, the PFES realizes perception of curvature (joint shape) and magnetism (joint position) information by mapping corresponding signals to the two-directional continuous distribution such that the two edges represent the contributions of curvature radius and magnetic field, respectively. By autonomously selecting knowledge closer to the user's hand movement characteristics, the reinforced knowledge distillation method is developed to learn and compress a teacher model for rapid deployment on wearable devices. The PFES integrating the autonomous learning algorithm can fuse curvature-magnetism dual information, ultimately achieving human machine interaction with gesture recognition and haptic feedback for cross-space perception and manipulation.</p><figure></figure>","PeriodicalId":18560,"journal":{"name":"Microsystems & Nanoengineering","volume":"42 1","pages":""},"PeriodicalIF":7.3000,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Shape-position perceptive fusion electronic skin with autonomous learning for gesture interaction\",\"authors\":\"Qian Wang, Mingming Li, Pingping Guo, Liang Gao, Ling Weng, Wenmei Huang\",\"doi\":\"10.1038/s41378-024-00739-9\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Wearable devices, such as data gloves and electronic skins, can perceive human instructions, behaviors and even emotions by tracking a hand's motion, with the help of knowledge learning. The shape or position single-mode sensor in such devices often lacks comprehensive information to perceive interactive gestures. Meanwhile, the limited computing power of wearable applications restricts the multimode fusion of different sensing data and the deployment of deep learning networks. We propose a perceptive fusion electronic skin (PFES) with a bioinspired hierarchical structure that utilizes the magnetization state of a magnetostrictive alloy film to be sensitive to external strain or magnetic field. Installed at the joints of a hand, the PFES realizes perception of curvature (joint shape) and magnetism (joint position) information by mapping corresponding signals to the two-directional continuous distribution such that the two edges represent the contributions of curvature radius and magnetic field, respectively. By autonomously selecting knowledge closer to the user's hand movement characteristics, the reinforced knowledge distillation method is developed to learn and compress a teacher model for rapid deployment on wearable devices. The PFES integrating the autonomous learning algorithm can fuse curvature-magnetism dual information, ultimately achieving human machine interaction with gesture recognition and haptic feedback for cross-space perception and manipulation.</p><figure></figure>\",\"PeriodicalId\":18560,\"journal\":{\"name\":\"Microsystems & Nanoengineering\",\"volume\":\"42 1\",\"pages\":\"\"},\"PeriodicalIF\":7.3000,\"publicationDate\":\"2024-07-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Microsystems & Nanoengineering\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://doi.org/10.1038/s41378-024-00739-9\",\"RegionNum\":1,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"INSTRUMENTS & INSTRUMENTATION\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Microsystems & Nanoengineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1038/s41378-024-00739-9","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"INSTRUMENTS & INSTRUMENTATION","Score":null,"Total":0}
引用次数: 0

摘要

数据手套和电子皮肤等可穿戴设备可以借助知识学习,通过跟踪手的运动来感知人类的指令、行为甚至情绪。这类设备中的形状或位置单模传感器往往缺乏感知交互手势的全面信息。同时,可穿戴应用的计算能力有限,限制了不同传感数据的多模融合和深度学习网络的部署。我们提出了一种具有生物启发分层结构的感知融合电子皮肤(PFES),它利用磁致伸缩合金薄膜的磁化状态对外部应变或磁场敏感。PFES 安装在手部关节处,通过将相应信号映射到双向连续分布上,使两个边缘分别代表曲率半径和磁场的贡献,从而实现对曲率(关节形状)和磁性(关节位置)信息的感知。通过自主选择更贴近用户手部运动特征的知识,开发了强化知识提炼方法,以学习和压缩教师模型,从而快速部署到可穿戴设备上。集成了自主学习算法的 PFES 可以融合曲率和磁场双重信息,最终实现具有手势识别和触觉反馈功能的人机交互,实现跨空间感知和操作。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

摘要图片

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Shape-position perceptive fusion electronic skin with autonomous learning for gesture interaction

Wearable devices, such as data gloves and electronic skins, can perceive human instructions, behaviors and even emotions by tracking a hand's motion, with the help of knowledge learning. The shape or position single-mode sensor in such devices often lacks comprehensive information to perceive interactive gestures. Meanwhile, the limited computing power of wearable applications restricts the multimode fusion of different sensing data and the deployment of deep learning networks. We propose a perceptive fusion electronic skin (PFES) with a bioinspired hierarchical structure that utilizes the magnetization state of a magnetostrictive alloy film to be sensitive to external strain or magnetic field. Installed at the joints of a hand, the PFES realizes perception of curvature (joint shape) and magnetism (joint position) information by mapping corresponding signals to the two-directional continuous distribution such that the two edges represent the contributions of curvature radius and magnetic field, respectively. By autonomously selecting knowledge closer to the user's hand movement characteristics, the reinforced knowledge distillation method is developed to learn and compress a teacher model for rapid deployment on wearable devices. The PFES integrating the autonomous learning algorithm can fuse curvature-magnetism dual information, ultimately achieving human machine interaction with gesture recognition and haptic feedback for cross-space perception and manipulation.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Microsystems & Nanoengineering
Microsystems & Nanoengineering Materials Science-Materials Science (miscellaneous)
CiteScore
12.00
自引率
3.80%
发文量
123
审稿时长
20 weeks
期刊介绍: Microsystems & Nanoengineering is a comprehensive online journal that focuses on the field of Micro and Nano Electro Mechanical Systems (MEMS and NEMS). It provides a platform for researchers to share their original research findings and review articles in this area. The journal covers a wide range of topics, from fundamental research to practical applications. Published by Springer Nature, in collaboration with the Aerospace Information Research Institute, Chinese Academy of Sciences, and with the support of the State Key Laboratory of Transducer Technology, it is an esteemed publication in the field. As an open access journal, it offers free access to its content, allowing readers from around the world to benefit from the latest developments in MEMS and NEMS.
期刊最新文献
Single-cell electro-mechanical shear flow deformability cytometry. Automating life science labs at the single-cell level through precise ultrasonic liquid sample ejection: PULSE. Bifunctional nanoprobe for simultaneous detection of intracellular reactive oxygen species and temperature in single cells. Sound innovations for biofabrication and tissue engineering. A novel gyroscope based on the slow surface acoustic wave in a phononic metamaterial.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1