首页 > 最新文献

Science Robotics最新文献

英文 中文
Biohybrid hand actuated by multiple human muscle tissues
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-02-12
Xinzhu Ren, Yuya Morimoto, Shoji Takeuchi
Cultured muscle tissue serves as a power source in biohybrid robots that demonstrate diverse motions. However, current designs typically only drive simple substrates on a small scale, limiting flexibility and controllability. To address this, we proposed a biohybrid hand with multijointed fingers powered by multiple muscle tissue actuators (MuMuTAs), bundles of thin muscle tissues. The MuMuTA can provide linear actuation with high contractile force (~8 millinewtons) and high contractile length (~4 millimeters), which can be converted into the flexion of multijointed fingers by a cable-driven mechanism. We successfully powered the biohybrid hand achieving individual control of fingers and a variety of motions using different signaling controls. This study showcases the potential of MuMuTAs as a driving source for advanced biohybrid robotics.
{"title":"Biohybrid hand actuated by multiple human muscle tissues","authors":"Xinzhu Ren,&nbsp;Yuya Morimoto,&nbsp;Shoji Takeuchi","doi":"","DOIUrl":"","url":null,"abstract":"<div >Cultured muscle tissue serves as a power source in biohybrid robots that demonstrate diverse motions. However, current designs typically only drive simple substrates on a small scale, limiting flexibility and controllability. To address this, we proposed a biohybrid hand with multijointed fingers powered by multiple muscle tissue actuators (MuMuTAs), bundles of thin muscle tissues. The MuMuTA can provide linear actuation with high contractile force (~8 millinewtons) and high contractile length (~4 millimeters), which can be converted into the flexion of multijointed fingers by a cable-driven mechanism. We successfully powered the biohybrid hand achieving individual control of fingers and a variety of motions using different signaling controls. This study showcases the potential of MuMuTAs as a driving source for advanced biohybrid robotics.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 99","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143397445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Bioinspired design of a tissue-engineered ray with machine learning
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-02-12
John F. Zimmerman, Daniel J. Drennan, James Ikeda, Qianru Jin, Herdeline Ann M. Ardoña, Sean L. Kim, Ryoma Ishii, Kevin Kit Parker
In biomimetic design, researchers recreate existing biological structures to form functional devices. For biohybrid robotic swimmers assembled with tissue engineering, this is problematic because most devices operate at different length scales than their naturally occurring counterparts, resulting in reduced performance. To overcome these challenges, here, we demonstrate how machine learning–directed optimization (ML-DO) can be used to inform the design of a biohybrid robot, outperforming other nonlinear optimization techniques, such as Bayesian optimization, in the selection of high-performance geometries. We show how this approach can be used to maximize the thrust generated by a tissue-engineered mobuliform miniray. This results in devices that can swim at the millimeter scale while more closely preserving natural locomotive scaling laws. Overall, this work provides a quantitatively rigorous approach for the engineering design of muscular structure-function relationships in an automated fashion.
{"title":"Bioinspired design of a tissue-engineered ray with machine learning","authors":"John F. Zimmerman,&nbsp;Daniel J. Drennan,&nbsp;James Ikeda,&nbsp;Qianru Jin,&nbsp;Herdeline Ann M. Ardoña,&nbsp;Sean L. Kim,&nbsp;Ryoma Ishii,&nbsp;Kevin Kit Parker","doi":"","DOIUrl":"","url":null,"abstract":"<div >In biomimetic design, researchers recreate existing biological structures to form functional devices. For biohybrid robotic swimmers assembled with tissue engineering, this is problematic because most devices operate at different length scales than their naturally occurring counterparts, resulting in reduced performance. To overcome these challenges, here, we demonstrate how machine learning–directed optimization (ML-DO) can be used to inform the design of a biohybrid robot, outperforming other nonlinear optimization techniques, such as Bayesian optimization, in the selection of high-performance geometries. We show how this approach can be used to maximize the thrust generated by a tissue-engineered mobuliform miniray. This results in devices that can swim at the millimeter scale while more closely preserving natural locomotive scaling laws. Overall, this work provides a quantitatively rigorous approach for the engineering design of muscular structure-function relationships in an automated fashion.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 99","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143397453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Biohybrid hand actuated by multiple human muscle tissues
IF 25 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-02-12 DOI: 10.1126/scirobotics.adr5512
Xinzhu Ren, Yuya Morimoto, Shoji Takeuchi
Cultured muscle tissue serves as a power source in biohybrid robots that demonstrate diverse motions. However, current designs typically only drive simple substrates on a small scale, limiting flexibility and controllability. To address this, we proposed a biohybrid hand with multijointed fingers powered by multiple muscle tissue actuators (MuMuTAs), bundles of thin muscle tissues. The MuMuTA can provide linear actuation with high contractile force (~8 millinewtons) and high contractile length (~4 millimeters), which can be converted into the flexion of multijointed fingers by a cable-driven mechanism. We successfully powered the biohybrid hand achieving individual control of fingers and a variety of motions using different signaling controls. This study showcases the potential of MuMuTAs as a driving source for advanced biohybrid robotics.
{"title":"Biohybrid hand actuated by multiple human muscle tissues","authors":"Xinzhu Ren, Yuya Morimoto, Shoji Takeuchi","doi":"10.1126/scirobotics.adr5512","DOIUrl":"https://doi.org/10.1126/scirobotics.adr5512","url":null,"abstract":"Cultured muscle tissue serves as a power source in biohybrid robots that demonstrate diverse motions. However, current designs typically only drive simple substrates on a small scale, limiting flexibility and controllability. To address this, we proposed a biohybrid hand with multijointed fingers powered by multiple muscle tissue actuators (MuMuTAs), bundles of thin muscle tissues. The MuMuTA can provide linear actuation with high contractile force (~8 millinewtons) and high contractile length (~4 millimeters), which can be converted into the flexion of multijointed fingers by a cable-driven mechanism. We successfully powered the biohybrid hand achieving individual control of fingers and a variety of motions using different signaling controls. This study showcases the potential of MuMuTAs as a driving source for advanced biohybrid robotics.","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"55 1","pages":""},"PeriodicalIF":25.0,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143393477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Bioinspired design of a tissue-engineered ray with machine learning
IF 25 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-02-12 DOI: 10.1126/scirobotics.adr6472
John F. Zimmerman, Daniel J. Drennan, James Ikeda, Qianru Jin, Herdeline Ann M. Ardoña, Sean L. Kim, Ryoma Ishii, Kevin Kit Parker
In biomimetic design, researchers recreate existing biological structures to form functional devices. For biohybrid robotic swimmers assembled with tissue engineering, this is problematic because most devices operate at different length scales than their naturally occurring counterparts, resulting in reduced performance. To overcome these challenges, here, we demonstrate how machine learning–directed optimization (ML-DO) can be used to inform the design of a biohybrid robot, outperforming other nonlinear optimization techniques, such as Bayesian optimization, in the selection of high-performance geometries. We show how this approach can be used to maximize the thrust generated by a tissue-engineered mobuliform miniray. This results in devices that can swim at the millimeter scale while more closely preserving natural locomotive scaling laws. Overall, this work provides a quantitatively rigorous approach for the engineering design of muscular structure-function relationships in an automated fashion.
{"title":"Bioinspired design of a tissue-engineered ray with machine learning","authors":"John F. Zimmerman, Daniel J. Drennan, James Ikeda, Qianru Jin, Herdeline Ann M. Ardoña, Sean L. Kim, Ryoma Ishii, Kevin Kit Parker","doi":"10.1126/scirobotics.adr6472","DOIUrl":"https://doi.org/10.1126/scirobotics.adr6472","url":null,"abstract":"In biomimetic design, researchers recreate existing biological structures to form functional devices. For biohybrid robotic swimmers assembled with tissue engineering, this is problematic because most devices operate at different length scales than their naturally occurring counterparts, resulting in reduced performance. To overcome these challenges, here, we demonstrate how machine learning–directed optimization (ML-DO) can be used to inform the design of a biohybrid robot, outperforming other nonlinear optimization techniques, such as Bayesian optimization, in the selection of high-performance geometries. We show how this approach can be used to maximize the thrust generated by a tissue-engineered mobuliform miniray. This results in devices that can swim at the millimeter scale while more closely preserving natural locomotive scaling laws. Overall, this work provides a quantitatively rigorous approach for the engineering design of muscular structure-function relationships in an automated fashion.","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"65 1","pages":""},"PeriodicalIF":25.0,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143393478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A hyperelastic torque-reversal mechanism for soft joints with compression-responsive transient bistability
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-01-29 DOI: 10.1126/scirobotics.ado7696
Woo-Young Choi, Woongbae Kim, Jae-Ryeong Choi, Sung Yol Yu, Seunguk Moon, Yong-Jai Park, Kyu-Jin Cho
Snap-through, a rapid transition of a system from an equilibrium state to a nonadjacent equilibrium state, is a valuable design element of soft devices for converting a monolithic stimulus into systematic responses with impulsive motions. A common way to benefit from snap-through is to embody it within structures and materials, such as bistable structures. Torque-reversal mechanisms discovered in nature, which harness snap-through instability via muscular forces, may have comparative advantages. However, the current intricacy of artificial torque-reversal mechanisms, which require sophisticated kinematics/kinetics, constrains design possibilities for soft joints and devices. Here, we harnessed hyperelasticity to implement a torque-reversal mechanism in a soft joint, generating repetitive cilia-like beating motions through an embedded tendon. The developed hyperelastic torque-reversal mechanism (HeTRM) exhibits transient bistability under a specific compressive displacement/force threshold, with snap-through occurring at the point where the transience ends. To validate the effectiveness of this design principle, we explored the functionalities of HeTRM in energy storage and release, dual modes for impulsive and continuous motion, mechanical fuse, and rapid three-dimensional motions, through proof-of-concept soft machines. We expect that this design principle provides insight into incorporating snap-through behavior in soft machines and may aid in understanding the relationship between torque-reversal mechanisms and bistability.
{"title":"A hyperelastic torque-reversal mechanism for soft joints with compression-responsive transient bistability","authors":"Woo-Young Choi,&nbsp;Woongbae Kim,&nbsp;Jae-Ryeong Choi,&nbsp;Sung Yol Yu,&nbsp;Seunguk Moon,&nbsp;Yong-Jai Park,&nbsp;Kyu-Jin Cho","doi":"10.1126/scirobotics.ado7696","DOIUrl":"10.1126/scirobotics.ado7696","url":null,"abstract":"<div >Snap-through, a rapid transition of a system from an equilibrium state to a nonadjacent equilibrium state, is a valuable design element of soft devices for converting a monolithic stimulus into systematic responses with impulsive motions. A common way to benefit from snap-through is to embody it within structures and materials, such as bistable structures. Torque-reversal mechanisms discovered in nature, which harness snap-through instability via muscular forces, may have comparative advantages. However, the current intricacy of artificial torque-reversal mechanisms, which require sophisticated kinematics/kinetics, constrains design possibilities for soft joints and devices. Here, we harnessed hyperelasticity to implement a torque-reversal mechanism in a soft joint, generating repetitive cilia-like beating motions through an embedded tendon. The developed hyperelastic torque-reversal mechanism (HeTRM) exhibits transient bistability under a specific compressive displacement/force threshold, with snap-through occurring at the point where the transience ends. To validate the effectiveness of this design principle, we explored the functionalities of HeTRM in energy storage and release, dual modes for impulsive and continuous motion, mechanical fuse, and rapid three-dimensional motions, through proof-of-concept soft machines. We expect that this design principle provides insight into incorporating snap-through behavior in soft machines and may aid in understanding the relationship between torque-reversal mechanisms and bistability.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 98","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143056632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The case against machine vision for the control of wearable robotics: Challenges for commercial adoption
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-01-29 DOI: 10.1126/scirobotics.adp5005
Keya Ghonasgi, Kyle J. Kaveny, David Langlois, Leifur D. Sigurðarson, Tim A. Swift, Jason Wheeler, Aaron J. Young
Deploying machine vision for wearable robot control faces challenges in terms of usability, reliability, privacy, and costs.
{"title":"The case against machine vision for the control of wearable robotics: Challenges for commercial adoption","authors":"Keya Ghonasgi,&nbsp;Kyle J. Kaveny,&nbsp;David Langlois,&nbsp;Leifur D. Sigurðarson,&nbsp;Tim A. Swift,&nbsp;Jason Wheeler,&nbsp;Aaron J. Young","doi":"10.1126/scirobotics.adp5005","DOIUrl":"10.1126/scirobotics.adp5005","url":null,"abstract":"<div >Deploying machine vision for wearable robot control faces challenges in terms of usability, reliability, privacy, and costs.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 98","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143069866","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Light-emitting, self-healing robotic fibers
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-01-29 DOI: 10.1126/scirobotics.adv7933
Melisa Yashinski
Optical and mechanical self-healing compatibility was achieved in a multilayered electroluminescent robotic soft fiber.
{"title":"Light-emitting, self-healing robotic fibers","authors":"Melisa Yashinski","doi":"10.1126/scirobotics.adv7933","DOIUrl":"10.1126/scirobotics.adv7933","url":null,"abstract":"<div >Optical and mechanical self-healing compatibility was achieved in a multilayered electroluminescent robotic soft fiber.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 98","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143069864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Safety-assured high-speed navigation for MAVs
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-01-29 DOI: 10.1126/scirobotics.ado6187
Yunfan Ren, Fangcheng Zhu, Guozheng Lu, Yixi Cai, Longji Yin, Fanze Kong, Jiarong Lin, Nan Chen, Fu Zhang
Micro air vehicles (MAVs) capable of high-speed autonomous navigation in unknown environments have the potential to improve applications like search and rescue and disaster relief, where timely and safe navigation is critical. However, achieving autonomous, safe, and high-speed MAV navigation faces systematic challenges, necessitating reduced vehicle weight and size for high-speed maneuvering, strong sensing capability for detecting obstacles at a distance, and advanced planning and control algorithms maximizing flight speed while ensuring obstacle avoidance. Here, we present the safety-assured high-speed aerial robot (SUPER), a compact MAV with a 280-millimeter wheelbase and a thrust-to-weight ratio greater than 5.0, enabling agile flight in cluttered environments. SUPER uses a lightweight three-dimensional light detection and ranging (LIDAR) sensor for accurate, long-range obstacle detection. To ensure high-speed flight while maintaining safety, we introduced an efficient planning framework that directly plans trajectories using LIDAR point clouds. In each replanning cycle, two trajectories were generated: one in known free spaces to ensure safety and another in both known and unknown spaces to maximize speed. Compared with baseline methods, this framework reduced failure rates by 35.9 times while flying faster and with half the planning time. In real-world tests, SUPER achieved autonomous flights at speeds exceeding 20 meters per second, successfully avoiding thin obstacles and navigating narrow spaces. SUPER represents a milestone in autonomous MAV systems, bridging the gap from laboratory research to real-world applications.
{"title":"Safety-assured high-speed navigation for MAVs","authors":"Yunfan Ren,&nbsp;Fangcheng Zhu,&nbsp;Guozheng Lu,&nbsp;Yixi Cai,&nbsp;Longji Yin,&nbsp;Fanze Kong,&nbsp;Jiarong Lin,&nbsp;Nan Chen,&nbsp;Fu Zhang","doi":"10.1126/scirobotics.ado6187","DOIUrl":"10.1126/scirobotics.ado6187","url":null,"abstract":"<div >Micro air vehicles (MAVs) capable of high-speed autonomous navigation in unknown environments have the potential to improve applications like search and rescue and disaster relief, where timely and safe navigation is critical. However, achieving autonomous, safe, and high-speed MAV navigation faces systematic challenges, necessitating reduced vehicle weight and size for high-speed maneuvering, strong sensing capability for detecting obstacles at a distance, and advanced planning and control algorithms maximizing flight speed while ensuring obstacle avoidance. Here, we present the safety-assured high-speed aerial robot (SUPER), a compact MAV with a 280-millimeter wheelbase and a thrust-to-weight ratio greater than 5.0, enabling agile flight in cluttered environments. SUPER uses a lightweight three-dimensional light detection and ranging (LIDAR) sensor for accurate, long-range obstacle detection. To ensure high-speed flight while maintaining safety, we introduced an efficient planning framework that directly plans trajectories using LIDAR point clouds. In each replanning cycle, two trajectories were generated: one in known free spaces to ensure safety and another in both known and unknown spaces to maximize speed. Compared with baseline methods, this framework reduced failure rates by 35.9 times while flying faster and with half the planning time. In real-world tests, SUPER achieved autonomous flights at speeds exceeding 20 meters per second, successfully avoiding thin obstacles and navigating narrow spaces. SUPER represents a milestone in autonomous MAV systems, bridging the gap from laboratory research to real-world applications.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 98","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.science.org/doi/reader/10.1126/scirobotics.ado6187","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143056756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Humanoid robot learning of complex behaviors with LLMs
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-01-22 DOI: 10.1126/scirobotics.adv4627
Amos Matsiko
Learning complex behaviors by humanoid robots could be achieved with natural interactions aided by large language models.
{"title":"Humanoid robot learning of complex behaviors with LLMs","authors":"Amos Matsiko","doi":"10.1126/scirobotics.adv4627","DOIUrl":"10.1126/scirobotics.adv4627","url":null,"abstract":"<div >Learning complex behaviors by humanoid robots could be achieved with natural interactions aided by large language models.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 98","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143025797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Development of compositionality through interactive learning of language and action of robots 通过语言和机器人动作的互动学习来发展组合性
IF 26.1 1区 计算机科学 Q1 ROBOTICS Pub Date : 2025-01-22 DOI: 10.1126/scirobotics.adp0751
Prasanna Vijayaraghavan, Jeffrey Frederic Queißer, Sergio Verduzco Flores, Jun Tani
Humans excel at applying learned behavior to unlearned situations. A crucial component of this generalization behavior is our ability to compose/decompose a whole into reusable parts, an attribute known as compositionality. One of the fundamental questions in robotics concerns this characteristic: How can linguistic compositionality be developed concomitantly with sensorimotor skills through associative learning, particularly when individuals only learn partial linguistic compositions and their corresponding sensorimotor patterns? To address this question, we propose a brain-inspired neural network model that integrates vision, proprioception, and language into a framework of predictive coding and active inference on the basis of the free-energy principle. The effectiveness and capabilities of this model were assessed through various simulation experiments conducted with a robot arm. Our results show that generalization in learning to unlearned verb-noun compositions is significantly enhanced when training variations of task composition are increased. We attribute this to self-organized compositional structures in linguistic latent state space being influenced substantially by sensorimotor learning. Ablation studies show that visual attention and working memory are essential to accurately generate visuomotor sequences to achieve linguistically represented goals. These insights advance our understanding of mechanisms underlying development of compositionality through interactions of linguistic and sensorimotor experience.
人类擅长将习得的行为应用于非习得的情境。这种泛化行为的一个关键组成部分是我们将整体组合/分解为可重用部分的能力,这种属性称为组合性。机器人技术的一个基本问题就是:语言组合性如何通过联想学习与感觉运动技能一起发展,特别是当个体只学习部分语言组合和相应的感觉运动模式时?为了解决这个问题,我们提出了一个基于自由能原理的脑启发神经网络模型,该模型将视觉、本体感觉和语言整合到一个预测编码和主动推理的框架中。通过机械臂进行的各种仿真实验,评估了该模型的有效性和能力。研究结果表明,当任务构成的训练变量增加时,非学习动词-名词组合的学习泛化能力显著增强。我们将此归因于语言潜态空间中的自组织组合结构受到感觉运动学习的影响。消融研究表明,视觉注意和工作记忆对于准确生成视觉运动序列以实现语言表征目标至关重要。这些见解促进了我们对通过语言和感觉运动经验的相互作用来发展组合性的机制的理解。
{"title":"Development of compositionality through interactive learning of language and action of robots","authors":"Prasanna Vijayaraghavan,&nbsp;Jeffrey Frederic Queißer,&nbsp;Sergio Verduzco Flores,&nbsp;Jun Tani","doi":"10.1126/scirobotics.adp0751","DOIUrl":"10.1126/scirobotics.adp0751","url":null,"abstract":"<div >Humans excel at applying learned behavior to unlearned situations. A crucial component of this generalization behavior is our ability to compose/decompose a whole into reusable parts, an attribute known as compositionality. One of the fundamental questions in robotics concerns this characteristic: How can linguistic compositionality be developed concomitantly with sensorimotor skills through associative learning, particularly when individuals only learn partial linguistic compositions and their corresponding sensorimotor patterns? To address this question, we propose a brain-inspired neural network model that integrates vision, proprioception, and language into a framework of predictive coding and active inference on the basis of the free-energy principle. The effectiveness and capabilities of this model were assessed through various simulation experiments conducted with a robot arm. Our results show that generalization in learning to unlearned verb-noun compositions is significantly enhanced when training variations of task composition are increased. We attribute this to self-organized compositional structures in linguistic latent state space being influenced substantially by sensorimotor learning. Ablation studies show that visual attention and working memory are essential to accurately generate visuomotor sequences to achieve linguistically represented goals. These insights advance our understanding of mechanisms underlying development of compositionality through interactions of linguistic and sensorimotor experience.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"10 98","pages":""},"PeriodicalIF":26.1,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143020311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Science Robotics
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1