Pub Date : 2024-11-27DOI: 10.1126/scirobotics.adp2309
Minh Tri Luu, Jonathan F. Berengut, Jiahe Li, Jing-Bing Chen, Jasleen Kaur Daljit Singh, Kanako Coffi Dit Glieze, Matthew Turner, Karuna Skipper, Sreelakshmi Meppat, Hannah Fowler, William Close, Jonathan P. K. Doye, Ali Abbas, Shelley F. J. Wickham
In cells, proteins rapidly self-assemble into sophisticated nanomachines. Bioinspired self-assembly approaches, such as DNA origami, have been used to achieve complex three-dimensional (3D) nanostructures and devices. However, current synthetic systems are limited by low yields in hierarchical assembly and challenges in rapid and efficient reconfiguration between diverse structures. Here, we developed a modular system of DNA origami “voxels” with programmable 3D connections. We demonstrate multifunctional pools of up to 12 unique voxels that can assemble into many shapes, prototyping 50 structures. Programmable switching of local connections between flexible and rigid states achieved rapid and reversible reconfiguration of global structures in three dimensions. Multistep assembly pathways were then explored to increase the yield. Voxels were assembled via flexible chain intermediates into rigid structures, increasing yield up to 100-fold. We envision that foldable chains of DNA origami voxels can achieve increased complexity in reconfigurable nanomaterials, providing modular components for the assembly of nanorobotic systems with future applications in synthetic biology, assembly of inorganic materials, and nanomedicine.
在细胞中，蛋白质可快速自组装成精密的纳米机器。受生物启发的自组装方法（如 DNA 折纸）已被用于实现复杂的三维（3D）纳米结构和器件。然而，目前的合成系统受限于分层组装的低产量，以及在不同结构之间快速高效地重新配置所面临的挑战。在这里，我们开发了一种具有可编程三维连接的 DNA 折纸 "体块 "模块化系统。我们展示了由多达 12 个独特体块组成的多功能池，这些体块可以组装成多种形状，原型结构多达 50 种。局部连接在柔性和刚性状态之间的可编程切换实现了三维全局结构的快速可逆重组。然后探索了多步骤组装途径，以提高产量。体素通过柔性链中间体组装成刚性结构，产量提高了 100 倍。我们设想，DNA 折纸体块的可折叠链可以增加可重构纳米材料的复杂性，为纳米机器人系统的组装提供模块化组件，未来可应用于合成生物学、无机材料组装和纳米医学。
{"title":"Reconfigurable nanomaterials folded from multicomponent chains of DNA origami voxels","authors":"Minh Tri Luu, Jonathan F. Berengut, Jiahe Li, Jing-Bing Chen, Jasleen Kaur Daljit Singh, Kanako Coffi Dit Glieze, Matthew Turner, Karuna Skipper, Sreelakshmi Meppat, Hannah Fowler, William Close, Jonathan P. K. Doye, Ali Abbas, Shelley F. J. Wickham","doi":"10.1126/scirobotics.adp2309","DOIUrl":"10.1126/scirobotics.adp2309","url":null,"abstract":"<div >In cells, proteins rapidly self-assemble into sophisticated nanomachines. Bioinspired self-assembly approaches, such as DNA origami, have been used to achieve complex three-dimensional (3D) nanostructures and devices. However, current synthetic systems are limited by low yields in hierarchical assembly and challenges in rapid and efficient reconfiguration between diverse structures. Here, we developed a modular system of DNA origami “voxels” with programmable 3D connections. We demonstrate multifunctional pools of up to 12 unique voxels that can assemble into many shapes, prototyping 50 structures. Programmable switching of local connections between flexible and rigid states achieved rapid and reversible reconfiguration of global structures in three dimensions. Multistep assembly pathways were then explored to increase the yield. Voxels were assembled via flexible chain intermediates into rigid structures, increasing yield up to 100-fold. We envision that foldable chains of DNA origami voxels can achieve increased complexity in reconfigurable nanomaterials, providing modular components for the assembly of nanorobotic systems with future applications in synthetic biology, assembly of inorganic materials, and nanomedicine.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142741538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-27DOI: 10.1126/scirobotics.adl0842
Henry Hess, Parag Katira, Juan B. Rodriguez III
Molecular motors generate force to individually power molecular machines or collectively drive macroscopic actuators. The force output of molecular and macroscale motors appears to be constrained by the same scaling law relating motor force and mass. Here, potential origins of these universal performance characteristics are discussed and the implications examined.
{"title":"The force has limits: Molecular motors in robotics","authors":"Henry Hess, Parag Katira, Juan B. Rodriguez III","doi":"10.1126/scirobotics.adl0842","DOIUrl":"10.1126/scirobotics.adl0842","url":null,"abstract":"<div >Molecular motors generate force to individually power molecular machines or collectively drive macroscopic actuators. The force output of molecular and macroscale motors appears to be constrained by the same scaling law relating motor force and mass. Here, potential origins of these universal performance characteristics are discussed and the implications examined.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142741539","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-27DOI: 10.1126/scirobotics.adi2084
Lifeng Zhou, Yanyu Xiong, Abhisek Dwivedy, Mengxi Zheng, Laura Cooper, Skye Shepherd, Tingjie Song, Wei Hong, Linh T. P. Le, Xin Chen, Saurabh Umrao, Lijun Rong, Tong Wang, Brian T. Cunningham, Xing Wang
DNA has shown great biocompatibility, programmable mechanical properties, and precise structural addressability at the nanometer scale, rendering it a material for constructing versatile nanorobots for biomedical applications. Here, we present the design principle, synthesis, and characterization of a DNA nanorobotic hand, called DNA NanoGripper, that contains a palm and four bendable fingers as inspired by naturally evolved human hands, bird claws, and bacteriophages. Each NanoGripper finger consists of three phalanges connected by three rotatable joints that are bendable in response to the binding of other entities. NanoGripper functions are enabled and driven by the interactions between moieties attached to the fingers and their binding partners. We demonstrate that the NanoGripper can be engineered to effectively interact with and capture nanometer-scale objects, including gold nanoparticles, gold NanoUrchins, and SARS-CoV-2 virions. With multiple DNA aptamer nanoswitches programmed to generate a fluorescent signal that is enhanced on a photonic crystal platform, the NanoGripper functions as a highly sensitive biosensor that selectively detects intact SARS-CoV-2 virions in human saliva with a limit of detection of ~100 copies per milliliter, providing a sensitivity equal to that of reverse transcription quantitative polymerase chain reaction (RT-qPCR). Quantified by flow cytometry assays, we demonstrated that the NanoGripper-aptamer complex can effectively block viral entry into the host cells, suggesting its potential for inhibiting virus infections. The design, synthesis, and characterization of a sophisticated nanomachine that can be tailored for specific applications highlight a promising pathway toward feasible and efficient solutions to the detection and potential inhibition of virus infections.
DNA 具有很好的生物相容性、可编程机械特性以及纳米尺度的精确结构可寻址性，因此是构建生物医学应用领域多功能纳米机器人的材料。在这里，我们介绍一种 DNA 纳米机器人手（DNA NanoGripper）的设计原理、合成和表征，它包含一个手掌和四个可弯曲的手指，其灵感来自自然进化的人手、鸟爪和噬菌体。每个 NanoGripper 手指都由三根指骨组成，三根指骨通过三个可旋转的关节连接在一起，这些关节可根据与其他实体的结合而弯曲。纳米抓手的功能是由附着在手指上的分子与其结合伙伴之间的相互作用实现和驱动的。我们的研究表明，纳米抓取器可以有效地与纳米级物体相互作用并捕获它们，包括金纳米粒子、金纳米乌尔奇蛋白和 SARS-CoV-2 病毒。NanoGripper 可作为高灵敏度的生物传感器，选择性地检测人体唾液中完整的 SARS-CoV-2 病毒，检测限为每毫升约 100 个拷贝，灵敏度与反转录定量聚合酶链反应（RT-qPCR）相当。我们通过流式细胞术测定进行了定量分析，结果表明纳米抓取器-aptamer 复合物能有效阻止病毒进入宿主细胞，这表明它具有抑制病毒感染的潜力。设计、合成和表征一种可为特定应用定制的精密纳米机器，为检测和潜在抑制病毒感染提供了一条可行而高效的途径。
{"title":"Bioinspired designer DNA NanoGripper for virus sensing and potential inhibition","authors":"Lifeng Zhou, Yanyu Xiong, Abhisek Dwivedy, Mengxi Zheng, Laura Cooper, Skye Shepherd, Tingjie Song, Wei Hong, Linh T. P. Le, Xin Chen, Saurabh Umrao, Lijun Rong, Tong Wang, Brian T. Cunningham, Xing Wang","doi":"10.1126/scirobotics.adi2084","DOIUrl":"10.1126/scirobotics.adi2084","url":null,"abstract":"<div >DNA has shown great biocompatibility, programmable mechanical properties, and precise structural addressability at the nanometer scale, rendering it a material for constructing versatile nanorobots for biomedical applications. Here, we present the design principle, synthesis, and characterization of a DNA nanorobotic hand, called DNA NanoGripper, that contains a palm and four bendable fingers as inspired by naturally evolved human hands, bird claws, and bacteriophages. Each NanoGripper finger consists of three phalanges connected by three rotatable joints that are bendable in response to the binding of other entities. NanoGripper functions are enabled and driven by the interactions between moieties attached to the fingers and their binding partners. We demonstrate that the NanoGripper can be engineered to effectively interact with and capture nanometer-scale objects, including gold nanoparticles, gold NanoUrchins, and SARS-CoV-2 virions. With multiple DNA aptamer nanoswitches programmed to generate a fluorescent signal that is enhanced on a photonic crystal platform, the NanoGripper functions as a highly sensitive biosensor that selectively detects intact SARS-CoV-2 virions in human saliva with a limit of detection of ~100 copies per milliliter, providing a sensitivity equal to that of reverse transcription quantitative polymerase chain reaction (RT-qPCR). Quantified by flow cytometry assays, we demonstrated that the NanoGripper-aptamer complex can effectively block viral entry into the host cells, suggesting its potential for inhibiting virus infections. The design, synthesis, and characterization of a sophisticated nanomachine that can be tailored for specific applications highlight a promising pathway toward feasible and efficient solutions to the detection and potential inhibition of virus infections.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142741536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cybernetic avatars integrate physical and virtual avatars to enhance human capabilities in diverse scales and contexts.
控制论化身将物理化身和虚拟化身结合起来，在不同规模和背景下增强人类的能力。
{"title":"Cybernetic avatars: Teleoperation technologies from in-body monitoring to social interaction","authors":"Norihiro Hagita, Ryota Kanai, Hiroshi Ishiguro, Kouta Minamizawa, Fumihito Arai, Fumio Shimpo, Takeshi Matsumura, Yoko Yamanishi","doi":"10.1126/scirobotics.adg1842","DOIUrl":"10.1126/scirobotics.adg1842","url":null,"abstract":"<div >Cybernetic avatars integrate physical and virtual avatars to enhance human capabilities in diverse scales and contexts.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142683762","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-20DOI: 10.1126/scirobotics.ado3890
Hoang-Vu Phan, Dario Floreano
A banked turn is a common flight maneuver observed in birds and aircraft. To initiate the turn, whereas traditional aircraft rely on the wing ailerons, most birds use a variety of asymmetric wing-morphing control techniques to roll their bodies and thus redirect the lift vector to the direction of the turn. Nevertheless, when searching for prey, soaring raptors execute steady banked turns without exhibiting observable wing movements apart from the tail twisting around the body axis. Although tail twisting can compensate for adverse yaw, functioning similarly to the vertical tail in aircraft, how raptors use only tail twisting to perform banked turns is still not well understood. Here, we developed and used a raptor-inspired feathered drone to find that the proximity of the tail to the wings causes asymmetric wing-induced flows over the twisted tail and thus lift asymmetry, resulting in both roll and yaw moments sufficient to coordinate banked turns. Moreover, twisting the tail induces a nose-up pitch moment that increases the angle of attack of the wings, thereby generating more lift to compensate for losses caused by the banking motion. Flight experiments confirm the effectiveness of tail twist to control not only low-speed steady banked turns but also high-speed sharp turns by means of coordinated tail twist and pitch with asymmetric wing shape morphing. These findings contribute to the understanding of avian flight behaviors that are difficult to study in controlled laboratory settings and provide effective control strategies for agile drones with morphing aerial surfaces.
{"title":"A twist of the tail in turning maneuvers of bird-inspired drones","authors":"Hoang-Vu Phan, Dario Floreano","doi":"10.1126/scirobotics.ado3890","DOIUrl":"10.1126/scirobotics.ado3890","url":null,"abstract":"<div >A banked turn is a common flight maneuver observed in birds and aircraft. To initiate the turn, whereas traditional aircraft rely on the wing ailerons, most birds use a variety of asymmetric wing-morphing control techniques to roll their bodies and thus redirect the lift vector to the direction of the turn. Nevertheless, when searching for prey, soaring raptors execute steady banked turns without exhibiting observable wing movements apart from the tail twisting around the body axis. Although tail twisting can compensate for adverse yaw, functioning similarly to the vertical tail in aircraft, how raptors use only tail twisting to perform banked turns is still not well understood. Here, we developed and used a raptor-inspired feathered drone to find that the proximity of the tail to the wings causes asymmetric wing-induced flows over the twisted tail and thus lift asymmetry, resulting in both roll and yaw moments sufficient to coordinate banked turns. Moreover, twisting the tail induces a nose-up pitch moment that increases the angle of attack of the wings, thereby generating more lift to compensate for losses caused by the banking motion. Flight experiments confirm the effectiveness of tail twist to control not only low-speed steady banked turns but also high-speed sharp turns by means of coordinated tail twist and pitch with asymmetric wing shape morphing. These findings contribute to the understanding of avian flight behaviors that are difficult to study in controlled laboratory settings and provide effective control strategies for agile drones with morphing aerial surfaces.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142678442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-20DOI: 10.1126/scirobotics.ado5566
Thomas Schmickl, Donato Romano
Interactively teaming up animals and robots could facilitate basic scientific research and address environmental and ecological crises.
动物和机器人的互动合作可以促进基础科学研究，解决环境和生态危机。
{"title":"Robots and animals teaming up in the wild to tackle ecosystem challenges","authors":"Thomas Schmickl, Donato Romano","doi":"10.1126/scirobotics.ado5566","DOIUrl":"10.1126/scirobotics.ado5566","url":null,"abstract":"<div >Interactively teaming up animals and robots could facilitate basic scientific research and address environmental and ecological crises.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142683768","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-20DOI: 10.1126/scirobotics.ado4535
Eric Chang, Diana D. Chin, David Lentink
Gliding birds lack a vertical tail, yet they fly stably rudderless in turbulence without needing discrete flaps to steer. In contrast, nearly all airplanes need vertical tails to damp Dutch roll oscillations and to control yaw. The few exceptions that lack a vertical tail either leverage differential drag-based yaw actuators or their fixed planforms are carefully tuned for passively stable Dutch roll and proverse yaw. Biologists hypothesize that birds stabilize and control gliding flight without rudders by using their wing and tail reflexes, but no rudderless airplane has a morphing wing or tail that can change shape like a bird. Our rudderless biohybrid robot, PigeonBot II, can damp its Dutch roll instability (caused by lacking a vertical tail) and control flight by morphing its biomimetic wing and tail reflexively like a bird. The bird-inspired adaptive reflexive controller was tuned in a wind tunnel to mitigate turbulent perturbations, which enabled PigeonBot II to fly autonomously in the atmosphere with pigeon-like poses. This work is a mechanistic confirmation of how birds accomplish rudderless flight via reflex functions, and it can inspire rudderless aircraft with reduced radar signature and increased efficacy.
滑翔翼鸟类没有垂直尾翼，但它们在乱流中却能稳定地无舵飞行，而不需要分散的襟翼来转向。相比之下，几乎所有飞机都需要垂直尾翼来抑制荷兰式滚转振荡并控制偏航。少数没有垂直尾翼的飞机要么利用了基于阻力的差动式偏航致动器，要么对其固定翼形进行了精心调整，以实现被动稳定的荷兰式滚转和逆向偏航。根据生物学家的假设，鸟类在没有方向舵的情况下，可以利用翅膀和尾部的反射来稳定和控制滑翔飞行，但没有一种无舵飞机拥有像鸟类一样可以改变形状的变形翅膀或尾部。我们的无舵生物混合机器人 PigeonBot II 可以抑制荷兰式的滚动不稳定性（由于缺乏垂直尾翼），并通过像鸟类一样反射性地变形其仿生翅膀和尾翼来控制飞行。受鸟类启发的自适应反射控制器在风洞中进行了调整，以减轻湍流扰动，从而使 PigeonBot II 能够在大气中以鸽子般的姿势自主飞行。这项工作从机理上证实了鸟类是如何通过反射功能实现无舵飞行的，它可以为无舵飞行器提供灵感，减少雷达信号并提高飞行效率。
{"title":"Bird-inspired reflexive morphing enables rudderless flight","authors":"Eric Chang, Diana D. Chin, David Lentink","doi":"10.1126/scirobotics.ado4535","DOIUrl":"10.1126/scirobotics.ado4535","url":null,"abstract":"<div >Gliding birds lack a vertical tail, yet they fly stably rudderless in turbulence without needing discrete flaps to steer. In contrast, nearly all airplanes need vertical tails to damp Dutch roll oscillations and to control yaw. The few exceptions that lack a vertical tail either leverage differential drag-based yaw actuators or their fixed planforms are carefully tuned for passively stable Dutch roll and proverse yaw. Biologists hypothesize that birds stabilize and control gliding flight without rudders by using their wing and tail reflexes, but no rudderless airplane has a morphing wing or tail that can change shape like a bird. Our rudderless biohybrid robot, PigeonBot II, can damp its Dutch roll instability (caused by lacking a vertical tail) and control flight by morphing its biomimetic wing and tail reflexively like a bird. The bird-inspired adaptive reflexive controller was tuned in a wind tunnel to mitigate turbulent perturbations, which enabled PigeonBot II to fly autonomously in the atmosphere with pigeon-like poses. This work is a mechanistic confirmation of how birds accomplish rudderless flight via reflex functions, and it can inspire rudderless aircraft with reduced radar signature and increased efficacy.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.science.org/doi/reader/10.1126/scirobotics.ado4535","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142679110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-13DOI: 10.1126/scirobotics.adl0628
Sudharshan Suresh, Haozhi Qi, Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mrinal Kalakrishnan, Roberto Calandra, Michael Kaess, Joseph Ortiz, Mustafa Mukadam
To achieve human-level dexterity, robots must infer spatial awareness from multimodal sensing to reason over contact interactions. During in-hand manipulation of novel objects, such spatial awareness involves estimating the object’s pose and shape. The status quo for in-hand perception primarily uses vision and is restricted to tracking a priori known objects. Moreover, visual occlusion of objects in hand is imminent during manipulation, preventing current systems from pushing beyond tasks without occlusion. We combined vision and touch sensing on a multifingered hand to estimate an object’s pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. We studied multimodal in-hand perception in simulation and the real world, interacting with different objects via a proprioception-driven policy. Our experiments showed final reconstruction F scores of 81% and average pose drifts of 4.7 millimeters, which was further reduced to 2.3 millimeters with known object models. In addition, we observed that, under heavy visual occlusion, we could achieve improvements in tracking up to 94% compared with vision-only methods. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation. We release our evaluation dataset of 70 experiments, FeelSight, as a step toward benchmarking in this domain. Our neural representation driven by multimodal sensing can serve as a perception backbone toward advancing robot dexterity.
{"title":"NeuralFeels with neural fields: Visuotactile perception for in-hand manipulation","authors":"Sudharshan Suresh, Haozhi Qi, Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mrinal Kalakrishnan, Roberto Calandra, Michael Kaess, Joseph Ortiz, Mustafa Mukadam","doi":"10.1126/scirobotics.adl0628","DOIUrl":"10.1126/scirobotics.adl0628","url":null,"abstract":"<div >To achieve human-level dexterity, robots must infer spatial awareness from multimodal sensing to reason over contact interactions. During in-hand manipulation of novel objects, such spatial awareness involves estimating the object’s pose and shape. The status quo for in-hand perception primarily uses vision and is restricted to tracking a priori known objects. Moreover, visual occlusion of objects in hand is imminent during manipulation, preventing current systems from pushing beyond tasks without occlusion. We combined vision and touch sensing on a multifingered hand to estimate an object’s pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. We studied multimodal in-hand perception in simulation and the real world, interacting with different objects via a proprioception-driven policy. Our experiments showed final reconstruction <i>F</i> scores of 81% and average pose drifts of 4.7 millimeters, which was further reduced to 2.3 millimeters with known object models. In addition, we observed that, under heavy visual occlusion, we could achieve improvements in tracking up to 94% compared with vision-only methods. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation. We release our evaluation dataset of 70 experiments, FeelSight, as a step toward benchmarking in this domain. Our neural representation driven by multimodal sensing can serve as a perception backbone toward advancing robot dexterity.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.science.org/doi/reader/10.1126/scirobotics.adl0628","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142601978","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-13DOI: 10.1126/scirobotics.adp2507
Keya Ghonasgi, Taylor Higgins, Meghan E. Huber, Marcia K. O’Malley
Holistic consideration of the human and the robot is necessary to overcome hurdles in human-robot interaction.
要克服人机交互中的障碍，就必须全面考虑人和机器人。
{"title":"Crucial hurdles to achieving human-robot harmony","authors":"Keya Ghonasgi, Taylor Higgins, Meghan E. Huber, Marcia K. O’Malley","doi":"10.1126/scirobotics.adp2507","DOIUrl":"10.1126/scirobotics.adp2507","url":null,"abstract":"<div >Holistic consideration of the human and the robot is necessary to overcome hurdles in human-robot interaction.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142632266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-13DOI: 10.1126/scirobotics.adt8902
Robin R. Murphy
Adrian Tchaikovsky’s new novel Service Model humorously imagines a robot Jeeves coping with the end of civilization.
Adrian Tchaikovsky 的新作《服务模式》幽默地想象了一个机器人吉夫斯如何应对文明的终结。
{"title":"How much initiative should a service robot have?","authors":"Robin R. Murphy","doi":"10.1126/scirobotics.adt8902","DOIUrl":"10.1126/scirobotics.adt8902","url":null,"abstract":"<div >Adrian Tchaikovsky’s new novel <i>Service Model</i> humorously imagines a robot Jeeves coping with the end of civilization.</div>","PeriodicalId":56029,"journal":{"name":"Science Robotics","volume":"9 96","pages":""},"PeriodicalIF":26.1,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142632309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号