Discovering neural policies to drive behaviour by integrating deep reinforcement learning agents with biological neural networks

IF 18.8 1区 计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Nature Machine Intelligence Pub Date : 2024-06-14 DOI:10.1038/s42256-024-00854-2
Chenguang Li, Gabriel Kreiman, Sharad Ramanathan
{"title":"Discovering neural policies to drive behaviour by integrating deep reinforcement learning agents with biological neural networks","authors":"Chenguang Li, Gabriel Kreiman, Sharad Ramanathan","doi":"10.1038/s42256-024-00854-2","DOIUrl":null,"url":null,"abstract":"Deep reinforcement learning (RL) has been successful in a variety of domains but has not yet been directly used to learn biological tasks by interacting with a living nervous system. As proof of principle, we show how to create such a hybrid system trained on a target-finding task. Using optogenetics, we interfaced the nervous system of the nematode Caenorhabditis elegans with a deep RL agent. Agents adapted to strikingly different sites of neural integration and learned site-specific activations to guide animals towards a target, including in cases where agents interfaced with sets of neurons with previously uncharacterized responses to optogenetic modulation. Agents were analysed by plotting their learned policies to understand how different sets of neurons were used to guide movement. Further, the animal and agent generalized to new environments using the same learned policies in food-search tasks, showing that the system achieved cooperative computation rather than the agent acting as a controller for a soft robot. Our system demonstrates that deep RL is a viable tool both for learning how neural circuits can produce goal-directed behaviour and for improving biologically relevant behaviour in a flexible way. Deep reinforcement learning (RL) has been successful in many fields but has not been used to directly improve behaviours by interfacing with living nervous systems. Li et al. present a framework that integrates deep RL agents with the nervous system of the nematode Caenorhabditis elegans. Their study shows that trained agents can assist animals in biologically relevant tasks and can be studied after training to map out effective neural policies.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8000,"publicationDate":"2024-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Nature Machine Intelligence","FirstCategoryId":"94","ListUrlMain":"https://www.nature.com/articles/s42256-024-00854-2","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

Deep reinforcement learning (RL) has been successful in a variety of domains but has not yet been directly used to learn biological tasks by interacting with a living nervous system. As proof of principle, we show how to create such a hybrid system trained on a target-finding task. Using optogenetics, we interfaced the nervous system of the nematode Caenorhabditis elegans with a deep RL agent. Agents adapted to strikingly different sites of neural integration and learned site-specific activations to guide animals towards a target, including in cases where agents interfaced with sets of neurons with previously uncharacterized responses to optogenetic modulation. Agents were analysed by plotting their learned policies to understand how different sets of neurons were used to guide movement. Further, the animal and agent generalized to new environments using the same learned policies in food-search tasks, showing that the system achieved cooperative computation rather than the agent acting as a controller for a soft robot. Our system demonstrates that deep RL is a viable tool both for learning how neural circuits can produce goal-directed behaviour and for improving biologically relevant behaviour in a flexible way. Deep reinforcement learning (RL) has been successful in many fields but has not been used to directly improve behaviours by interfacing with living nervous systems. Li et al. present a framework that integrates deep RL agents with the nervous system of the nematode Caenorhabditis elegans. Their study shows that trained agents can assist animals in biologically relevant tasks and can be studied after training to map out effective neural policies.

Abstract Image

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
通过整合深度强化学习代理与生物神经网络,发现驱动行为的神经政策
深度强化学习(RL)已在多个领域取得成功,但尚未直接用于通过与活体神经系统交互来学习生物任务。作为原理证明,我们展示了如何创建这样一个混合系统,并对其进行目标搜索任务训练。利用光遗传学,我们将线虫的神经系统与深度 RL 代理进行了连接。代理适应了显著不同的神经整合部位,并学会了特定部位的激活,以引导动物找到目标,包括在代理与一组神经元连接的情况下,这组神经元对光遗传学调制的反应以前从未表征过。研究人员通过绘制神经元的学习策略图来分析神经元,以了解如何利用不同的神经元组来引导运动。此外,动物和代理在寻找食物的任务中使用相同的学习策略泛化到新的环境,这表明该系统实现了合作计算,而不是代理充当软体机器人的控制器。我们的系统证明,深度 RL 是一种可行的工具,既能用于学习神经回路如何产生目标导向行为,也能用于以灵活的方式改进生物相关行为。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
36.90
自引率
2.10%
发文量
127
期刊介绍: Nature Machine Intelligence is a distinguished publication that presents original research and reviews on various topics in machine learning, robotics, and AI. Our focus extends beyond these fields, exploring their profound impact on other scientific disciplines, as well as societal and industrial aspects. We recognize limitless possibilities wherein machine intelligence can augment human capabilities and knowledge in domains like scientific exploration, healthcare, medical diagnostics, and the creation of safe and sustainable cities, transportation, and agriculture. Simultaneously, we acknowledge the emergence of ethical, social, and legal concerns due to the rapid pace of advancements. To foster interdisciplinary discussions on these far-reaching implications, Nature Machine Intelligence serves as a platform for dialogue facilitated through Comments, News Features, News & Views articles, and Correspondence. Our goal is to encourage a comprehensive examination of these subjects. Similar to all Nature-branded journals, Nature Machine Intelligence operates under the guidance of a team of skilled editors. We adhere to a fair and rigorous peer-review process, ensuring high standards of copy-editing and production, swift publication, and editorial independence.
期刊最新文献
General-purpose foundation models for increased autonomy in robot-assisted surgery Results from the autoPET challenge on fully automated lesion segmentation in oncologic PET/CT imaging Deep learning prediction of ribosome profiling with Translatomer reveals translational regulation and interprets disease variants Epitope-anchored contrastive transfer learning for paired CD8+ T cell receptor–antigen recognition Leveraging language model for advanced multiproperty molecular optimization via prompt engineering
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1