Pub Date : 2024-07-11DOI: 10.1038/s42256-024-00866-y
Shuangxia Ren, Gregory F. Cooper, Lujia Chen, Xinghua Lu
Cancers result from aberrations in cellular signalling systems, typically resulting from driver somatic genome alterations (SGAs) in individual tumours. Precision oncology requires understanding the cellular state and selecting medications that induce vulnerability in cancer cells under such conditions. To this end, we developed a computational framework consisting of two components: (1) a representation-learning component, which learns a representation of the cellular signalling systems when perturbed by SGAs and uses a biologically motivated and interpretable deep learning model, and (2) a drug-response prediction component, which predicts drug responses by leveraging the information of the cellular state of the cancer cells derived by the first component. Our cell-state-oriented framework notably improves the accuracy of predictions of drug responses compared to models using SGAs directly in cell lines. Moreover, our model performs well with real patient data. Importantly, our framework enables the prediction of responses to chemotherapy agents based on SGAs, thus expanding genome-informed precision oncology beyond molecularly targeted drugs. Precision oncology requires analysis of genomic alterations in cancer cells. Ren et al. develop an interpretable artificial intelligence framework that transforms somatic genomic alterations into representations of cellular signalling systems and accurately predicts cells’ responses to anticancer drugs.
{"title":"An interpretable deep learning framework for genome-informed precision oncology","authors":"Shuangxia Ren, Gregory F. Cooper, Lujia Chen, Xinghua Lu","doi":"10.1038/s42256-024-00866-y","DOIUrl":"10.1038/s42256-024-00866-y","url":null,"abstract":"Cancers result from aberrations in cellular signalling systems, typically resulting from driver somatic genome alterations (SGAs) in individual tumours. Precision oncology requires understanding the cellular state and selecting medications that induce vulnerability in cancer cells under such conditions. To this end, we developed a computational framework consisting of two components: (1) a representation-learning component, which learns a representation of the cellular signalling systems when perturbed by SGAs and uses a biologically motivated and interpretable deep learning model, and (2) a drug-response prediction component, which predicts drug responses by leveraging the information of the cellular state of the cancer cells derived by the first component. Our cell-state-oriented framework notably improves the accuracy of predictions of drug responses compared to models using SGAs directly in cell lines. Moreover, our model performs well with real patient data. Importantly, our framework enables the prediction of responses to chemotherapy agents based on SGAs, thus expanding genome-informed precision oncology beyond molecularly targeted drugs. Precision oncology requires analysis of genomic alterations in cancer cells. Ren et al. develop an interpretable artificial intelligence framework that transforms somatic genomic alterations into representations of cellular signalling systems and accurately predicts cells’ responses to anticancer drugs.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141584143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-11DOI: 10.1038/s42256-024-00865-z
Gaoyang Liu, Chen Wang, Tian Xia
Differential privacy offers protection in medical image processing but is traditionally thought to hinder accuracy. A recent study offers a reality check on the relationship between privacy measures and the ability of an artificial intelligence (AI) model to accurately analyse medical images.
{"title":"Shielding sensitive medical imaging data","authors":"Gaoyang Liu, Chen Wang, Tian Xia","doi":"10.1038/s42256-024-00865-z","DOIUrl":"10.1038/s42256-024-00865-z","url":null,"abstract":"Differential privacy offers protection in medical image processing but is traditionally thought to hinder accuracy. A recent study offers a reality check on the relationship between privacy measures and the ability of an artificial intelligence (AI) model to accurately analyse medical images.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141584292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Knowledge from animals and humans inspires robotic innovations. Numerous efforts have been made to achieve agile locomotion in quadrupedal robots through classical controllers or reinforcement learning approaches. These methods usually rely on physical models or handcrafted rewards to accurately describe the specific system, rather than on a generalized understanding like animals do. Here we propose a hierarchical framework to construct primitive-, environmental- and strategic-level knowledge that are all pre-trainable, reusable and enrichable for legged robots. The primitive module summarizes knowledge from animal motion data, where, inspired by large pre-trained models in language and image understanding, we introduce deep generative models to produce motor control signals stimulating legged robots to act like real animals. Then, we shape various traversing capabilities at a higher level to align with the environment by reusing the primitive module. Finally, a strategic module is trained focusing on complex downstream tasks by reusing the knowledge from previous levels. We apply the trained hierarchical controllers to the MAX robot, a quadrupedal robot developed in-house, to mimic animals, traverse complex obstacles and play in a designed challenging multi-agent chase tag game, where lifelike agility and strategy emerge in the robots. A key challenge in robotics is leveraging pre-training as a form of knowledge to generate movements. The authors propose a general learning framework for reusing pre-trained knowledge across different perception and task levels. The deployed robots exhibit lifelike agility and sophisticated game-playing strategies.
来自动物和人类的知识激发了机器人的创新。为了通过经典控制器或强化学习方法实现四足机器人的敏捷运动，人们做出了许多努力。这些方法通常依赖物理模型或手工制作的奖励来准确描述特定系统，而不是像动物那样依赖广义的理解。在这里，我们提出了一个分层框架，用于构建原始、环境和策略层面的知识，这些知识都是可预先训练、可重复使用和可丰富的，适用于有腿机器人。原始模块总结了来自动物运动数据的知识，受语言和图像理解方面的大型预训练模型的启发，我们引入了深度生成模型，以产生运动控制信号，刺激有腿机器人像真实动物一样行动。然后，我们在更高层次上塑造各种穿越能力，通过重复使用原始模块与环境保持一致。最后，通过重复使用前几级的知识，训练出一个战略模块，专注于复杂的下游任务。我们将训练好的分层控制器应用于 MAX 机器人（一种自主开发的四足机器人），让它模仿动物，穿越复杂的障碍物，并参与设计好的具有挑战性的多代理追逐游戏，在游戏中，机器人表现出栩栩如生的敏捷性和策略性。
{"title":"Lifelike agility and play in quadrupedal robots using reinforcement learning and generative pre-trained models","authors":"Lei Han, Qingxu Zhu, Jiapeng Sheng, Chong Zhang, Tingguang Li, Yizheng Zhang, He Zhang, Yuzhen Liu, Cheng Zhou, Rui Zhao, Jie Li, Yufeng Zhang, Rui Wang, Wanchao Chi, Xiong Li, Yonghui Zhu, Lingzhu Xiang, Xiao Teng, Zhengyou Zhang","doi":"10.1038/s42256-024-00861-3","DOIUrl":"10.1038/s42256-024-00861-3","url":null,"abstract":"Knowledge from animals and humans inspires robotic innovations. Numerous efforts have been made to achieve agile locomotion in quadrupedal robots through classical controllers or reinforcement learning approaches. These methods usually rely on physical models or handcrafted rewards to accurately describe the specific system, rather than on a generalized understanding like animals do. Here we propose a hierarchical framework to construct primitive-, environmental- and strategic-level knowledge that are all pre-trainable, reusable and enrichable for legged robots. The primitive module summarizes knowledge from animal motion data, where, inspired by large pre-trained models in language and image understanding, we introduce deep generative models to produce motor control signals stimulating legged robots to act like real animals. Then, we shape various traversing capabilities at a higher level to align with the environment by reusing the primitive module. Finally, a strategic module is trained focusing on complex downstream tasks by reusing the knowledge from previous levels. We apply the trained hierarchical controllers to the MAX robot, a quadrupedal robot developed in-house, to mimic animals, traverse complex obstacles and play in a designed challenging multi-agent chase tag game, where lifelike agility and strategy emerge in the robots. A key challenge in robotics is leveraging pre-training as a form of knowledge to generate movements. The authors propose a general learning framework for reusing pre-trained knowledge across different perception and task levels. The deployed robots exhibit lifelike agility and sophisticated game-playing strategies.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141553468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-05DOI: 10.1038/s42256-024-00856-0
Maria Boulougouri, Pierre Vandergheynst, Daniel Probst
Computational representation of molecules can take many forms, including graphs, string encodings of graphs, binary vectors or learned embeddings in the form of real-valued vectors. These representations are then used in downstream classification and regression tasks using a wide range of machine learning models. However, existing models come with limitations, such as the requirement for clearly defined chemical bonds, which often do not represent the true underlying nature of a molecule. Here we propose a framework for molecular machine learning tasks based on set representation learning. We show that learning on sets of atom invariants alone reaches the performance of state-of-the-art graph-based models on the most-used chemical benchmark datasets and that introducing a set representation layer into graph neural networks can surpass the performance of established methods in the domains of chemistry, biology and material science. We introduce specialized set representation-based neural network architectures for reaction-yield and protein–ligand binding-affinity prediction. Overall, we show that the technique we denote molecular set representation learning is both an alternative and an extension to graph neural network architectures for machine learning tasks on molecules, molecule complexes and chemical reactions. Machine learning methods for molecule predictions use various representations of molecules such as in the form of strings or graphs. As an extension of graph representation learning, Probst and colleagues propose to represent a molecule as a set of atoms, to better capture the underlying chemical nature, and demonstrate improved performance in a range of machine learning tasks.
{"title":"Molecular set representation learning","authors":"Maria Boulougouri, Pierre Vandergheynst, Daniel Probst","doi":"10.1038/s42256-024-00856-0","DOIUrl":"10.1038/s42256-024-00856-0","url":null,"abstract":"Computational representation of molecules can take many forms, including graphs, string encodings of graphs, binary vectors or learned embeddings in the form of real-valued vectors. These representations are then used in downstream classification and regression tasks using a wide range of machine learning models. However, existing models come with limitations, such as the requirement for clearly defined chemical bonds, which often do not represent the true underlying nature of a molecule. Here we propose a framework for molecular machine learning tasks based on set representation learning. We show that learning on sets of atom invariants alone reaches the performance of state-of-the-art graph-based models on the most-used chemical benchmark datasets and that introducing a set representation layer into graph neural networks can surpass the performance of established methods in the domains of chemistry, biology and material science. We introduce specialized set representation-based neural network architectures for reaction-yield and protein–ligand binding-affinity prediction. Overall, we show that the technique we denote molecular set representation learning is both an alternative and an extension to graph neural network architectures for machine learning tasks on molecules, molecule complexes and chemical reactions. Machine learning methods for molecule predictions use various representations of molecules such as in the form of strings or graphs. As an extension of graph representation learning, Probst and colleagues propose to represent a molecule as a set of atoms, to better capture the underlying chemical nature, and demonstrate improved performance in a range of machine learning tasks.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00856-0.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141553473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-27DOI: 10.1038/s42256-024-00846-2
Alpha Renner, Lazar Supic, Andreea Danielescu, Giacomo Indiveri, E. Paxon Frady, Friedrich T. Sommer, Yulia Sandamirskaya
Visual odometry (VO) is a method used to estimate self-motion of a mobile robot using visual sensors. Unlike odometry based on integrating differential measurements that can accumulate errors, such as inertial sensors or wheel encoders, VO is not compromised by drift. However, image-based VO is computationally demanding, limiting its application in use cases with low-latency, low-memory and low-energy requirements. Neuromorphic hardware offers low-power solutions to many vision and artificial intelligence problems, but designing such solutions is complicated and often has to be assembled from scratch. Here we propose the use of vector symbolic architecture (VSA) as an abstraction layer to design algorithms compatible with neuromorphic hardware. Building from a VSA model for scene analysis, described in our companion paper, we present a modular neuromorphic algorithm that achieves state-of-the-art performance on two-dimensional VO tasks. Specifically, the proposed algorithm stores and updates a working memory of the presented visual environment. Based on this working memory, a resonator network estimates the changing location and orientation of the camera. We experimentally validate the neuromorphic VSA-based approach to VO with two benchmarks: one based on an event-camera dataset and the other in a dynamic scene with a robotic task. Visual odometry, or self-motion estimation, is a fundamental task in robotics. Renner, Supic and colleagues introduce a neuromorphic algorithm for visual odometry that leverages hyperdimensional computing and hierarchical resonators. The approach estimates a robot’s motion from event-based vision, a step towards low-power machine vision for robotics.
视觉里程计（VO）是一种利用视觉传感器估算移动机器人自我运动的方法。不同于惯性传感器或轮子编码器等基于积分差分测量的里程测量法，视觉里程测量法不会受到漂移的影响。然而，基于图像的 VO 对计算要求很高，这限制了它在低延迟、低内存和低能耗要求的使用案例中的应用。神经形态硬件为许多视觉和人工智能问题提供了低功耗解决方案，但设计这种解决方案非常复杂，通常需要从头开始组装。在此，我们建议使用矢量符号架构（VSA）作为抽象层，设计与神经形态硬件兼容的算法。在我们的配套论文中描述的用于场景分析的 VSA 模型的基础上，我们提出了一种模块化神经形态算法，该算法在二维 VO 任务上实现了最先进的性能。具体来说，所提出的算法存储并更新呈现视觉环境的工作记忆。在此工作记忆的基础上，谐振器网络会估算出摄像头不断变化的位置和方向。我们通过两个基准实验验证了基于神经形态 VSA 的虚拟现实方法：一个基于事件摄像机数据集，另一个基于机器人任务的动态场景。
{"title":"Visual odometry with neuromorphic resonator networks","authors":"Alpha Renner, Lazar Supic, Andreea Danielescu, Giacomo Indiveri, E. Paxon Frady, Friedrich T. Sommer, Yulia Sandamirskaya","doi":"10.1038/s42256-024-00846-2","DOIUrl":"10.1038/s42256-024-00846-2","url":null,"abstract":"Visual odometry (VO) is a method used to estimate self-motion of a mobile robot using visual sensors. Unlike odometry based on integrating differential measurements that can accumulate errors, such as inertial sensors or wheel encoders, VO is not compromised by drift. However, image-based VO is computationally demanding, limiting its application in use cases with low-latency, low-memory and low-energy requirements. Neuromorphic hardware offers low-power solutions to many vision and artificial intelligence problems, but designing such solutions is complicated and often has to be assembled from scratch. Here we propose the use of vector symbolic architecture (VSA) as an abstraction layer to design algorithms compatible with neuromorphic hardware. Building from a VSA model for scene analysis, described in our companion paper, we present a modular neuromorphic algorithm that achieves state-of-the-art performance on two-dimensional VO tasks. Specifically, the proposed algorithm stores and updates a working memory of the presented visual environment. Based on this working memory, a resonator network estimates the changing location and orientation of the camera. We experimentally validate the neuromorphic VSA-based approach to VO with two benchmarks: one based on an event-camera dataset and the other in a dynamic scene with a robotic task. Visual odometry, or self-motion estimation, is a fundamental task in robotics. Renner, Supic and colleagues introduce a neuromorphic algorithm for visual odometry that leverages hyperdimensional computing and hierarchical resonators. The approach estimates a robot’s motion from event-based vision, a step towards low-power machine vision for robotics.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141462393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-27DOI: 10.1038/s42256-024-00859-x
Lidong Yang, Jialin Jiang, Fengtong Ji, Yangmin Li, Kai-Leung Yung, Antoine Ferreira, Li Zhang
Machine learning (ML) has revolutionized robotics by enhancing perception, adaptability, decision-making and more, enabling robots to work in complex scenarios beyond the capabilities of traditional approaches. However, the downsizing of robots to micro- and nanoscales introduces new challenges. For example, complexities in the actuation and locomotion of micro- and nanorobots defy traditional modelling methods, while control and navigation are complicated by strong environmental disruptions, and tracking in vivo encounters substantial noise interference. Recently, ML has also been shown to offer a promising avenue to tackle these complexities. Here we discuss how ML advances many crucial aspects of micro- and nanorobots, that is, in their design, actuation, locomotion, planning, tracking and navigation. Any application that can benefit from these fundamental advancements will be a potential beneficiary of this field, including micromanipulation, targeted delivery and therapy, bio-sensing, diagnosis and so on. This Review aims to provide an accessible and comprehensive survey for readers to quickly appreciate recent exciting accomplishments in ML for micro- and nanorobots. We also discuss potential issues and prospects of this burgeoning research direction. We hope this Review can foster interdisciplinary collaborations across robotics, computer science, material science and allied disciplines, to develop ML techniques that surmount fundamental challenges and further expand the application horizons of micro- and nanorobotics in biomedicine. Machine learning approaches in micro- and nanorobotics promise to overcome challenges encountered by applying traditional control methods at the microscopic scale. Lidong Yang et al. review this emerging area in robotics and discuss machine learning developments in design, actuation, locomotion, planning, tracking and navigation of microrobots.
机器学习（ML）通过增强感知能力、适应能力、决策能力等，使机器人能够在复杂场景中工作，超越了传统方法的能力范围，从而彻底改变了机器人技术。然而，将机器人缩小到微米级和纳米级带来了新的挑战。例如，微型和纳米机器人的驱动和运动的复杂性使传统的建模方法望而却步，而控制和导航因强烈的环境干扰而变得复杂，体内跟踪也会遇到大量噪声干扰。最近的研究表明，ML 为解决这些复杂问题提供了一条很有前景的途径。在此，我们将讨论 ML 如何在微型和纳米机器人的设计、驱动、运动、规划、跟踪和导航等许多关键方面取得进展。任何能从这些基本进步中获益的应用都将是这一领域的潜在受益者，包括微操纵、定向输送和治疗、生物传感、诊断等。本综述旨在为读者提供一份通俗易懂的综合调查报告，以便读者快速了解微机器人和纳米机器人在智能语言方面最近取得的令人振奋的成就。我们还讨论了这一新兴研究方向的潜在问题和前景。我们希望这篇综述能促进机器人学、计算机科学、材料科学和相关学科之间的跨学科合作，以开发克服基本挑战的 ML 技术，并进一步拓展微纳机器人在生物医学中的应用范围。
{"title":"Machine learning for micro- and nanorobots","authors":"Lidong Yang, Jialin Jiang, Fengtong Ji, Yangmin Li, Kai-Leung Yung, Antoine Ferreira, Li Zhang","doi":"10.1038/s42256-024-00859-x","DOIUrl":"10.1038/s42256-024-00859-x","url":null,"abstract":"Machine learning (ML) has revolutionized robotics by enhancing perception, adaptability, decision-making and more, enabling robots to work in complex scenarios beyond the capabilities of traditional approaches. However, the downsizing of robots to micro- and nanoscales introduces new challenges. For example, complexities in the actuation and locomotion of micro- and nanorobots defy traditional modelling methods, while control and navigation are complicated by strong environmental disruptions, and tracking in vivo encounters substantial noise interference. Recently, ML has also been shown to offer a promising avenue to tackle these complexities. Here we discuss how ML advances many crucial aspects of micro- and nanorobots, that is, in their design, actuation, locomotion, planning, tracking and navigation. Any application that can benefit from these fundamental advancements will be a potential beneficiary of this field, including micromanipulation, targeted delivery and therapy, bio-sensing, diagnosis and so on. This Review aims to provide an accessible and comprehensive survey for readers to quickly appreciate recent exciting accomplishments in ML for micro- and nanorobots. We also discuss potential issues and prospects of this burgeoning research direction. We hope this Review can foster interdisciplinary collaborations across robotics, computer science, material science and allied disciplines, to develop ML techniques that surmount fundamental challenges and further expand the application horizons of micro- and nanorobotics in biomedicine. Machine learning approaches in micro- and nanorobotics promise to overcome challenges encountered by applying traditional control methods at the microscopic scale. Lidong Yang et al. review this emerging area in robotics and discuss machine learning developments in design, actuation, locomotion, planning, tracking and navigation of microrobots.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141462446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-27DOI: 10.1038/s42256-024-00862-2
In the current wave of excitement about applying large vision–language models and generative AI to robotics, expectations are running high, but conquering real-world complexities remains challenging for robots.
{"title":"Will generative AI transform robotics?","authors":"","doi":"10.1038/s42256-024-00862-2","DOIUrl":"10.1038/s42256-024-00862-2","url":null,"abstract":"In the current wave of excitement about applying large vision–language models and generative AI to robotics, expectations are running high, but conquering real-world complexities remains challenging for robots.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00862-2.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141462592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-27DOI: 10.1038/s42256-024-00860-4
Osama Abdin, Philip M. Kim
Deep learning approaches have spurred substantial advances in the single-state prediction of biomolecular structures. The function of biomolecules is, however, dependent on the range of conformations they can assume. This is especially true for peptides, a highly flexible class of molecules that are involved in numerous biological processes and are of high interest as therapeutics. Here we introduce PepFlow, a transferable generative model that enables direct all-atom sampling from the allowable conformational space of input peptides. We train the model in a diffusion framework and subsequently use an equivalent flow to perform conformational sampling. To overcome the prohibitive cost of generalized all-atom modelling, we modularize the generation process and integrate a hypernetwork to predict sequence-specific network parameters. PepFlow accurately predicts peptide structures and effectively recapitulates experimental peptide ensembles at a fraction of the running time of traditional approaches. PepFlow can also be used to sample conformations that satisfy constraints such as macrocyclization. Modelling the different structures a peptide can assume is integral to understanding their function. The authors introduce PepFlow, a sequence-conditioned deep learning model that is shown to accurately and efficiently generate peptide conformations.
{"title":"Direct conformational sampling from peptide energy landscapes through hypernetwork-conditioned diffusion","authors":"Osama Abdin, Philip M. Kim","doi":"10.1038/s42256-024-00860-4","DOIUrl":"10.1038/s42256-024-00860-4","url":null,"abstract":"Deep learning approaches have spurred substantial advances in the single-state prediction of biomolecular structures. The function of biomolecules is, however, dependent on the range of conformations they can assume. This is especially true for peptides, a highly flexible class of molecules that are involved in numerous biological processes and are of high interest as therapeutics. Here we introduce PepFlow, a transferable generative model that enables direct all-atom sampling from the allowable conformational space of input peptides. We train the model in a diffusion framework and subsequently use an equivalent flow to perform conformational sampling. To overcome the prohibitive cost of generalized all-atom modelling, we modularize the generation process and integrate a hypernetwork to predict sequence-specific network parameters. PepFlow accurately predicts peptide structures and effectively recapitulates experimental peptide ensembles at a fraction of the running time of traditional approaches. PepFlow can also be used to sample conformations that satisfy constraints such as macrocyclization. Modelling the different structures a peptide can assume is integral to understanding their function. The authors introduce PepFlow, a sequence-conditioned deep learning model that is shown to accurately and efficiently generate peptide conformations.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141461885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-27DOI: 10.1038/s42256-024-00848-0
Alpha Renner, Lazar Supic, Andreea Danielescu, Giacomo Indiveri, Bruno A. Olshausen, Yulia Sandamirskaya, Friedrich T. Sommer, E. Paxon Frady
Analysing a visual scene by inferring the configuration of a generative model is widely considered the most flexible and generalizable approach to scene understanding. Yet, one major problem is the computational challenge of the inference procedure, involving a combinatorial search across object identities and poses. Here we propose a neuromorphic solution exploiting three key concepts: (1) a computational framework based on vector symbolic architectures (VSAs) with complex-valued vectors, (2) the design of hierarchical resonator networks to factorize the non-commutative transforms translation and rotation in visual scenes and (3) the design of a multi-compartment spiking phasor neuron model for implementing complex-valued resonator networks on neuromorphic hardware. The VSA framework uses vector binding operations to form a generative image model in which binding acts as the equivariant operation for geometric transformations. A scene can therefore be described as a sum of vector products, which can then be efficiently factorized by a resonator network to infer objects and their poses. The hierarchical resonator network features a partitioned architecture in which vector binding is equivariant for horizontal and vertical translation within one partition and for rotation and scaling within the other partition. The spiking neuron model allows mapping the resonator network onto efficient and low-power neuromorphic hardware. Our approach is demonstrated on synthetic scenes composed of simple two-dimensional shapes undergoing rigid geometric transformations and colour changes. A companion paper demonstrates the same approach in real-world application scenarios for machine vision and robotics. The inference procedure for analysing a visual scene presents a computational challenge. Renner, Supic and colleagues develop a neural network model, the hierarchical resonator, to determine the generative factors of variation of objects in simple scenes. The resonator was implemented on neuromorphic hardware, using a spike-timing code for complex numbers.
{"title":"Neuromorphic visual scene understanding with resonator networks","authors":"Alpha Renner, Lazar Supic, Andreea Danielescu, Giacomo Indiveri, Bruno A. Olshausen, Yulia Sandamirskaya, Friedrich T. Sommer, E. Paxon Frady","doi":"10.1038/s42256-024-00848-0","DOIUrl":"10.1038/s42256-024-00848-0","url":null,"abstract":"Analysing a visual scene by inferring the configuration of a generative model is widely considered the most flexible and generalizable approach to scene understanding. Yet, one major problem is the computational challenge of the inference procedure, involving a combinatorial search across object identities and poses. Here we propose a neuromorphic solution exploiting three key concepts: (1) a computational framework based on vector symbolic architectures (VSAs) with complex-valued vectors, (2) the design of hierarchical resonator networks to factorize the non-commutative transforms translation and rotation in visual scenes and (3) the design of a multi-compartment spiking phasor neuron model for implementing complex-valued resonator networks on neuromorphic hardware. The VSA framework uses vector binding operations to form a generative image model in which binding acts as the equivariant operation for geometric transformations. A scene can therefore be described as a sum of vector products, which can then be efficiently factorized by a resonator network to infer objects and their poses. The hierarchical resonator network features a partitioned architecture in which vector binding is equivariant for horizontal and vertical translation within one partition and for rotation and scaling within the other partition. The spiking neuron model allows mapping the resonator network onto efficient and low-power neuromorphic hardware. Our approach is demonstrated on synthetic scenes composed of simple two-dimensional shapes undergoing rigid geometric transformations and colour changes. A companion paper demonstrates the same approach in real-world application scenarios for machine vision and robotics. The inference procedure for analysing a visual scene presents a computational challenge. Renner, Supic and colleagues develop a neural network model, the hierarchical resonator, to determine the generative factors of variation of objects in simple scenes. The resonator was implemented on neuromorphic hardware, using a spike-timing code for complex numbers.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141462588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-24DOI: 10.1038/s42256-024-00853-3
Iksung Kang, Qinrong Zhang, Stella X. Yu, Na Ji
Widefield microscopy is widely used for non-invasive imaging of biological structures at subcellular resolution. When applied to a complex specimen, its image quality is degraded by sample-induced optical aberration. Adaptive optics can correct wavefront distortion and restore diffraction-limited resolution but require wavefront sensing and corrective devices, increasing system complexity and cost. Here we describe a self-supervised machine learning algorithm, CoCoA, that performs joint wavefront estimation and three-dimensional structural information extraction from a single-input three-dimensional image stack without the need for external training datasets. We implemented CoCoA for widefield imaging of mouse brain tissues and validated its performance with direct-wavefront-sensing-based adaptive optics. Importantly, we systematically explored and quantitatively characterized the limiting factors of CoCoA’s performance. Using CoCoA, we demonstrated in vivo widefield mouse brain imaging using machine learning-based adaptive optics. Incorporating coordinate-based neural representations and a forward physics model, the self-supervised scheme of CoCoA should be applicable to microscopy modalities in general. Adaptive optics (AO) corrects aberrations and restores resolution but requires specialized hardware. Kang et al. introduce a self-supervised AO method (CoCoA) for widefield microscopy, achieving in vivo mouse brain imaging without wavefront sensors.
{"title":"Coordinate-based neural representations for computational adaptive optics in widefield microscopy","authors":"Iksung Kang, Qinrong Zhang, Stella X. Yu, Na Ji","doi":"10.1038/s42256-024-00853-3","DOIUrl":"10.1038/s42256-024-00853-3","url":null,"abstract":"Widefield microscopy is widely used for non-invasive imaging of biological structures at subcellular resolution. When applied to a complex specimen, its image quality is degraded by sample-induced optical aberration. Adaptive optics can correct wavefront distortion and restore diffraction-limited resolution but require wavefront sensing and corrective devices, increasing system complexity and cost. Here we describe a self-supervised machine learning algorithm, CoCoA, that performs joint wavefront estimation and three-dimensional structural information extraction from a single-input three-dimensional image stack without the need for external training datasets. We implemented CoCoA for widefield imaging of mouse brain tissues and validated its performance with direct-wavefront-sensing-based adaptive optics. Importantly, we systematically explored and quantitatively characterized the limiting factors of CoCoA’s performance. Using CoCoA, we demonstrated in vivo widefield mouse brain imaging using machine learning-based adaptive optics. Incorporating coordinate-based neural representations and a forward physics model, the self-supervised scheme of CoCoA should be applicable to microscopy modalities in general. Adaptive optics (AO) corrects aberrations and restores resolution but requires specialized hardware. Kang et al. introduce a self-supervised AO method (CoCoA) for widefield microscopy, achieving in vivo mouse brain imaging without wavefront sensors.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":null,"pages":null},"PeriodicalIF":18.8,"publicationDate":"2024-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141448383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：