Multi-agent policy transfer via task relationship modeling

IF 7.3 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS Science China Information Sciences Pub Date : 2024-07-22 DOI:10.1007/s11432-023-3862-1
Rongjun Qin, Feng Chen, Tonghan Wang, Lei Yuan, Xiaoran Wu, Yipeng Kang, Zongzhang Zhang, Chongjie Zhang, Yang Yu
{"title":"Multi-agent policy transfer via task relationship modeling","authors":"Rongjun Qin, Feng Chen, Tonghan Wang, Lei Yuan, Xiaoran Wu, Yipeng Kang, Zongzhang Zhang, Chongjie Zhang, Yang Yu","doi":"10.1007/s11432-023-3862-1","DOIUrl":null,"url":null,"abstract":"<p>Team adaptation to new cooperative tasks is a hallmark of human intelligence, which has yet to be fully realized in learning agents. Previous studies on multi-agent transfer learning have accommodated teams of different sizes but heavily relied on the generalization ability of neural networks for adapting to unseen tasks. We posit that the relationship among tasks provides key information for policy adaptation. We utilize this relationship for efficient transfer by attempting to discover and exploit the knowledge among tasks from different teams, proposing to learn an effect-based task representation as a common latent space among tasks, and using it to build an alternatively fixed training scheme. Herein, we demonstrate that task representation can capture the relationship among teams and generalize to unseen tasks. Thus, the proposed method helps transfer the learned cooperation knowledge to new tasks after training on a few source tasks. Furthermore, the learned transferred policies help solve tasks that are difficult to learn from scratch.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"5 1","pages":""},"PeriodicalIF":7.3000,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science China Information Sciences","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s11432-023-3862-1","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

Team adaptation to new cooperative tasks is a hallmark of human intelligence, which has yet to be fully realized in learning agents. Previous studies on multi-agent transfer learning have accommodated teams of different sizes but heavily relied on the generalization ability of neural networks for adapting to unseen tasks. We posit that the relationship among tasks provides key information for policy adaptation. We utilize this relationship for efficient transfer by attempting to discover and exploit the knowledge among tasks from different teams, proposing to learn an effect-based task representation as a common latent space among tasks, and using it to build an alternatively fixed training scheme. Herein, we demonstrate that task representation can capture the relationship among teams and generalize to unseen tasks. Thus, the proposed method helps transfer the learned cooperation knowledge to new tasks after training on a few source tasks. Furthermore, the learned transferred policies help solve tasks that are difficult to learn from scratch.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
通过任务关系建模实现多代理策略转移
团队适应新的合作任务是人类智能的一个标志,而这一点尚未在学习型代理中完全实现。以往关于多代理迁移学习的研究已经适应了不同规模的团队,但主要依赖神经网络的泛化能力来适应未见任务。我们认为,任务之间的关系为策略适应提供了关键信息。我们试图发现并利用来自不同团队的任务之间的知识,提出学习基于效果的任务表示法作为任务之间的共同潜空间,并利用它建立替代固定训练方案,从而利用这种关系实现高效迁移。在这里,我们证明了任务表示法可以捕捉团队之间的关系,并推广到未见过的任务中。因此,在对少数源任务进行训练后,所提出的方法有助于将学到的合作知识迁移到新任务中。此外,学习到的转移策略还能帮助解决难以从头开始学习的任务。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Science China Information Sciences
Science China Information Sciences COMPUTER SCIENCE, INFORMATION SYSTEMS-
CiteScore
12.60
自引率
5.70%
发文量
224
审稿时长
8.3 months
期刊介绍: Science China Information Sciences is a dedicated journal that showcases high-quality, original research across various domains of information sciences. It encompasses Computer Science & Technologies, Control Science & Engineering, Information & Communication Engineering, Microelectronics & Solid-State Electronics, and Quantum Information, providing a platform for the dissemination of significant contributions in these fields.
期刊最新文献
Weighted sum power maximization for STAR-RIS-aided SWIPT systems with nonlinear energy harvesting TSCompiler: efficient compilation framework for dynamic-shape models NeurDB: an AI-powered autonomous data system State and parameter identification of linearized water wave equation via adjoint method An STP look at logical blocking of finite state machines: formulation, detection, and search
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1