The Design of ϵ-Optimal Strategy for Two-Person Zero-Sum Markov Games

IF 2.4 Q2 AUTOMATION & CONTROL SYSTEMS IEEE Control Systems Letters Pub Date : 2024-10-04 DOI:10.1109/LCSYS.2024.3474057
Kaiyun Xie;Junlin Xiong
{"title":"The Design of ϵ-Optimal Strategy for Two-Person Zero-Sum Markov Games","authors":"Kaiyun Xie;Junlin Xiong","doi":"10.1109/LCSYS.2024.3474057","DOIUrl":null,"url":null,"abstract":"This letter focuses on designing approximate Nash strategies for the two-person zero-sum Markov game. Using the receding horizon method, the \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n-optimal strategies are designed to approximate Nash strategies by executing finite Gauss-Seidel iterations. The relationship between the approximation value of \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n and the number of iterations is also analyzed. Additionally, the \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n-optimal strategies are designed for two scenarios with imprecise parameters. For scenarios with imprecise values, the value of \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n is determined based on the errors between imprecise and iteration values. It provides a theoretical basis for efficiently designing \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n-optimal strategies using heuristic algorithms or approximate dynamic programming. For scenarios with imprecise transition probabilities, the value of \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n is determined based on the errors between the estimated and practical transition probabilities. It enables the use of pattern recognition technology or other methods to estimate practical transition probabilities for designing \n<inline-formula> <tex-math>$\\epsilon $ </tex-math></inline-formula>\n-optimal strategies.","PeriodicalId":37235,"journal":{"name":"IEEE Control Systems Letters","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2024-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Control Systems Letters","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10705104/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

This letter focuses on designing approximate Nash strategies for the two-person zero-sum Markov game. Using the receding horizon method, the $\epsilon $ -optimal strategies are designed to approximate Nash strategies by executing finite Gauss-Seidel iterations. The relationship between the approximation value of $\epsilon $ and the number of iterations is also analyzed. Additionally, the $\epsilon $ -optimal strategies are designed for two scenarios with imprecise parameters. For scenarios with imprecise values, the value of $\epsilon $ is determined based on the errors between imprecise and iteration values. It provides a theoretical basis for efficiently designing $\epsilon $ -optimal strategies using heuristic algorithms or approximate dynamic programming. For scenarios with imprecise transition probabilities, the value of $\epsilon $ is determined based on the errors between the estimated and practical transition probabilities. It enables the use of pattern recognition technology or other methods to estimate practical transition probabilities for designing $\epsilon $ -optimal strategies.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
两人零和马尔可夫博弈的ϵ-最优策略设计
这封信的重点是设计两人零和马尔可夫博弈的近似纳什策略。利用后退视界法,通过执行有限的高斯-赛德尔迭代,设计了$epsilon $最优策略来近似纳什策略。同时还分析了 $\epsilon $ 的近似值与迭代次数之间的关系。此外,还针对两种参数不精确的情况设计了$\epsilon $最优策略。对于参数值不精确的情况,$epsilon $ 的值是根据不精确值和迭代值之间的误差确定的。它为使用启发式算法或近似动态编程有效设计 $\epsilon $ 最佳策略提供了理论基础。对于过渡概率不精确的情况,$epsilon $ 的值是根据估计过渡概率和实际过渡概率之间的误差来确定的。它可以使用模式识别技术或其他方法来估计实际过渡概率,从而设计出最优策略。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
IEEE Control Systems Letters
IEEE Control Systems Letters Mathematics-Control and Optimization
CiteScore
4.40
自引率
13.30%
发文量
471
期刊最新文献
Rationality of Learning Algorithms in Repeated Normal-Form Games Impact of Opinion on Disease Transmission With Waterborne Pathogen and Stubborn Community Numerical and Lyapunov-Based Investigation of the Effect of Stenosis on Blood Transport Stability Using a Control-Theoretic PDE Model of Cardiovascular Flow Almost Sure Convergence and Non-Asymptotic Concentration Bounds for Stochastic Mirror Descent Algorithm Opinion Dynamics With Set-Based Confidence: Convergence Criteria and Periodic Solutions
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1