首页 > 最新文献

IEEE Transactions on Computational Intelligence and AI in Games最新文献

英文 中文
Extending Real-Time Challenge Balancing to Multiplayer Games: A Study on Eco-Driving 将实时挑战平衡扩展到多人游戏:关于生态驾驶的研究
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2364258
H. Prendinger, Kamthorn Puntumapon, Marconi Madruga Filho
Multiplayer games are an important and popular game mode for networked players. Since games are played by a diverse audience, it is important to scale the difficulty, or challenge, according to the skill level of the players. However, current approaches to real-time challenge balancing (RCB) in games are only applicable to single-player scenarios. In multiplayer scenarios, players with different skill levels may be present in the same area, and hence adjusting the game difficulty to match the skill of one player may affect the other players in an undesirable way. To address this problem, we have previously developed a new approach based on distributed constraint optimization, which achieves the optimal challenge level for multiple players in real-time. The main contribution of this paper is an experiment that was performed with our new multiplayer real-time challenge balancing method applied to eco-driving. The results of the experiment suggest the effectiveness of RCB.
多人游戏是网络玩家的一种重要而流行的游戏模式。因为玩游戏的是各种各样的用户,所以根据玩家的技能水平来调整难度或挑战是很重要的。然而,当前游戏中的实时挑战平衡(RCB)方法只适用于单人游戏场景。在多人游戏场景中,不同技能水平的玩家可能出现在同一区域,因此调整游戏难度以匹配一个玩家的技能可能会以一种不受欢迎的方式影响到其他玩家。为了解决这个问题,我们之前开发了一种基于分布式约束优化的新方法,该方法可以实时实现多个玩家的最佳挑战级别。本文的主要贡献是将我们的新多人实时挑战平衡方法应用于生态驾驶的实验。实验结果表明了RCB的有效性。
{"title":"Extending Real-Time Challenge Balancing to Multiplayer Games: A Study on Eco-Driving","authors":"H. Prendinger, Kamthorn Puntumapon, Marconi Madruga Filho","doi":"10.1109/TCIAIG.2014.2364258","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2364258","url":null,"abstract":"Multiplayer games are an important and popular game mode for networked players. Since games are played by a diverse audience, it is important to scale the difficulty, or challenge, according to the skill level of the players. However, current approaches to real-time challenge balancing (RCB) in games are only applicable to single-player scenarios. In multiplayer scenarios, players with different skill levels may be present in the same area, and hence adjusting the game difficulty to match the skill of one player may affect the other players in an undesirable way. To address this problem, we have previously developed a new approach based on distributed constraint optimization, which achieves the optimal challenge level for multiple players in real-time. The main contribution of this paper is an experiment that was performed with our new multiplayer real-time challenge balancing method applied to eco-driving. The results of the experiment suggest the effectiveness of RCB.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"27-32"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2364258","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Prolog-Scripted Tactics Negotiation and Coordinated Team Actions for Counter-Strike Game Bots 《反恐精英》游戏机器人的序言脚本战术谈判和协调团队行动
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2331972
G. Jaskiewicz
κ-labs is a research project exploring the possibilities of the logic programming paradigm in bot behavior programming for first-person shooter (FPS) games. The focus of previous work was to make Prolog a usable tool for bot programming and a baseline for further extensions. This paper presents one such extension, which makes it possible to script tactics of the entire team of bots. The algorithm was tested by bot-to-bot computer tests and by running surveys among human players who volunteered to take part in the research. The results of the both tests are presented in this paper. The extension itself demonstrates the flexibility of the framework. Although the proposed method for defining team behaviors relies solely on the knowledge of the bot's designer, alternative approaches, which use rules that are obtained by computational techniques, can also be developed. Such approaches are also being investigated as part of the κ-labs project.
κ-labs是一个研究项目,旨在探索逻辑编程范式在第一人称射击游戏(FPS)机器人行为编程中的可能性。以前的工作重点是使Prolog成为一个可用的机器人编程工具,并为进一步扩展奠定基础。本文提出了一个这样的扩展,它使整个机器人团队的脚本战术成为可能。该算法通过机器人对机器人的计算机测试和对自愿参加研究的人类玩家的调查进行了测试。本文给出了两种试验的结果。扩展本身展示了框架的灵活性。尽管所提出的定义团队行为的方法完全依赖于机器人设计者的知识,但也可以开发使用通过计算技术获得的规则的替代方法。这些方法也正在作为κ-labs项目的一部分进行研究。
{"title":"Prolog-Scripted Tactics Negotiation and Coordinated Team Actions for Counter-Strike Game Bots","authors":"G. Jaskiewicz","doi":"10.1109/TCIAIG.2014.2331972","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2331972","url":null,"abstract":"κ-labs is a research project exploring the possibilities of the logic programming paradigm in bot behavior programming for first-person shooter (FPS) games. The focus of previous work was to make Prolog a usable tool for bot programming and a baseline for further extensions. This paper presents one such extension, which makes it possible to script tactics of the entire team of bots. The algorithm was tested by bot-to-bot computer tests and by running surveys among human players who volunteered to take part in the research. The results of the both tests are presented in this paper. The extension itself demonstrates the flexibility of the framework. Although the proposed method for defining team behaviors relies solely on the knowledge of the bot's designer, alternative approaches, which use rules that are obtained by computational techniques, can also be developed. Such approaches are also being investigated as part of the κ-labs project.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"20 1","pages":"82-88"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2331972","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Intelligent Game Engine for Rehabilitation (IGER) 智能康复游戏引擎(IGER)
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2368392
Michele Pirovano, R. Mainetti, G. Baud-Bovy, P. Lanzi, N. A. Borghese
Computer games are a promising tool to support intensive rehabilitation. However, at present, they do not incorporate the supervision provided by a real therapist and do not allow safe and effective use at a patient's home. We show how specifically tailored computational intelligence based techniques allow extending exergames with functionalities that make rehabilitation at home effective and safe. The main function is in monitoring the correctness of motion, which is fundamental in avoiding developing wrong motion patterns, making rehabilitation more harmful than effective. Fuzzy systems enable us to capture the knowledge of the therapist and to provide real-time feedback of the patient's motion quality with a novel informative color coding applied to the patient's avatar. This feedback is complemented with a therapist avatar that, in extreme cases, explains the correct way to carry out the movements required by the exergames. The avatar also welcomes the patient and summarizes the therapy results to him/her. Text to speech and simple animation improve the engagement. Another important element is adaptation. Only the proper level of challenge exercises can be both effective and safe. For this reason exergames can be fully configured by therapists in terms of speed, range of motion, or accuracy. These parameters are then tuned during exercise to the patient's performance through a Bayesian framework that also takes into account input from the therapist. A log of all the interaction data is stored for clinicians to assess and tune the therapy, and to advise patients. All this functionality has been added to a classical game engine that is extended to embody a virtual therapist aimed at supervising the motion, which is the final goal of the exergames for rehabilitation. This approach can be of broad interest in the serious games domain. Preliminary results with patients and therapists suggest that the approach can maintain a proper challenge level while keeping the patient motivated, safe, and supervised.
电脑游戏是一种很有前途的辅助强化康复的工具。然而,目前,它们没有纳入真正的治疗师提供的监督,也不允许在患者家中安全有效地使用。我们展示了专门定制的基于计算智能的技术如何允许扩展exergames的功能,使家庭康复有效和安全。主要功能是监测运动的正确性,这是避免发展错误的运动模式的基础,使康复弊大于利。模糊系统使我们能够捕捉治疗师的知识,并通过应用于患者化身的新颖信息颜色编码提供患者运动质量的实时反馈。这种反馈与治疗师化身相辅相成,在极端情况下,治疗师化身会解释执行游戏所需动作的正确方法。化身也欢迎病人,并向他/她总结治疗结果。文本到语音和简单的动画提高了用户粘性。另一个重要因素是适应。只有适当水平的挑战练习才能既有效又安全。出于这个原因,治疗师可以根据速度、运动范围或准确性对游戏进行充分配置。然后,这些参数在锻炼过程中通过贝叶斯框架(也考虑到治疗师的输入)根据患者的表现进行调整。所有相互作用数据的日志被存储,供临床医生评估和调整治疗,并建议患者。所有这些功能都被添加到一个经典的游戏引擎中,该引擎扩展为一个虚拟治疗师,旨在监督运动,这是康复游戏的最终目标。这种方法可能会引起严肃游戏领域的广泛兴趣。患者和治疗师的初步结果表明,该方法可以保持适当的挑战水平,同时保持患者的积极性,安全性和监督。
{"title":"Intelligent Game Engine for Rehabilitation (IGER)","authors":"Michele Pirovano, R. Mainetti, G. Baud-Bovy, P. Lanzi, N. A. Borghese","doi":"10.1109/TCIAIG.2014.2368392","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2368392","url":null,"abstract":"Computer games are a promising tool to support intensive rehabilitation. However, at present, they do not incorporate the supervision provided by a real therapist and do not allow safe and effective use at a patient's home. We show how specifically tailored computational intelligence based techniques allow extending exergames with functionalities that make rehabilitation at home effective and safe. The main function is in monitoring the correctness of motion, which is fundamental in avoiding developing wrong motion patterns, making rehabilitation more harmful than effective. Fuzzy systems enable us to capture the knowledge of the therapist and to provide real-time feedback of the patient's motion quality with a novel informative color coding applied to the patient's avatar. This feedback is complemented with a therapist avatar that, in extreme cases, explains the correct way to carry out the movements required by the exergames. The avatar also welcomes the patient and summarizes the therapy results to him/her. Text to speech and simple animation improve the engagement. Another important element is adaptation. Only the proper level of challenge exercises can be both effective and safe. For this reason exergames can be fully configured by therapists in terms of speed, range of motion, or accuracy. These parameters are then tuned during exercise to the patient's performance through a Bayesian framework that also takes into account input from the therapist. A log of all the interaction data is stored for clinicians to assess and tune the therapy, and to advise patients. All this functionality has been added to a classical game engine that is extended to embody a virtual therapist aimed at supervising the motion, which is the final goal of the exergames for rehabilitation. This approach can be of broad interest in the serious games domain. Preliminary results with patients and therapists suggest that the approach can maintain a proper challenge level while keeping the patient motivated, safe, and supervised.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"43-55"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2368392","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 41
How to Run a Successful Game-Based AI Competition 如何成功举办基于游戏的AI竞赛
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2365470
J. Togelius
Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.
基于游戏的竞赛通常用于游戏社区中的计算智能(CI)和人工智能(AI),以测试算法并吸引新的研究人员。虽然许多比赛都是基于不同的游戏而组织的,但这些比赛的成功程度却参差不齐。这篇短文是竞赛组织者和有抱负的竞赛组织者的自助论文。在分析了近年来一些比赛的命运后,列出了一些可能影响比赛成败的因素,并提出了一套具体的建议。我们还讨论了如何编写基于游戏的AI竞赛,以及我们最终可以从中学到什么。
{"title":"How to Run a Successful Game-Based AI Competition","authors":"J. Togelius","doi":"10.1109/TCIAIG.2014.2365470","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2365470","url":null,"abstract":"Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"95-100"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2365470","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592737","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 34
Online Adaptable Learning Rates for the Game Connect-4 游戏Connect-4的在线适应性学习率
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2367105
Samineh Bagheri, Markus Thill, P. Koch, W. Konen
Learning board games by self-play has a long tradition in computational intelligence for games. Based on Tesauro's seminal success with TD-Gammon in 1994, many successful agents use temporal difference learning today. But in order to be successful with temporal difference learning on game tasks, often a careful selection of features and a large number of training games is necessary. Even for board games of moderate complexity like Connect-4, we found in previous work that a very rich initial feature set and several millions of game plays are required. In this work we investigate different approaches of online-adaptable learning rates like Incremental Delta Bar Delta (IDBD) or temporal coherence learning (TCL) whether they have the potential to speed up learning for such a complex task. We propose a new variant of TCL with geometric step size changes. We compare those algorithms with several other state-of-the-art learning rate adaptation algorithms and perform a case study on the sensitivity with respect to their meta parameters. We show that in this set of learning algorithms those with geometric step size changes outperform those other algorithms with constant step size changes. Algorithms with nonlinear output functions are slightly better than linear ones. Algorithms with geometric step size changes learn faster by a factor of 4 as compared to previously published results on the task Connect-4.
通过自玩学习棋类游戏在游戏计算智能领域有着悠久的传统。基于Tesauro在1994年对TD-Gammon的开创性成功,今天许多成功的代理都使用了时间差异学习。但是为了在游戏任务中成功地进行时间差异学习,通常需要仔细选择特征和大量的训练游戏。即使是像《Connect-4》这样中等复杂度的桌面游戏,我们也需要非常丰富的初始功能集和数百万的游戏玩法。在这项工作中,我们研究了不同的在线适应学习率方法,如增量增量条形增量(IDBD)或时间相干学习(TCL),它们是否有可能加速这种复杂任务的学习。我们提出了一个具有几何步长变化的TCL的新变体。我们将这些算法与其他几种最先进的学习率自适应算法进行比较,并对其元参数的敏感性进行案例研究。我们表明,在这组学习算法中,那些具有几何步长变化的算法优于那些具有恒定步长变化的算法。具有非线性输出函数的算法略优于线性输出函数的算法。与先前发表的关于Connect-4任务的结果相比,具有几何步长变化的算法的学习速度提高了4倍。
{"title":"Online Adaptable Learning Rates for the Game Connect-4","authors":"Samineh Bagheri, Markus Thill, P. Koch, W. Konen","doi":"10.1109/TCIAIG.2014.2367105","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2367105","url":null,"abstract":"Learning board games by self-play has a long tradition in computational intelligence for games. Based on Tesauro's seminal success with TD-Gammon in 1994, many successful agents use temporal difference learning today. But in order to be successful with temporal difference learning on game tasks, often a careful selection of features and a large number of training games is necessary. Even for board games of moderate complexity like Connect-4, we found in previous work that a very rich initial feature set and several millions of game plays are required. In this work we investigate different approaches of online-adaptable learning rates like Incremental Delta Bar Delta (IDBD) or temporal coherence learning (TCL) whether they have the potential to speed up learning for such a complex task. We propose a new variant of TCL with geometric step size changes. We compare those algorithms with several other state-of-the-art learning rate adaptation algorithms and perform a case study on the sensitivity with respect to their meta parameters. We show that in this set of learning algorithms those with geometric step size changes outperform those other algorithms with constant step size changes. Algorithms with nonlinear output functions are slightly better than linear ones. Algorithms with geometric step size changes learn faster by a factor of 4 as compared to previously published results on the task Connect-4.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"33-42"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2367105","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 25
Predicting Dominance Rankings for Score-Based Games 预测基于分数的游戏的优势排名
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2346242
Spyridon Samothrakis, Diego Perez Liebana, S. Lucas, Philipp Rohlfshagen
Game competitions may involve different player roles and be score-based rather than win/loss based. This raises the issue of how best to draw opponents for matches in ongoing competitions, and how best to rank the players in each role. An example is the Ms Pac-Man versus Ghosts Competition which requires competitors to develop software controllers to take charge of the game's protagonists: participants may develop software controllers for either or both Ms Pac-Man and the team of four ghosts. In this paper, we compare two ranking schemes for win-loss games, Bayes Elo and Glicko. We convert the game into one of win-loss (“dominance”) by matching controllers of identical type against the same opponent in a series of pair-wise comparisons. This implicitly creates a “solution concept” as to what a constitutes a good player. We analyze how many games are needed under two popular ranking algorithms, Glicko and Bayes Elo, before one can infer the strength of the players, according to our proposed solution concept, without performing an exhaustive evaluation. We show that Glicko should be the method of choice for online score-based game competitions.
游戏竞争可能涉及不同的玩家角色,基于分数而非输赢。这就引出了如何在正在进行的比赛中最好地吸引对手,以及如何在每个角色中最好地对玩家进行排名的问题。例如,《吃豆人小姐与幽灵竞赛》要求参赛者开发软件控制器来控制游戏主角:参与者可以为吃豆人小姐和四个幽灵组成的团队开发软件控制器。在本文中,我们比较了两种输赢博弈的排名方案,Bayes Elo和Glicko。我们通过在一系列成对比较中匹配相同类型的控制器来对抗相同的对手,从而将游戏转化为一种输赢(“支配”)。这就隐含地创造了一个“解决方案概念”,即如何构成优秀玩家。我们分析了两种流行的排名算法(Glicko和Bayes Elo)下需要多少场比赛,然后根据我们提出的解决方案概念推断出玩家的实力,而无需执行详尽的评估。我们认为Glicko应该成为基于分数的在线游戏竞赛的首选方法。
{"title":"Predicting Dominance Rankings for Score-Based Games","authors":"Spyridon Samothrakis, Diego Perez Liebana, S. Lucas, Philipp Rohlfshagen","doi":"10.1109/TCIAIG.2014.2346242","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2346242","url":null,"abstract":"Game competitions may involve different player roles and be score-based rather than win/loss based. This raises the issue of how best to draw opponents for matches in ongoing competitions, and how best to rank the players in each role. An example is the Ms Pac-Man versus Ghosts Competition which requires competitors to develop software controllers to take charge of the game's protagonists: participants may develop software controllers for either or both Ms Pac-Man and the team of four ghosts. In this paper, we compare two ranking schemes for win-loss games, Bayes Elo and Glicko. We convert the game into one of win-loss (“dominance”) by matching controllers of identical type against the same opponent in a series of pair-wise comparisons. This implicitly creates a “solution concept” as to what a constitutes a good player. We analyze how many games are needed under two popular ranking algorithms, Glicko and Bayes Elo, before one can infer the strength of the players, according to our proposed solution concept, without performing an exhaustive evaluation. We show that Glicko should be the method of choice for online score-based game competitions.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"1-12"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2346242","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
Reinforcement Learning in Video Games Using Nearest Neighbor Interpolation and Metric Learning 基于最近邻插值和度量学习的电子游戏强化学习
Q2 Computer Science Pub Date : 2016-03-01 DOI: 10.1109/TCIAIG.2014.2369345
Matthew S. Emigh, E. Kriminger, A. Brockmeier, J. Príncipe, P. Pardalos
Reinforcement learning (RL) has had mixed success when applied to games. Large state spaces and the curse of dimensionality have limited the ability for RL techniques to learn to play complex games in a reasonable length of time. We discuss a modification of Q-learning to use nearest neighbor states to exploit previous experience in the early stages of learning. A weighting on the state features is learned using metric learning techniques, such that neighboring states represent similar game situations. Our method is tested on the arcade game Frogger, and it is shown that some of the effects of the curse of dimensionality can be mitigated.
强化学习(RL)在应用于游戏时取得了不同程度的成功。大的状态空间和维度的诅咒限制了强化学习技术在合理时间内学习复杂游戏的能力。我们讨论了对q学习的一种修改,即在学习的早期阶段使用最近邻状态来利用先前的经验。使用度量学习技术来学习状态特征的权重,这样相邻的状态就代表了类似的游戏情境。我们的方法在街机游戏《青蛙过河》上进行了测试,结果表明,维度诅咒的一些影响是可以减轻的。
{"title":"Reinforcement Learning in Video Games Using Nearest Neighbor Interpolation and Metric Learning","authors":"Matthew S. Emigh, E. Kriminger, A. Brockmeier, J. Príncipe, P. Pardalos","doi":"10.1109/TCIAIG.2014.2369345","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2369345","url":null,"abstract":"Reinforcement learning (RL) has had mixed success when applied to games. Large state spaces and the curse of dimensionality have limited the ability for RL techniques to learn to play complex games in a reasonable length of time. We discuss a modification of Q-learning to use nearest neighbor states to exploit previous experience in the early stages of learning. A weighting on the state features is learned using metric learning techniques, such that neighboring states represent similar game situations. Our method is tested on the arcade game Frogger, and it is shown that some of the effects of the curse of dimensionality can be mitigated.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"56-66"},"PeriodicalIF":0.0,"publicationDate":"2016-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2369345","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 25
Competitive Algorithms for Coevolving Both Game Content and AI. A Case Study: Planet Wars 游戏内容和AI共同进化的竞争算法案例研究:《星球大战》
Q2 Computer Science Pub Date : 2016-01-01 DOI: 10.1109/TCIAIG.2015.2499281
M. Nogueira, C. Cotta, Antonio J. Fernández
The classical approach of Competitive Coevolution (CC) applied in games tries to exploit an arms race between coevolving populations that belong to the same species (or at least to the same biotic niche), namely strategies, rules, tracks for racing, or any other. This paper proposes the co-evolution of entities belonging to different realms (namely biotic and abiotic) via a competitive approach. More precisely, we aim to coevolutionarily optimize both virtual players and game content. From a general perspective, our proposal can be viewed as a method of procedural content generation combined with a technique for generating game Artificial Intelligence (AI). This approach can not only help game designers in game creation but also generate content personalized to both specific players’ profiles and game designer’s objectives (e.g., create content that favors novice players over skillful players). As a case study we use Planet Wars, the Real Time Strategy (RTS) game associated with the 2010 Google AI Challenge contest, and demonstrate (via an empirical study) the validity of our approach.
在游戏中应用的竞争性共同进化(CC)的经典方法试图利用属于同一物种(或至少属于同一生物生态位)的共同进化群体之间的军备竞赛,即策略,规则,赛道或其他任何东西。本文提出了不同领域(即生物和非生物)的实体通过竞争的方式共同进化。更准确地说,我们的目标是共同进化优化虚拟玩家和游戏内容。从一般角度来看,我们的建议可以被视为一种程序内容生成方法与生成游戏人工智能(AI)的技术相结合。这种方法不仅可以帮助游戏设计师进行游戏创作,还可以根据特定玩家的个人资料和游戏设计师的目标生成个性化的内容(游戏邦注:例如,比起熟练玩家,创造更有利于新手玩家的内容)。作为案例研究,我们使用了《星球大战》,这是一款与2010 b谷歌AI挑战赛相关的即时战略(RTS)游戏,并通过实证研究证明了我们方法的有效性。
{"title":"Competitive Algorithms for Coevolving Both Game Content and AI. A Case Study: Planet Wars","authors":"M. Nogueira, C. Cotta, Antonio J. Fernández","doi":"10.1109/TCIAIG.2015.2499281","DOIUrl":"https://doi.org/10.1109/TCIAIG.2015.2499281","url":null,"abstract":"The classical approach of Competitive Coevolution (CC) applied in games tries to exploit an arms race between coevolving populations that belong to the same species (or at least to the same biotic niche), namely strategies, rules, tracks for racing, or any other. This paper proposes the co-evolution of entities belonging to different realms (namely biotic and abiotic) via a competitive approach. More precisely, we aim to coevolutionarily optimize both virtual players and game content. From a general perspective, our proposal can be viewed as a method of procedural content generation combined with a technique for generating game Artificial Intelligence (AI). This approach can not only help game designers in game creation but also generate content personalized to both specific players’ profiles and game designer’s objectives (e.g., create content that favors novice players over skillful players). As a case study we use Planet Wars, the Real Time Strategy (RTS) game associated with the 2010 Google AI Challenge contest, and demonstrate (via an empirical study) the validity of our approach.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"8 1","pages":"325-337"},"PeriodicalIF":0.0,"publicationDate":"2016-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2015.2499281","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62593230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Clustering Game Behavior Data 聚类游戏行为数据
Q2 Computer Science Pub Date : 2015-12-04 DOI: 10.1109/TCIAIG.2014.2376982
C. Bauckhage, Anders Drachen, R. Sifa
Recent years have seen a deluge of behavioral data from players hitting the game industry. Reasons for this data surge are many and include the introduction of new business models, technical innovations, the popularity of online games, and the increasing persistence of games. Irrespective of the causes, the proliferation of behavioral data poses the problem of how to derive insights therefrom. Behavioral data sets can be large, time-dependent and high-dimensional. Clustering offers a way to explore such data and to discover patterns that can reduce the overall complexity of the data. Clustering and other techniques for player profiling and play style analysis have, therefore, become popular in the nascent field of game analytics. However, the proper use of clustering techniques requires expertise and an understanding of games is essential to evaluate results. With this paper, we address game data scientists and present a review and tutorial focusing on the application of clustering techniques to mine behavioral game data. Several algorithms are reviewed and examples of their application shown. Key topics such as feature normalization are discussed and open problems in the context of game analytics are pointed out.
近年来,游戏行业出现了大量来自玩家的行为数据。这种数据激增的原因有很多,包括新业务模式的引入、技术创新、在线游戏的普及以及游戏的持久性增加。不管原因是什么,行为数据的激增带来了如何从中获得见解的问题。行为数据集可以是大的、时间依赖的和高维的。集群提供了一种探索此类数据和发现可以降低数据总体复杂性的模式的方法。因此,聚类和其他用于玩家分析和游戏风格分析的技术在游戏分析的新兴领域变得流行起来。然而,正确使用聚类技术需要专业知识和对游戏的理解,这对评估结果至关重要。在本文中,我们讨论了游戏数据科学家,并介绍了一篇关于聚类技术在挖掘行为游戏数据中的应用的综述和教程。介绍了几种算法,并给出了应用实例。讨论了游戏分析中的关键话题,如特征归一化,并指出了游戏分析中的开放性问题。
{"title":"Clustering Game Behavior Data","authors":"C. Bauckhage, Anders Drachen, R. Sifa","doi":"10.1109/TCIAIG.2014.2376982","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2376982","url":null,"abstract":"Recent years have seen a deluge of behavioral data from players hitting the game industry. Reasons for this data surge are many and include the introduction of new business models, technical innovations, the popularity of online games, and the increasing persistence of games. Irrespective of the causes, the proliferation of behavioral data poses the problem of how to derive insights therefrom. Behavioral data sets can be large, time-dependent and high-dimensional. Clustering offers a way to explore such data and to discover patterns that can reduce the overall complexity of the data. Clustering and other techniques for player profiling and play style analysis have, therefore, become popular in the nascent field of game analytics. However, the proper use of clustering techniques requires expertise and an understanding of games is essential to evaluate results. With this paper, we address game data scientists and present a review and tutorial focusing on the application of clustering techniques to mine behavioral game data. Several algorithms are reviewed and examples of their application shown. Key topics such as feature normalization are discussed and open problems in the context of game analytics are pointed out.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"7 1","pages":"266-278"},"PeriodicalIF":0.0,"publicationDate":"2015-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2376982","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 98
Temporal Game Challenge Tailoring 时间游戏挑战裁剪
Q2 Computer Science Pub Date : 2015-12-01 DOI: 10.1109/TCIAIG.2014.2342934
Alexander Zook, Mark O. Riedl
Digital games often center on a series of challenges designed to vary in difficulty over the course of the game. Designers, however, lack ways to ensure challenges are suitably tailored to the abilities of each game player, often resulting in player boredom or frustration. Challenge tailoring refers to the general problem of matching designer-intended challenges to player abilities. We present an approach to predict temporal player performance and select appropriate content to solve the challenge tailoring problem. Our temporal collaborative filtering approach-tensor factorization-captures similarities among players and the challenges they face to predict player performance on unseen, future challenges. Tensor factorization accounts for varying player abilities over time and is a generic approach capable of modeling many kinds of players. We use constraint solving to optimize content selection to match player skills to a designer-specified level of performance and present a model-performance curves-for designers to specify desired, temporally changing player behavior. We evaluate our approach in a role-playing game through two empirical studies of humans and one study using simulated agents. Our studies show tensor factorization scales in multiple game-relevant data dimensions, can be used for modestly effective game adaptation, and can predict divergent player learning trends.
数字游戏通常以一系列挑战为中心,在游戏过程中设计不同难度的挑战。然而,设计师缺乏确保挑战适合每个玩家能力的方法,这通常会导致玩家感到无聊或受挫。挑战裁剪指的是将设计师设计的挑战与玩家能力相匹配的问题。我们提出了一种方法来预测玩家的时间表现,并选择合适的内容来解决挑战裁剪问题。我们的时间协同过滤方法——张量分解——捕捉玩家之间的相似性和他们面临的挑战,以预测玩家在未知的未来挑战中的表现。张量分解解释了玩家能力随时间的变化,是一种能够模拟多种玩家的通用方法。我们使用约束求解来优化内容选择,将玩家技能与设计师指定的性能水平相匹配,并呈现一个模型-性能曲线-供设计师指定所需的,暂时改变的玩家行为。我们通过两项人类的实证研究和一项使用模拟代理的研究来评估我们在角色扮演游戏中的方法。我们的研究表明,在多个游戏相关的数据维度中,张量分解尺度可以用于适度有效的游戏适应,并且可以预测不同的玩家学习趋势。
{"title":"Temporal Game Challenge Tailoring","authors":"Alexander Zook, Mark O. Riedl","doi":"10.1109/TCIAIG.2014.2342934","DOIUrl":"https://doi.org/10.1109/TCIAIG.2014.2342934","url":null,"abstract":"Digital games often center on a series of challenges designed to vary in difficulty over the course of the game. Designers, however, lack ways to ensure challenges are suitably tailored to the abilities of each game player, often resulting in player boredom or frustration. Challenge tailoring refers to the general problem of matching designer-intended challenges to player abilities. We present an approach to predict temporal player performance and select appropriate content to solve the challenge tailoring problem. Our temporal collaborative filtering approach-tensor factorization-captures similarities among players and the challenges they face to predict player performance on unseen, future challenges. Tensor factorization accounts for varying player abilities over time and is a generic approach capable of modeling many kinds of players. We use constraint solving to optimize content selection to match player skills to a designer-specified level of performance and present a model-performance curves-for designers to specify desired, temporally changing player behavior. We evaluate our approach in a role-playing game through two empirical studies of humans and one study using simulated agents. Our studies show tensor factorization scales in multiple game-relevant data dimensions, can be used for modestly effective game adaptation, and can predict divergent player learning trends.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":"7 1","pages":"336-346"},"PeriodicalIF":0.0,"publicationDate":"2015-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2014.2342934","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62592330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
期刊
IEEE Transactions on Computational Intelligence and AI in Games
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1