Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286481
J. Togelius, S. Karakovskiy, J. Koutník, J. Schmidhuber
We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning betterperforming strategies are discussed.
{"title":"Super mario evolution","authors":"J. Togelius, S. Karakovskiy, J. Koutník, J. Schmidhuber","doi":"10.1109/CIG.2009.5286481","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286481","url":null,"abstract":"We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning betterperforming strategies are discussed.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125682224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286475
Wojciech Jaśkowski, K. Krawiec
A two-player game given in the normal form of payoff matrix may be alternatively viewed as a list of the outcomes of binary interactions between two sets of entities, solutions and tests. The internal structure of such interactions may be characterized by an appropriately constructed coordinate system, which spatially arranges the solutions with respect to coordinates identified with tests, while preserving their mutual relations as given by the matrix. Of particular interest are coordinate systems of minimal size that give rise to the notion of dimension of a game. Following [1], we investigate such coordinate systems and relate their features to properties of partially ordered sets (posets), mostly to poset width and poset dimension. We propose an exact algorithm for constructing a minimal correct coordinate system and prove its correctness. In the experimental part, we compare the exact algorithm to the heuristics proposed in [1] on a sample of random payoff matrices of different sizes to demonstrate that the heuristics heavily overestimates the size of the minimal coordinate system. Finally, we show how the game dimension relate to the a priori dimension of a game.
{"title":"Formal analysis and algorithms for extracting coordinate systems of games","authors":"Wojciech Jaśkowski, K. Krawiec","doi":"10.1109/CIG.2009.5286475","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286475","url":null,"abstract":"A two-player game given in the normal form of payoff matrix may be alternatively viewed as a list of the outcomes of binary interactions between two sets of entities, solutions and tests. The internal structure of such interactions may be characterized by an appropriately constructed coordinate system, which spatially arranges the solutions with respect to coordinates identified with tests, while preserving their mutual relations as given by the matrix. Of particular interest are coordinate systems of minimal size that give rise to the notion of dimension of a game. Following [1], we investigate such coordinate systems and relate their features to properties of partially ordered sets (posets), mostly to poset width and poset dimension. We propose an exact algorithm for constructing a minimal correct coordinate system and prove its correctness. In the experimental part, we compare the exact algorithm to the heuristics proposed in [1] on a sample of random payoff matrices of different sizes to demonstrate that the heuristics heavily overestimates the size of the minimal coordinate system. Finally, we show how the game dimension relate to the a priori dimension of a game.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114617835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286479
Ben Cowley, D. Charles, Michaela M. Black, R. Hickey
We describe the results of a modeling methodology that combines the formal choice-system representation of decision theory with a human player-focused description of the behavioral features of game play in Pacman. This predictive player modeler addresses issues raised in previous work [1] and [2], to produce reliable accuracy. This paper focuses on using player-centric knowledge to reason about player behavior, utilizing a set of features which describe game-play to obtain quantitative data corresponding to qualitative behavioral concepts.
{"title":"Analyzing player behavior in Pacman using feature-driven decision theoretic predictive modeling","authors":"Ben Cowley, D. Charles, Michaela M. Black, R. Hickey","doi":"10.1109/CIG.2009.5286479","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286479","url":null,"abstract":"We describe the results of a modeling methodology that combines the formal choice-system representation of decision theory with a human player-focused description of the behavioral features of game play in Pacman. This predictive player modeler addresses issues raised in previous work [1] and [2], to produce reliable accuracy. This paper focuses on using player-centric knowledge to reason about player behavior, utilizing a set of features which describe game-play to obtain quantitative data corresponding to qualitative behavioral concepts.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"116 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125971598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286492
Luis Peña, Sascha Ossowski, J. Sánchez
Strategy games such as Warcraft™or UFO™franchises or RPG games like Never Winter Nights™or Baldur Gate™are successful blockbusters in video game industry. These games are based on battles simulated individual per individual. These type of games is a very interesting scenario to develop multilevel strategies or emergent behavior in multiagent systems.
{"title":"vBattle: A new framework to simulate medium-scale battles in individual-per-individual basis","authors":"Luis Peña, Sascha Ossowski, J. Sánchez","doi":"10.1109/CIG.2009.5286492","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286492","url":null,"abstract":"Strategy games such as Warcraft<sup>™</sup>or UFO<sup>™</sup>franchises or RPG games like Never Winter Nights<sup>™</sup>or Baldur Gate<sup>™</sup>are successful blockbusters in video game industry. These games are based on battles simulated individual per individual. These type of games is a very interesting scenario to develop multilevel strategies or emergent behavior in multiagent systems.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133181945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286452
Thomas Thompson, Fraser Milne, A. Andrew, J. Levine
In this paper we further explore the potential of a decentralised controller architecture that places multi-layer perceptrons within a subsumption hierarchy. Previous research exploring this approach proved successful in generating agents that could solve problems while coping with new reactive stimuli. However there were many unresolved questions that we wished to explore. In this paper we explore the use of our architecture with iterative training, increased controller modularity and conflicting goals. Results provide some interesting insights into the potential this method could have to agent designers.
{"title":"Improving control through subsumption in the EvoTanks domain","authors":"Thomas Thompson, Fraser Milne, A. Andrew, J. Levine","doi":"10.1109/CIG.2009.5286452","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286452","url":null,"abstract":"In this paper we further explore the potential of a decentralised controller architecture that places multi-layer perceptrons within a subsumption hierarchy. Previous research exploring this approach proved successful in generating agents that could solve problems while coping with new reactive stimuli. However there were many unresolved questions that we wished to explore. In this paper we explore the use of our architecture with iterative training, increased controller modularity and conflicting goals. Results provide some interesting insights into the potential this method could have to agent designers.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128894987","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286488
K. Jantke
Dramaturgy is the design of emotional experience. For digital games that are intended to tell a story, game design includes the anticipation of the players' experiences which shall lead to excitement, fascination, thrill, perhaps to immersion and flow, but not to boredom or confusion. What players will experience takes place over time. Events that happen are linearly ordered and those that may potentially happen form a partially orderded space-the game's story space. Dramaturgical game design is the anticipation of varying experiences and their thoughtful arrangment in a partially ordered space of events which players may possibly experience when playing the game. This may be seen as planning as demonstrated in an original game design case study. The approach particularly applies to those digital games that bear the potentials of telling a story. The inductive approach to AI planning is introduced into dramaturgical design.
{"title":"Dramaturgical Design of the Narrative in Digital Games: AI planning of conflicts in non-linear spaces of time","authors":"K. Jantke","doi":"10.1109/CIG.2009.5286488","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286488","url":null,"abstract":"Dramaturgy is the design of emotional experience. For digital games that are intended to tell a story, game design includes the anticipation of the players' experiences which shall lead to excitement, fascination, thrill, perhaps to immersion and flow, but not to boredom or confusion. What players will experience takes place over time. Events that happen are linearly ordered and those that may potentially happen form a partially orderded space-the game's story space. Dramaturgical game design is the anticipation of varying experiences and their thoughtful arrangment in a partially ordered space of events which players may possibly experience when playing the game. This may be seen as planning as demonstrated in an original game design case study. The approach particularly applies to those digital games that bear the potentials of telling a story. The inductive approach to AI planning is introduced into dramaturgical design.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129228576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286497
L. Galway, D. Charles, Michaela M. Black
This paper investigates the use of a dynamically generated exploration rate when using a reinforcement learning-based game agent controller within a dynamic digital game environment. Temporal Difference learning has been employed for the real-time gereration of reactive game agent behaviors within a variation of classic arcade game Pac-Man. Due to the dynamic nature of the game environment initial experiments made use of static, low value for the exploration rate utilized by action selection during learning. However, further experiments were conducted which dynamically generated a value for the exploration rate prior to learning using a genetic algorithm. Results obtained have shown that an improvement in the overall performance of the game agent controller may be achieved when a dynamic exploration rate is used. In particular, if the use of the genetic algorithm is controlled by a measure of the current performance of the game agent, further gains in the overall performance of the game agent may be achieved.
{"title":"Improving Temporal Difference game agent control using a dynamic exploration during control learning","authors":"L. Galway, D. Charles, Michaela M. Black","doi":"10.1109/CIG.2009.5286497","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286497","url":null,"abstract":"This paper investigates the use of a dynamically generated exploration rate when using a reinforcement learning-based game agent controller within a dynamic digital game environment. Temporal Difference learning has been employed for the real-time gereration of reactive game agent behaviors within a variation of classic arcade game Pac-Man. Due to the dynamic nature of the game environment initial experiments made use of static, low value for the exploration rate utilized by action selection during learning. However, further experiments were conducted which dynamically generated a value for the exploration rate prior to learning using a genetic algorithm. Results obtained have shown that an improvement in the overall performance of the game agent controller may be achieved when a dynamic exploration rate is used. In particular, if the use of the genetic algorithm is controlled by a measure of the current performance of the game agent, further gains in the overall performance of the game agent may be achieved.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121070808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286505
P. Hingston
The aim of the contest is to see if a computer game playing bot can play like a human. In the contest, bots try to convince a panel of expert judges that they are actually human players. Computers are superbly fast and accurate at playing games, but can they be programmed to be more fun to play - to play like you and me? People like to play against opponents who are like themselves - opponents with personality, who can surprise, who sometimes make mistakes, yet don't blindly make the same mistakes over and over. Can a computer be programmed to seem to have personality, fallibility and cunning?
{"title":"The 2K BotPrize","authors":"P. Hingston","doi":"10.1109/CIG.2009.5286505","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286505","url":null,"abstract":"The aim of the contest is to see if a computer game playing bot can play like a human. In the contest, bots try to convince a panel of expert judges that they are actually human players. Computers are superbly fast and accurate at playing games, but can they be programmed to be more fun to play - to play like you and me? People like to play against opponents who are like themselves - opponents with personality, who can surprise, who sometimes make mistakes, yet don't blindly make the same mistakes over and over. Can a computer be programmed to seem to have personality, fallibility and cunning?","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"117 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122442973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286464
Jorge Muñoz, G. Gutiérrez, A. Sanchis
This paper is an initial approach to create a controller for the game TORCS by learning how another controller or humans play the game. We used data obtained from two controllers and from one human player. The first controller is the winner of the WCCI 2008 Simulated Car Racing Competition, and the second one is a hand coded controller that performs a complete lap in all tracks. First, each kind of controller is imitated separately, then a mix of the data is used to create new controllers. The imitation is performed by means of training a feed forward neural network with the data, using the backpropagation algorithm for learning.
{"title":"Controller for TORCS created by imitation","authors":"Jorge Muñoz, G. Gutiérrez, A. Sanchis","doi":"10.1109/CIG.2009.5286464","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286464","url":null,"abstract":"This paper is an initial approach to create a controller for the game TORCS by learning how another controller or humans play the game. We used data obtained from two controllers and from one human player. The first controller is the winner of the WCCI 2008 Simulated Car Racing Competition, and the second one is a hand coded controller that performs a complete lap in all tracks. First, each kind of controller is imitated separately, then a mix of the data is used to create new controllers. The imitation is performed by means of training a feed forward neural network with the data, using the backpropagation algorithm for learning.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131153646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-07DOI: 10.1109/CIG.2009.5286469
David Robles, S. Lucas
Ms. Pac-Man is a challenging game for software agents that has been the focus of a significant amount of research. This paper describes the current state of a tree-search software agent that will be entered into the IEEE CIG 2009 screen-capture based Ms. Pac-Man software agent competition. While game-tree search is a staple technique for many games, this paper is, perhaps surprisingly, the first attempt we know of to apply it to Ms. Pac-Man. The approach we take is to expand a route-tree based on possible moves that the Ms. Pac-Man agent can take to depth 40, and evaluate which path is best using hand-coded heuristics. On a simulator of the game our agent has achieved a high score of 40,000, but only around 15,000 on the original game using a screen-capture interface. Our next steps are focussed on using an improved screen-capture system, and on using evolutionary algorithms to tune the parameters of the agent.
{"title":"A simple tree search method for playing Ms. Pac-Man","authors":"David Robles, S. Lucas","doi":"10.1109/CIG.2009.5286469","DOIUrl":"https://doi.org/10.1109/CIG.2009.5286469","url":null,"abstract":"Ms. Pac-Man is a challenging game for software agents that has been the focus of a significant amount of research. This paper describes the current state of a tree-search software agent that will be entered into the IEEE CIG 2009 screen-capture based Ms. Pac-Man software agent competition. While game-tree search is a staple technique for many games, this paper is, perhaps surprisingly, the first attempt we know of to apply it to Ms. Pac-Man. The approach we take is to expand a route-tree based on possible moves that the Ms. Pac-Man agent can take to depth 40, and evaluate which path is best using hand-coded heuristics. On a simulator of the game our agent has achieved a high score of 40,000, but only around 15,000 on the original game using a screen-capture interface. Our next steps are focussed on using an improved screen-capture system, and on using evolutionary algorithms to tune the parameters of the agent.","PeriodicalId":358795,"journal":{"name":"2009 IEEE Symposium on Computational Intelligence and Games","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133561841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：