Machine learning methods for control of non-playable characters behaviour in multiplayer RPG

This article covers the problem of developing a control system for non-player characters in a multiplayer RPG. Commercial projects in the field of videogames and RPG (Role-Playing Game) projects in particular seldom use machine learning models for the implementation of character behaviour. The most common approach is to use primitive preprogrammed rules, or to implement a finite state machine. Such approaches ruin the immersion of playing with real creatures, since various predefined rules make the characters predictable. A good game AI is supposed to give the player an impression of interacting with real characters, that make various decisions, sometimes unpredictable. To achieve this goal, this article covers an approach with using various machine learning models in conjunction with a traditional finite state machine. A videogame developed earlier is used as the basis for problem solution. The article conducts an analysis of the existing works in the field of videogame AI. Next, the implementation of control system is described. This system utilizes a couple selected and successfully trained machine learning models. A multitude of models were tested, eventually a decision tree model and a neural network were selected, since they yielded the best results. The process of development and implementation of a control system involving machine learning models is then described. The approaches of teaching such models are described, and finally the achieved results are analyzed. To gauge the results, different models were compared against each other in test battles. The decision tree model showed results slightly better than the traditional finite state machine. Meanwhile, the neural network performed significantly better, beating other models far more often. Achieved results can be developed further, utilizing more complex models and improving training methods, which will result in even more sophisticated characters. Presence of such characters in the game will qualitatively improve the gameplay. Obtained system was integrated into the videogame, that may potentially become a commercial product.