A Deep Reinforcement Learning Technique for PNPSC Net Player Strategies

Abstract

Petri Nets with Players, Strategies, and Costs (PNPSC) is an extension of Petri nets specifically designed to model cyberattacks. The PNPSC formalism includes a representation of the strategies for the competing "players," i.e., the attacker and defender. Developing well-performing strategies for players in PNPSC nets is challenging for both game tree and reinforcement learning algorithms. This paper presents a method of modeling the PNPSC net player strategies as a game tree and using a combination of Monte Carlo Tree Search (MCTS) and deep reinforcement learning to effectively improve the players' strategies. The performance of this combination method is compared with standard action selection with the deep Q-learning algorithm.