Hierarchical Action Embedding for Effective Autonomous Penetration Testing

Abstract

Penetration testing is an efficient technique in cyber-security. Using reinforcement learning to enhance the automation and accuracy of penetration testing is a promising approach. However, intricate network systems and the lack of a cyber-security knowledge base remain obstacles to this approach. Here, we propose a hierarchical action embedding that represents penetration testing action space. It helps improve the tactic of re-inforcement learning agents in complicated network scenarios by indicating the relation between actions using MITRE ATT&CK knowledge. The results of three testing configurations s how that the hierarchical action embedding improves the effectiveness of reinforcement learning compared to previous algorithms.