ROPK++: An enhanced ROP attack detection framework for Linux operating system

Abstract

A major security challenge for today's computer software is buffer overflow and other memory-related attacks. To exploit buffer overflow vulnerabilities in presence of the classical defense mechanisms such as write-xor-execute, attackers take advantage of code reuse attacks. The code reuse attacks allow an adversary to perform arbitrary operations on a victim's system by constructing a chain of small code sequences called gadgets that are present in vulnerable program's memory. In order to remedy code reuse attacks, many defense approaches have been proposed, each using a different mechanism for detecting attacks and having its own merits and downsides. In this paper, we analyze and scrutinize one of the most influential Linux-based defense mechanisms called ROPecker. Our analysis shows that ROPecker has weaknesses that may allow an attacker to bypass detection. Then we propose ROPK++ which by adding additional integrity checks, fixes the weaknesses in ROPecker and offers a more effective defensive approach against code reuse attacks in Linux-based systems. We compare the proposed approach with ROPecker in terms of security features and performance overhead and show its superiority and advantages.