On the Detection of Kernel-Level Rootkits Using Hardware Performance Counters

Abstract

Recent work has investigated the use of hardware performance counters (HPCs) for the detection of malware running on a system. These works gather traces of HPCs for a variety of applications (both malicious and non-malicious) and then apply machine learning to train a detector to distinguish between benign applications and malware. In this work, we provide a more comprehensive analysis of the applicability of using machine learning and HPCs for a specific subset of malware: kernel rootkits. We design five synthetic rootkits, each providing a single piece of rootkit functionality, and execute each while collecting HPC traces of its impact on a specific benchmark application. We then apply machine learning feature selection techniques in order to determine the most relevant HPCs for the detection of these rootkits. We identify 16 HPCs that are useful for the detection of hooking based roots, and also find that rootkits employing direct kernel object manipulation (DKOM) do not significantly impact HPCs. We then use these synthetic rootkit traces to train a detection system capable of detecting new rootkits it has not seen previously with an accuracy of over 99%. Our results indicate that HPCs have the potential to be an effective tool for rootkit detection, even against new rootkits not previously seen by the detector.