Information theoretic method for classification of packed and encoded files

Abstract

Malware authors make use of some anti-reverse engineering and obfuscation techniques like packing and encoding in-order to conceal their malicious payload. These techniques succeeded in evading the traditional signature based AV scanners. Packed or encoded malware samples are difficult to be analysed directly by the AV scanners. So, such samples must be initially unpacked or decoded for efficient analysis of the malicious code. This paper illustrates a static information theoretic method for the classification of packed and encoded files. The proposed method extracts fragments of fixed size from the files and calculates the entropy scores of the fragments. These entropy scores are then used for computing the Similarity Distance Matrix for fragments in a file-pair. The proposed system classifies all the encoded and packed samples properly, thereby obtaining improved detection. The proposed system is also capable of differentiating the type of packers used for the packing or encoding process.