Undecidable problems in malware analysis

Abstract

Malware analysis is a challenging task in the theory as well as the practice of computer science. Many important problems in malware analysis have been shown to be undecidable. These problems include virus detection, detecting unpacking execution, matching malware samples against a set of given templates, and detecting trigger-based behavior. In this paper, we will give a review of the undecidability results in malware analysis and discuss what can be done in practice.