Easy to Fool? Testing the Anti-evasion Capabilities of PDF Malware Scanners

Malware scanners try to protect users from opening malicious documents by statically or dynamically analyzing documents. However, malware developers may apply evasions that conceal the maliciousness of a document. Given the variety of existing evasions, systematically assessing the impact of evasions on malware scanners remains an open challenge. This paper presents a novel methodology for testing the capability of malware scanners to cope with evasions. We apply the methodology to malicious Portable Document Format (PDF) documents and present an in-depth study of how current PDF evasions affect 41 state-of-the-art malware scanners. The study is based on a framework for creating malicious PDF documents that use one or more evasions. Based on such documents, we measure how effective different evasions are at concealing the maliciousness of a document. We find that many static and dynamic scanners can be easily fooled by relatively simple evasions and that the effectiveness of different evasions varies drastically. Our work not only is a call to arms for improving current malware scanners, but by providing a large-scale corpus of malicious PDF documents with evasions, we directly support the development of improved tools to detect document-based malware. Moreover, our methodology paves the way for a quantitative evaluation of evasions in other kinds of malware.