Antivirus hackers handbook

134 Part II ■ Antivirus Software Evasion
engines based on static data (data extracted from the portable executable, or PE,
header). Another tip is that, in general, if a file format cannot be correctly parsed
by the scanner or engine responsible for handling a specific file format (such
as a “malformed” PE file), it will be discarded from any and all PE routines,
but cyclic redundancy check (CRC) signatures may still be applied to the file
(for example, CRCs at some specific offset). Later in this chapter, you will see
examples with various file formats.
Another trick is that instead of trying to make it difficult for the antivirus
engine to parse the file format, you can try to fool one or more of the core’s support
functionalities or libraries. The typical core support functionality resides
in the emulator and the disassembler. As far as I know, every antivirus engine,
except ClamAV, contains an emulator for at least Intel 8086 and a disassembler
for Intel x86. Can you attack the disassembler or the emulator to affect or evade
the scanner? Many analysis routines rely on the emulation and disassembling
functionality to gather evidence and behavioral data from malware. If you can
somehow manage to execute invalid instructions in the emulator or if you can
craft valid but unimplemented or incorrectly implemented instructions in the
disassembly engine, you get the same behavior in most AV scanners: no analysis
routine is able to navigate through the disassembly of your file because the core
kernel support functionality is flawed.
The following sections discuss more tricks that you can use to evade scanners.
Fingerprinting Emulators
Fingerprinting emulators is one of the most commonly used evasion techniques.
Malware samples usually become a more likely candidate for emulation when
they contain polymorphic or metamorphic code. Using a static analysis engine
is not enough because writing a complex and foolproof static analysis engine is
too expensive. To identify an emulator in an AV kernel, you can rely on the fact
that the emulator may correctly or fully emulate not a whole operating system
but only the most commonly executed functions. In many cases, you can give
the illusion that all the operating system functions are implemented by creating
stubs for those functions that, very often, return hard-coded values. The following
example uses the Comodo antivirus emulator for Linux. If you open the
library libMACH32.so (which is full of symbols, something that is very helpful)
in IDA, you will discover functions like the following one:
.text:000000000018B93A ; PRUint32 __cdecl Emu_OpenMutexW
(void *pVMClass)
.text:000000000018B93A
public _Z14Emu_OpenMutexWPv
.text:000000000018B93A _Z14Emu_OpenMutexWPv proc near
; DATA XREF: .data:kernel32ApiInf
.text:000000000018B93A pVMClass = rdi
; void *
.text:000000000018B93A mov eax, 0BBBBh