6th European Conference - Academic Conferences

Jaime Acosta
The research presented here uses a dataset that consists of sandbox event traces of 3131 malware
instances. Manual observation of the dataset revealed many behavior patterns that were shared
across many instances such as file replacements (which involve a series of system calls), that at first
glance seem complex and overwhelming, but were made simple by replacing these common
behaviors with short annotations. This paper is a step in automating this process.
The following are the contributions resulting from the work described in this paper.
This research provides a methodology shows how the longest common substring algorithm can
be modified to conduct similarity analysis on malware using dynamic event traces. This similarity
may be due to code reuse, which arises from legitimate third-party libraries and also by reusing
infected or malicious code.
Use of this algorithm shows that in this dataset of malware, even though the instances are of
different types (assigned by anti-virus programs), there are a large number of common behaviors.
This means that it is the case that malware authors reuse code, and that an analyst could use this
to eliminate duplicate processing.
This research shows that the common behaviors identified are not limited to short trivial event
sequences; there are many large sequences. This indicates that it may be possible to replace
semantically rich events with natural language annotations to facilitate analysis.
2. Related work
Because of the large growth of malware instances being introduced each year, there has been a large
amount of work to aid in each stage of the malware analysis workflow.
The first step in analysis is data collection. Tools that aid in this collection include Nepenthes
(Baecher et al., 2006), Amun (Göbel, 2009), and HoneyPots (Provos, 2004). After collection, the
malware instances are analyzed using static (source code) or dynamic (event traces) techniques. In
the past decade there have been a wide variety of techniques used for static and dynamic analysis of
legitimate source code, with the goal of exploiting program semantics in an efficient way (Cornelissen,
2009). Related to malware, there have been many techniques that exploit characteristics unique to
malware, including malicious behavior, small program size, and code reuse among instances.
In both static and dynamic analysis techniques, one method that has had recent attention is using
machine learning to cluster similar malware instances. Clustering methods are useful because they
generalize large sets of malware into categories with limited need for manual human intervention.
Jang and Brumley (2009) perform static analysis by identifying areas of code reuse by clustering
malware binaries. His clustering method uses bloom filters, which identify similarity of malware
instances by applying hashing techniques to fixed size chunks of the malware executable code.
On the other hand, Bayer et al. (2009) use machine learning algorithms to identify similarities in
malware instances by comparing their dynamic event traces, which include system calls, their
dependencies, and network behavior. Next, the malware instances are clustered based on their
dynamic behavior. A limitation of this approach is that the algorithm is trained with a fixed set of
malware. It does not allow retraining with additional malware samples during the clustering phase.
Rieck extends this with his Malheur (Rieck et al., 2010) system by establishing an iterative
mechanism that consists of clustering and then classifying new instances into existing clusters. In his
work, similarity is determined by the presence of shared fixed-length instruction sequences. In
addition, Rieck also uses a dynamic trace representation format called MIST (Trinius et al., 2010) that
allows prioritization of event parameters (e.g., an openfile system call may have the file name, file
type, and the file path as parameters). This is meant to allow more efficient processing for machine
learning algorithms by reducing the input file size by leaving out less-critical parameters. MIST also
provides a common file format to which many of the available sandbox output can be converted.
After the instances are clustered, an analyst may have to conduct deeper investigation, such as exact
differences and similarities in the binaries. It may be the case that malware in different clusters share
common behaviors. This results in redundant analysis by a human analyst. Another issue is that
instances in a cluster are not exactly the same. There may be malicious behavior that is unique to one
instance within a cluster. One way to alleviate these issues is to, instead of determining similarity by
using fixed size sequences as in previous work, develop techniques that are not tied to sequence
length and automatically detect varied sized semantically-representative sequences.
2