Views
5 years ago

April 10, 2011 Salzburg, Austria - WOMBAT project

April 10, 2011 Salzburg, Austria - WOMBAT project

Figure 2 presents the

Figure 2 presents the AES experimentation process. The coordinators (part of the AES Farm) are computer systems running the AES Engine code to control the virtual machines of the VLab. This process is repeated for each experiment. Each coordinator of the AES Farm is able to execute one experiment at a time. However, many coordinators can be used in parallel on a physical machine (Section 3.3). Note that the network topology in Figure 2 is similar to the one of the experiment presented in Figure 1, where the experiment contains an attacker (Attacker), a target system (Target) and network infrastructure services (Network Services). Here are the phases of the AES experimentation process (the phase numbers refer to the numbers in Figure 2): 1. Experiment Negotiation. Each coordinator negotiates the experiment to conduct with the other coordinators to ensure that it does not conduct an experiment that was already conducted or that is currently conducted by another coordinator. When the coordinator has identified an experiment that is available, it locks the experiment so no other coordinator can conduct it. 2. VLab Setup. Based on the network topology of the experiment, the coordinator builds the virtual network where the experiment is to be conducted. 3. Opening Communication Channel. Once the virtual network is ready, the coordinator opens the communication channels with the actors of the experiments. These communication channels (e.g. a hard drive shared via VMware, a virtual parallel port, a virtual serial port, ssh, etc.) are the only way the coordinator can communicate the experiment steps to be executed to the actors. This enables the isolation of the virtual network (from a networking point of view) while keeping some communication capabilities. 4. Experiment Execution. The coordinator executes the EEG by providing commands to the virtual machine in the VLab. 5. Tear Down. The coordinator gathers the experiment results (e.g., the traffic traces) using the communication channel. Then, the coordinator documents and stores the experiment results and restores the virtual machines to their initial state. 3.1.3 Documentation Process After an experiment is completed by the AES Engine, its results are documented. In the example of Figure 1, the AES Engine gathers the results of the execution of jill and the traffic trace that contains the execution of this attack. Thus, the experiment file (jill.exp.xml) is stored with the results (jill.res.txt and jill.res.pcap) and a report describing the experiment execution (jill.aes.xml). This report file (Appendix 3.4.2) keeps among other things the success or not of the experiment, the time required to execute each step of the experiment, which coordinator conducted the experiment and which non-mandatory step were not successfully executed. This documentation process, in combination with the experimentation specification, are essential for another computer 5 21 system (e.g., BEAVER) to automatically use the experiment results as well as to make this experiment repeatable (the actors are also required), so the results can be analyzed independently to confirm or refute the claims derived from these results. It is also important to document the experiments to share results with other researchers (e.g., to make the traffic traces publicly available), so they can understand and reproduce the experiments. 17 3.2 Virtual Laboratory The VLab is the environment where the experiments are conducted. It is constructed from a collection of virtual machine images (VLab database). The AES Engine is the system that automatically conducts the experiments within the VLab environment. In its current implementation, the AES does not support on-the-fly configuration of the actors. The virtual machines have to be created and configured (e.g., the IP addresses have to be set) prior to conducting the experiments. Our VLab database is a collection of more than 200 WMware Workstation and ESX virtual machine images. These virtual machines can be grouped into four categories. The Attacker group is composed of virtual machines loaded with vulnerability exploitation programs and frameworks (e.g., MetaSploit 18 ) and vulnerability scanners (e.g., Nessus 19 ). The Network Infrastructure group is composed of various networking components (e.g., DNS, router) to facilitate the construction of the network topology required for an experiment. The Security Tools group is composed of virtual machines loaded with security protection software such as IDSs and anti-virus. The Target group is composed of the installation of nearly all Windows, RedHat Linux, SUSE Linux, FreeBSD, NetBSD and OpenBSD operating systems available. Every operating system and network service is installed using its default settingd. This consists of more than 150 different operating system versions. These are used in our experiments as the cyber attack victims. 3.3 AES Farm Our first AES Farm, that was used to generate our original operating system fingerprinting [6] and IDS evaluation [14] data sets, was composed of five Pentium 4 computers with 1 GB of RAM each, running Windows 2000. Over time, we have performed several upgrades to our farm, which is now composed of about 50 desktop computers with i7 processors and 12 GB of RAM. Each of these machines has five hard drives: one for the operating system, four for the virtual machine images. Since we discovered that hard drives are a bottleneck, we assigned one hard drive per experiment coordinator (each of these desktop computers runs four experiments simultaneously). Of these 50 machines, around 20 are used with VMware Workstation running on Windows XP, 7 or 2008 and 30 are used with VMware ESX. 17 Note that most of the experiment outputs presented in Section 4 are available upon request by sending an email to networksystem-security@crc.gc.ca with the subject Experimentation Data Request, your affiliation, a brief description of your research project and an explanation of why this data could be useful to your research. 18 www.metasploit.com 19 www.nessus.org

Figure 3: AES Monitor We also have five server-class computers running VMware ESX, two with 8 Intel Xeon dual core processors and 64 GB of RAM, three with 2 Intel Xeon quad core processors and 32 GB of RAM. Since the AES is designed to conduct large numbers of lightweight (small) experiments (i.e., using small networks) as quickly as possible, pulling the virtual machine images from the hard drive (at the beginning of every experiment) is a time-consuming process. Consequently, the desktop computers are more cost-effective for us in terms of experiment per minute per dollar. 3.4 AES Dashboard Simultaneously conducting dozens of experiments can easily become overwhelming in terms of monitoring the progression of the whole experiment set. A given experiment may stall, one of the virtual machines may freeze, steps may require more time than expected, etc. For this reason, we developed tools to monitor the system. We present them in this section, as well as some other existing tools that we found useful. 3.4.1 Progression We developed a web-based application allowing to visualize the overall progress and speed of the AES. Figure 3 shows the main view of this application. On top, a graphical view of the respective speed (in terms of experiments per minute) of each experiment coordinator is provided. This allows us to compare the performance of the different software and hardware configurations of the various servers within our AES Farm. Detailed information in terms of the number of experiments performed, with a distinction between those that failed and those that succeeded, is also provided for each experiment coordinator. Finally, global performance of the AES is also provided. 6 22 Figure 4: AES Profiler 3.4.2 Profiling When designing an experiment, it is not always clear which steps will be time consuming, and optimizing an experiment is not a trivial process. To facilitate this process, the AES produces a report which is stored with each experiment results. This report contains the time required for each step. It also contains the time it is using for its hidden steps, such as powering on the virtual machines, setting up the topology, etc. Figure 4 shows a screenshot of a web-based application that we developed to visualize the average time of each step of an experiment set. In this case, each experiment is made of eight steps, and there are six hidden steps (four to prepare the experiment, and two to shut it down). The most timeconsuming step, in this case, is the third hidden step of the experiment setup, which consists in powering on the virtual machines. This is due to the significant amount of time that is required to pull the virtual machine images from the hard drive. Once we know that, we know that the hard drive is a bottleneck in our system. Incidentally, we are currently investigating the possibility of using solid state hard drives to improve the overall performance of the AES. 3.4.3 Error Reporting The code of the AES has its own error handling mechanism. When an error occurs, causing the experiment to fail, it is logged and the experiment remains in the queue of experiments to be performed. Sometimes, the error is only incidental and the experiment completes successfully on the next attempt. However, it could be due to a design

D06 (D3.1) Infrastructure Design - WOMBAT project
6-9 December 2012, Salzburg, Austria Social Programme
D I P L O M A R B E I T - Salzburg Research
D I P L O M A R B E I T - Salzburg Research
D I P L O M A R B E I T - Salzburg Research
ECCMID meeting Vienna, Austria 10-13 April 2010 - European ...
Communication Plan for EGU 2011 April 3-8, 2011, Vienna, Austria
8th Liquid Matter Conference September 6-10, 2011 Wien, Austria ...
8th Liquid Matter Conference September 6-10, 2011 Wien, Austria ...
April 10, 2011 - University of Cambridge
Top 10 Project Management Trends for 2011 from ESI International