6th European Conference - Academic Conferences

Manoj Cherukuri and Srinivas Mukkamala
Figure 5: Flowchart for the process of malicious websites collection.
All the malicious websites listed in these sources were collected and stored in the database. Crawling
was performed on the collected malicious websites up to the third hop as described in the section 4.2
to build the link structure for all the malicious websites.
Finally, the domains obtained during the first two processes of the data collection were translated to
their geographical location using the process described in the section 4.3. To perform a comparative
analysis of the malicious websites against the legitimate websites, the top 1500 webistes were
downloaded from Alexa (Top Sites, 2010), a source for top websites and were crawled upto the
second hop. This domain is reffered as a set of legitimate or non-malicious websites in the remaining
part of this paper.
Around 350,000 distinct malicious websites were collected from the previously mentioned sources.
Since these domains were detected and flagged as malicious, major portion of them were down at the
time of the analysis. Only about 20,000 distinct URLs were alive at the time of our analysis. About
only 5.7% of the malicious websites collected were alive for our analysis. Link analysis was performed
on about 19,000 malicious websites of the 20,000 live malicious websites. Remaining websites did
not have any text to perform link analysis as they were pointing out to files like executables, jars,
binaries etc.
Around 600,000 Uniform Resource Locators (URLs) were crawled during the collection of our dataset.
The URLs were crawled at the rate 50 URLs per minute. Of the live malicious websites, 14,970
domains were hosted in United States. The top five countries contributed 83% of the total malicious
websites of our dataset.
56