6th European Conference - Academic Conferences

Manoj Cherukuri and Srinivas Mukkamala
1
where ‘C’ represents the clustering coefficient of the graph, Ci represents the clustering coefficient of
the node ‘i’ and ‘n’ is the total number of nodes in the graph (Clustering Coefficient, 2010). The
clustering coefficient of the graph in the figure above is 0.466 (i.e.1/5(0.33+1+1+0+0).
3. Processes
Construction of our dataset is composed of three processes.
The first process deals with the collection of malicious websites from multiple sources
The second process deals with the construction of link structure for the malicious domains
obtained in the previous process
The third process deals with computing the geographical location of the websites based on the IP
address to geographical location mapping
3.1 Collection of malicious websites
The process of collecting the malicious websites is initialized by identifying the potential sources using
the meta-search engines. These sources publish the websites with different sets of associated
attributes. Some of the attributes associated with these domains are date, type of attack, executable
name and IP address. A custom parser was used for each website source to retrieve the domain
name and the IP address if available. All the malicious websites collected from these sources are
stored in the database.
3.2 Construction of link structure for malicious websites
Crawling is performed on the malicious websites obtained from the previous process using a custom
program. The malicious websites are crawled till the second hop. The flowchart for the process of
building the link structure is shown in the figure 6. All the malicious domains retrieved from various
sources are loaded which act as the seeds for the crawling process and ‘n’ is the total number of
malicious websites. A custom crawler was built to retrieve the content of the malicious websites and
parse all the anchor tags from the content. The parsing of the anchor tags from the website content is
done using the BeautifulSoup (Beautiful Soup, 2010), widely used html parser. All the links originating
from respective malicious websites are stored in the database. The domain is parsed from each such
link and is stored in the tuple corresponding to that link. If the link URL is not listed in the malicious
websites then it is added to a list called as LinkWebsites to avoid duplicate URLs. Once all the
malicious domains are crawled and their corresponding links are stored, the LinkWebsites list is
loaded into MaliciousWebsites list to crawl all the new websites obtained in the first hop from the
malicious websites. Again the process of crawling, parsing anchor tags and storing the links is
performed. Thus, all the links within the first two hops emerging from the malicious websites are
obtained.
3.3 Computing geographical location of websites
The location of the server on which the website is hosted was determined using the IP address to
geographic location mapping. All the distinct domains that are stored during the link analysis phase
are translated to their corresponding IP addresses using the Domain Name Servers (DNS). A custom
script is used to perform the domain name to IP address translation using the ‘nslookup’ command.
The IP address to location mapping is done using the open source database. All the IP addresses for
which the location is not identified are mapped to latitude and longitude values 0 and 0 respectively.
4. Dataset
For our experiments, all the potential sources for the malicious websites were identified using the
meta-search engines. Such identified sources used for the study were PhishTank, Malware Domain
Blocklist, abuse.ch, MalwarePatrol, joewein.de LLC. and Malware Domain List.
55