Hadoop Development - CSC

Real world example - a Cyber problem (cont’d)
• As always, there is more than one way to approach this, but one simple way would
seem to be to have a map-reduce job that takes as input:
– The auditd logs, and
– A list of parent process ids that we want to trace back
• and which outputs a file that contains:
– A list of parent process-ids as the key to a list of auditd events that all have this process-id as a ppid entry.
– A list of parent process-ids found that also have "terminal" entries (i.e. for which we have now found the user).
• We can then repeatedly run this map-reduce job using the output of one run as input,
with the auditd logs, to the next, until there is no difference in the output file between
two consecutive runs, or until the output file is empty (whichever occurs first).
(Note that since we will be dealing with audit logs from many different servers, we will need to use the IP
address of the server with the process-id to form a unique key)
TBSC 2009
11/10/2011 12:53 PM 0725-23_TBSC 2009 26