Learning Data Mining with Python

Authorship Attribution
We then record the class number we used for this author and then increment it:
authors[user] = author_num
author_num += 1
We then check if we have enough authors and, if so, we break out of the loop to
return the dataset. The code is as follows:
if author_num >= num_authors or author_num >=
len(email_addresses):
break
We then return the datatset's documents and classes, along with our author
mapping. The code is as follows:
return documents, np.array(classes), authors
Outside this function, we can now get a dataset by making the following function
call. We are going to use a random state of 14 here (as always in this book), but
you can try other values or set it to none to get a random set each time the function
is called:
documents, classes, authors = get_enron_corpus(data_folder=enron_data_
folder, random_state=14)
If you have a look at the dataset, there is still a further preprocessing set we need to
undertake. Our e-mails are quite messy, but one of the worst bits (from a data analysis
perspective) is that these e-mails contain writings from other authors, in the form of
attached replies. Take the following e-mail, which is documents[100], for instance:
I am disappointed on the timing but I understand. Thanks. Mark
-----Original Message-----
From: Greenberg, Mark
Sent: Friday, September 28, 2001 4:19 PM
To: Haedicke, Mark E.
Subject: Web Site
Mark -
FYI - I have attached below a screen shot of the proposed new look and feel for the
site. We have a couple of tweaks to make, but I believe this is a much cleaner look
than what we have now.
[ 204 ]