Learning Data Mining with Python

Chapter 9
Accessing the Enron dataset
The full set of Enron e-mails is available at https://www.cs.cmu.edu/~./enron/.
The full dataset is 423 MB in a compression format called gzip. If you
don't have a Linux-based machine to decompress (unzip) this file, get
an alternative program, such as 7-zip (http://www.7-zip.org/).
Download the full corpus and decompress it into your data folder. By default, this
will decompress into a folder called enron_mail_20110402.
As we are looking for authorship information, we only want the e-mails we can
attribute to a specific author. For that reason, we will look in each user's sent
folder—that is, e-mails they have sent.
In the Notebook, setup the data folder for the Enron dataset:
enron_data_folder = os.path.join(os.path.expanduser("~"), "Data",
"enron_mail_20110402", "maildir")
Creating a dataset loader
We can now create a function that will choose a couple of authors at random and
return each of the emails in their sent folder. Specifically, we are looking for the
payloads—that is, the content rather than the e-mails themselves. For that, we will
need an e-mail parser. The code is as follows:
from email.parser import Parser
p = Parser()
We will be using this later to extract the payloads from the e-mail files that are
in the data folder.
We will be choosing authors at random, so we will be using a random state that
allows us to replicate the results if we want:
from sklearn.utils import check_random_state
[ 201 ]