Learning Data Mining with Python

Social Media Insight Using Naive Bayes
As a last bit of JavaScript for this chapter (I promise), we call the load_next_
tweet() function. This will set the first tweet to be labeled and then close off the
JavaScript. The code is as follows:
load_next_tweet();
After you run this cell, you will get an HTML textbox, alongside the first tweet's text.
Click in the textbox and enter 1 if it is relevant to our goal (in this case, it means is the
tweet related to the programming language Python) and a 0 if it is not. After you do this,
the next tweet will load. Enter the label and the next one will load. This continues
until the tweets run out.
When you finish all of this, simply save the labels to the output filename we defined
earlier for the class values:
with open(labels_filename, 'w') as outf:
json.dump(labels, outf)
You can call the preceding code even if you haven't finished. Any labeling you have
done to that point will be saved. Running this Notebook again will pick up where
you left off and you can keep labeling your tweets.
This might take a while to do this! If you have a lot of tweets in your dataset, you'll
need to classify all of them. If you are pushed for time, you can download the same
dataset I used, which contains classifications.
Creating a replicable dataset from Twitter
In data mining, there are lots of variables. These aren't just in the data mining
algorithms—they also appear in the data collection, environment, and many other
factors. Being able to replicate your results is important as it enables you to verify
or improve upon your results.
Getting 80 percent accuracy on one dataset with algorithm X, and
90 percent accuracy on another dataset with algorithm Y doesn't
mean that Y is better. We need to be able to test on the same
dataset in the same conditions to be able to properly compare.
[ 114 ]