Building Machine Learning Systems with Python - Richert, Coelho

Classification II – Sentiment Analysis
With our first try of using Naive Bayes on vectorized TF-IDF trigram features, we get
an accuracy of 80.5 percent and a P/R AUC of 87.8 percent. Looking at the P/R chart
shown in the following screenshot, it shows a much more encouraging behavior than
the plots we saw in the previous chapter:
For the first time, the results are quite encouraging. They get even more impressive
when we realize that 100 percent accuracy is probably never achievable in a
sentiment classification task. For some tweets, even humans often do not really
agree on the same classification label.
Using all the classes
But again, we simplified our task a bit, since we used only positive or negative
tweets. That means we assumed a perfect classifier that classified upfront whether
the tweet contains a sentiment and forwarded that to our Naive Bayes classifier.
So, how well do we perform if we also classify whether a tweet contains any
sentiment at all? To find that out, let us first write a convenience function that
returns a modified class array that provides a list of sentiments that we would like
to interpret as positive
def tweak_labels(Y, pos_sent_list):
pos = Y==pos_sent_list[0]
for sent_label in pos_sent_list[1:]:
pos |= Y==sent_label
[ 130 ]