Building Machine Learning Systems with Python - Richert, Coelho

Classification II – Sentiment Analysis
The devastating results for positive tweets against the rest and negative tweets
against the rest will improve if we configure the vectorizer and classifier with those
parameters that we have just found out:
== Pos vs. rest ==
0.883 0.005 0.520 0.028
== Neg vs. rest ==
0.888 0.009 0.631 0.031
Indeed, the P/R curves look much better (note that the graphs are from the medium
of the fold classifiers, thus have slightly diverging AUC values):
Nevertheless, we probably still wouldn't use those classifiers. Time for something
completely different!
Cleaning tweets
New constraints lead to new forms. Twitter is no exception in this regard. Because
text has to fit into 140 characters, people naturally develop new language shortcuts
to say the same in less characters. So far, we have ignored all the diverse emoticons
and abbreviations. Let's see how much we can improve by taking that into
account. For this endeavor, we will have to provide our own preprocessor() to
TfidfVectorizer.
[ 136 ]