Building Machine Learning Systems with Python - Richert, Coelho

Chapter 6
Our first estimator
Now we have everything in place to create our first vectorizer. The most convenient
way to do it is to inherit it from BaseEstimator. It requires us to implement the
following three methods:
• get_feature_names(): This returns a list of strings of the features that we
will return in transform().
• fit(document, y=None): As we are not implementing a classifier, we can
ignore this one and simply return self.
• transform(documents): This returns numpy.array(), containing an array of
shape (len(documents), len(get_feature_names)). This means that for
every document in documents, it has to return a value for every feature name
in get_feature_names().
Let us now implement these methods:
sent_word_net = load_sent_word_net()
class LinguisticVectorizer(BaseEstimator):
def get_feature_names(self):
return np.array(['sent_neut', 'sent_pos', 'sent_neg',
'nouns', 'adjectives', 'verbs', 'adverbs',
'allcaps', 'exclamation', 'question', 'hashtag',
'mentioning'])
# we don't fit here but need to return the reference
# so that it can be used like fit(d).transform(d)
def fit(self, documents, y=None):
return self
def _get_sentiments(self, d):
sent = tuple(d.split())
tagged = nltk.pos_tag(sent)
pos_vals = []
neg_vals = []
nouns = 0.
adjectives = 0.
verbs = 0.
adverbs = 0.
[ 143 ]