Learning Data Mining with Python

Classifying with function words
Next, we import our classes. The only new thing here is the support vector
machines, which we will cover in the next section (for now, just consider it
a standard classification algorithm). We import the SVC class, an SVM for
classification, as well as the other standard workflow tools we have seen before:
from sklearn.svm import SVC
from sklearn.cross_validation import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn import grid_search
Chapter 9
Support vector machines take a number of parameters. As I said, we will use one
blindly here, before going into detail in the next section. We then use a dictionary
to set which parameters we are going to search. For the kernel parameter, we will
try linear and rbf. For C, we will try values of 1 and 10 (descriptions of these
parameters are covered in the next section). We then create a grid search to search
these parameters for the best choices:
parameters = {'kernel':('linear', 'rbf'), 'C':[1, 10]}
svr = SVC()
grid = grid_search.GridSearchCV(svr, parameters)
Gaussian kernels (such as rbf) only work for reasonably sized datasets,
such as when the number of features is fewer than about 10,000.
Next, we set up a pipeline that takes the feature extraction step using the
CountVectorizer (only using function words), along with our grid search using
SVM. The code is as follows:
pipeline1 = Pipeline([('feature_extraction', extractor),
('clf', grid)
])
Next, we apply cross_val_score to get our cross validated score for this pipeline.
The result is 0.811, which means we approximately get 80 percent of the predictions
correct. For 7 authors, this is a good result!
[ 195 ]