Building Machine Learning Systems with Python - Richert, Coelho

Classification – Detecting Poor Answers
>>> clf.fit(X, y)
>>> print(np.exp(clf.intercept_), np.exp(clf.coef_.ravel()))
[ 0.09437188] [ 1.80094112]
>>> def lr_model(clf, X):
return 1 / (1 + np.exp(-(clf.intercept_ + clf.coef_*X)))
>>> print("P(x=-1)=%.2f\tP(x=7)=%.2f"%(lr_model(clf, -1), lr_
model(clf, 7)))
P(x=-1)=0.05 P(x=7)=0.85
You might have noticed that Scikit-learn exposes the first coefficient through the
special field intercept_.
If we plot the fitted model, we see that it makes perfect sense given the data:
Applying logistic regression to our
postclassification problem
Admittedly, the example in the previous section was created to show the beauty of
logistic regression. How does it perform on the extremely noisy data?
Comparing it to the best nearest neighbour classifier (k = 90) as a baseline, we see
that it performs a bit better, but also won't change the situation a whole lot:
Method mean(scores) stddev(scores)
LogReg C=0.1 0.6310 0.02791
LogReg C=100.00 0.6300 0.03170
LogReg C=10.00 0.6300 0.03170
LogReg C=0.01 0.6295 0.02752
[ 108 ]