Machine Learning in Python Essential Techniques for Predictive Analysis by Michael Bowles (z-lib.org).epub

Recommendations

Info

How This Book Is StructuredThis book follows the basic order in which you would approach anew prediction problem. The beginning involves developing anunderstanding of the data and determining how to formulate theproblem, and then proceeds to try an algorithm and measure theperformance. In the midst of this sequence, the book outlines themethods and reasons for the steps as they come up. Chapter 1 gives amore thorough description of the types of problems that this bookcovers and the methods that are used. The book uses several data setsfrom the UC Irvine data repository as examples, and Chapter 2exhibits some of the methods and tools that you can use fordeveloping insight into a new data set. Chapter 3, “Predictive ModelBuilding: Balancing Performance, Complexity, and Big Data,” talksabout the difficulties of predictive analytics and techniques foraddressing them. It outlines the relationships between problemcomplexity, model complexity, data set size, and predictiveperformance. It discusses overfitting and how to reliably senseoverfitting. It talks about performance metrics for different types ofproblems. Chapters 4 and 5, respectively, cover the background onpenalized linear regression and its application to problems exploredin Chapter 2. Chapters 6 and 7 cover background and application forensemble methods.What You Need to Use This BookTo run the code examples in the book, you need to have Python 2.x,SciPy, NumPy, Pandas, and scikit-learn. These can be difficult toinstall due to cross-dependencies and version issues. To make theinstallation easy, I’ve used a free distribution of these packages that’savailable from Continuum Analytics (http://continuum.io/). TheirAnaconda product is a free download and includes Python 2.x and allthe packages you need to run the code in this book (and more). I’verun the examples on Ubuntu 14.04 Linux but haven’t tried them onother operating systems.
ConventionsTo help you get the most from the text and keep track of what’shappening, we’ve used a number of conventions throughout the book.WARNING Boxes like this one hold important, not-to-beforgotten information that is directly relevant to the surroundingtext.NOTE Notes, tips, hints, tricks, and asides to the currentdiscussion are offset and appear like this.As for styles in the text:We highlight new terms and important words when we introducethem.We show keyboard strokes like this: Ctrl+A.We show filenames, URLs, and code within the text like so:persistence.properties.We present code in two different ways:We use a monofont type with no highlighting for most codeexamples.We use bold to emphasize code that's particularlyimportant in the present context.Source CodeAs you work through the examples in this book, you may chooseeither to type in all the code manually or to use the source code filesthat accompany the book. All the source code used in this book isavailable for download fromhttp://www.wiley.com/go/pythonmachinelearning. You will find thecode snippets from the source code are accompanied by a downloadicon and note indicating the name of the program so that you know
Page 3 and 4: IntroductionExtracting actionable i
Page 5 and 6: Who This Book Is ForThis book is in
Page 7 and 8: save Chapter 2 until they start loo
Page 9: evolved from overcoming the limitat
Page 13 and 14: CHAPTER 1The Two Essential Algorith
Page 15 and 16: the flight path, and so on. Having
Page 17 and 18: become an issue. In some problems,
Page 19 and 20: Figure 1.2 Fitting lines with only
Page 21 and 22: Figure 1.3 Binary decision tree exa
Page 23 and 24: For example, if the training set is
Page 25 and 26: NOTE The first pass can usually be
Page 27 and 28: convergence divergence), and RSI (r
Page 29 and 30: runs through code for the core algo
Page 31 and 32: CHAPTER 2Understand the Problem byU
Page 33 and 34: A unique identifier may or may not
Page 35 and 36: The attributes shown in Table 2.1 c
Page 37 and 38: THINGS TO NOTICE ABOUT YOUR NEW DAT
Page 39 and 40: PHYSICAL CHARACTERISTICS OF THE ROC
Page 41 and 42: facilitate iterating through the pr
Page 43 and 44: colCounts.append(type)type = [0]*3s
Page 45 and 46: LISTING 2-3: SUMMARY STATISTICS FOR
Page 47 and 48: catCount = [0]*2for elt in colData:
Page 49 and 50: from the line. That means that the
Page 51 and 52: Figure 2.1 Quantile-quantile plot o
Page 53 and 54: makes it possible to read data into
Page 55 and 56: LISTING 2-5: USING PYTHON PANDAS TO
Page 57 and 58: Pandas makes it possible to automat
Page 59 and 60: Figure 2.3 Parallel coordinates gra
Page 61 and 62:
of representative pairs of attribut
Page 63 and 64:
Figure 2.5 Cross- plot of rocks ver
Page 65 and 66:
Figure 2.7 Target-attribute cross-p
Page 67 and 68:
# and add some ditherif rocksVMines
Page 69 and 70:
Equation 2-2: Average values of the
Page 71 and 72:
(sqrt(var2*var3) * numElt)corr221 +
Page 73 and 74:
Figure 2.8 Heat map showing attribu
Page 75 and 76:
that you saw in “Classification P
Page 77 and 78:
plot.ylabel(("Quartile Ranges"))sho
Page 79 and 80:
As an alternative to the listing of
Page 81 and 82:
Figure 2.10 Box plot of real-valued
Page 83 and 84:
work for the abalone problem. Rocks
Page 85 and 86:
for i in range(nrows):#plot rows of
Page 87 and 88:
Figure 2.13 Graph of the logit func
Page 89 and 90:
LISTING 2-12: CORRELATIONCALCULATIO
Page 91 and 92:
Figure 2.15 Correlation heat map fo
Page 93 and 94:
LISTING 2-13: WINE DATA SUMMARY—W
Page 95 and 96:
mean 0.087467 15.87492246.467792 0.
Page 97 and 98:
Figure 2.17 Parallel coordinate plo
Page 99 and 100:
#Try again with normalized valuesfo
Page 101 and 102:
Figure 2.19 Correlation heat map fo
Page 103 and 104:
LISTING 2-15: SUMMARY OF GLASS DATA
Page 105 and 106:
75% 0.610000 9.172500 0.000000 0.10
Page 107 and 108:
Figure 2.21 Parallel coordinate plo
Page 109 and 110:
plot.ylabel(("Attribute Values"))pl
Page 111 and 112:
building predictive models. The too
Page 113 and 114:
The variable that you are attemptin
Page 115 and 116:
Referring to the data set in Table
Page 117 and 118:
ASSESSING PERFORMANCE OF PREDICTIVE
Page 119 and 120:
training on the remaining data. Sta
Page 121 and 122:
Figure 3.2 A complicated classifica
Page 123 and 124:
CONTRAST BETWEEN A SIMPLE MODEL AND
Page 125 and 126:
Figure 3.5 Linear model fit to comp
Page 127 and 128:
Figure 3.7 Linear model fit to smal
Page 129 and 130:
used for predictive modeling. Addin
Page 131 and 132:
Model to Balance Problem Complexity
Page 133 and 134:
LISTING 3-1: COMPARISON OF MSE, MAE
Page 135 and 136:
from the mean) and the standard dev
Page 137 and 138:
Figure 3.9 Confusion matrix example
Page 139 and 140:
LISTING 3-2: MEASURING PERFORMANCEF
Page 141 and 142:
print("Shape of yTrain array", yTra
Page 143 and 144:
examples after it has been deployed
Page 145 and 146:
associated with its removal. If the
Page 147 and 148:
Figure 3.10 In-sample ROC for rocks
Page 149 and 150:
demonstration that performance esti
Page 151 and 152:
preserve the statistical peculiarit
Page 153 and 154:
vector Y contains the labels. And t
Page 155 and 156:
Initialize: Out_of_sample_error = N
Page 157 and 158:
indices = range(len(xList))xListTes
Page 159 and 160:
#scatter plot of actual versus pred
Page 161 and 162:
LISTING 3-4: FORWARD STEPWISEREGRES
Page 163 and 164:
Figure 3.15 Histogram of wine taste
Page 165 and 166:
regression throttle back ordinary r
Page 167 and 168:
xTrain = numpy.array(xListTrain); y
Page 169 and 170:
Figure 3.16 Wine quality prediction
Page 171 and 172:
Figure 3.18 Histogram of wine taste
Page 173 and 174:
LISTING 3-7: ROCKS VERSUS MINES USI
Page 175 and 176:
Listing 3-8 shows the AUC and assoc
Page 177 and 178:
Figure 3.19 AUC for the rocks-versu
Page 179 and 180:
different problem types (regression
Page 181 and 182:
penalty. This chapter explains how
Page 183 and 184:
model for evaluation speed. The num
Page 185 and 186:
In this table, the outcomes are rea
Page 187 and 188:
Equation 4-4: Linear relation betwe
Page 189 and 190:
real numbers—the ones included in
Page 191 and 192:
between zero and the vector space p
Page 193 and 194:
Figure 4.2 Optimum solutions with s
Page 195 and 196:
the sum of squares penalties (the c
Page 197 and 198:
refinement to the forward stepwise
Page 199 and 200:
LISTING 4-1: LARS ALGORITHM FORPRED
Page 201 and 202:
#calculate correlation between attr
Page 203 and 204:
numeric values that are fixed by th
Page 205 and 206:
Figure 4.3 Coefficient curves for L
Page 207 and 208:
LISTING 4-2: 10-FOLD CROSS-VALIDATI
Page 209 and 210:
#Define test and training index set
Page 211 and 212:
looping nxval times. In this case n
Page 213 and 214:
predictably on new data. The more c
Page 215 and 216:
ElasticNet problem given by Equatio
Page 217 and 218:
Initializing and Iterating the Glmn
Page 219 and 220:
#calculate means and variancesxMean
Page 221 and 222:
value of betalabelHat = sum([xNorma
Page 223 and 224:
Figure 4.7 Coefficient curves for r
Page 225 and 226:
This section has gone through two s
Page 227 and 228:
LISTING 4-4: CONVERTING ACLASSIFICA
Page 229 and 230:
#number of steps to takenSteps = 35
Page 231 and 232:
Some problems require deciding amon
Page 233 and 234:
LISTING 4-5: BASIS EXPANSION FOR WI
Page 235 and 236:
squared, logarithmic, and sinusoida
Page 237 and 238:
numRow = [float(row[i]) for i inran
Page 239 and 240:
print(nameList)for i in range(ncols
Page 241 and 242:
classification problem to an ordina
Page 243 and 244:
CHAPTER 5Building Predictive Models
Page 245 and 246:
to use the cross-validation version
Page 247 and 248:
Another way to think about how to p
Page 249 and 250:
LISTING 5-1: USING CROSS-VALIDATION
Page 251 and 252:
#Call LassoCV from sklearn.linear_m
Page 253 and 254:
Figure 5.2 Out-of-sample error with
Page 255 and 256:
make a material difference in the r
Page 257 and 258:
#calculate means and variancesxMean
Page 259 and 260:
#different orderingabsCoef = [abs(a
Page 261 and 262:
Changing Y to un-normalized changes
Page 263 and 264:
LISTING 5-3: USING OUT-OF-SAMPLEERR
Page 265 and 266:
#Convert list of list to np array f
Page 267 and 268:
Figure 5.6 Cross-validation error c
Page 269 and 270:
other. Listing 5-4 initializes an e
Page 271 and 272:
tendency to be overfit. It’s more
Page 273 and 274:
Figure 5.9 Receiver operating chara
Page 275 and 276:
You can accomplish this by replicat
Page 277 and 278:
attrRow = [float(elt) for elt in ro
Page 279 and 280:
elif (predList[irow] >= 0.0) and (y
Page 281 and 282:
TP = tpr[52] * P#FN = False negativ
Page 283 and 284:
LISTING 5-5: COEFFICIENT TRAJECTORI
Page 285 and 286:
alphas, coefs, _ = enet_path(X, Y,l
Page 287 and 288:
0.038531154796719078, 0.00355153481
Page 289 and 290:
Figure 5.10 plots the coefficient c
Page 291 and 292:
the coefficients problematic becaus
Page 293 and 294:
xList.append(row)#separate labels f
Page 295 and 296:
#begin iterationnSteps = 100lamMult
Page 297 and 298:
sumBeta = sum([abs(betaInner[n]) fo
Page 299 and 300:
class. Ones that are large negative
Page 301 and 302:
LISTING 5-7: MULTICLASSCLASSIFICATI
Page 303 and 304:
ySD.append(stdDev)yNormalized = []f
Page 305 and 306:
groups, one plane will do it. If yo
Page 307 and 308:
the prediction. Then that is compar
Page 309 and 310:
CHAPTER 6Ensemble MethodsEnsemble m
Page 311 and 312:
LISTING 6-1: BUILDING A DECISION TR
Page 313 and 314:
HOW A BINARY DECISION TREE GENERATE
Page 315 and 316:
LISTING 6-2: TRAINING A DECISION TR
Page 317 and 318:
lhSse = sum([(s - lhAvg) * (s - lhA
Page 319 and 320:
Figure 6.3 Block diagram of depth 1
Page 321 and 322:
Figure 6.5 shows how the sum square
Page 323 and 324:
Figure 6.6 Prediction using depth 2
Page 325 and 326:
Figure 6.8 Prediction using depth 6
Page 327 and 328:
LISTING 6-3: CROSS-VALIDATION AT AR
Page 329 and 330:
Figure 6.9 Out-of-sample error vers
Page 331 and 332:
different figures of merit than reg
Page 333 and 334:
plot is similar to the plot of the
Page 335 and 336:
#maximum number of models to genera
Page 337 and 338:
Figure 6.11 MSE versus number of tr
Page 339 and 340:
Figure 6.13 shows the curve of MSE
Page 341 and 342:
LISTING 6-5: PREDICTING WINE QUALIT
Page 343 and 344:
mse = []allPredictions = []for iMod
Page 345 and 346:
solitary attributes and therefore c
Page 347 and 348:
demonstrate its variance reduction
Page 349 and 350:
#maximum number of models to genera
Page 351 and 352:
With gradient boosting, tree depth
Page 353 and 354:
Figure 6.19 Gradient boosting predi
Page 355 and 356:
Page 357 and 358:
Page 359 and 360:
LISTING 6-7: GRADIENT BOOSTING FORP
Page 361 and 362:
#build cumulative prediction from f
Page 363 and 364:
only needing tree depth when there
Page 365 and 366:
idxTest = random.sample(range(nrows
Page 367 and 368:
plot.plot(nModels,mse)plot.axis('ti
Page 369 and 370:
Page 371 and 372:
Page 373 and 374:
1. 1. Panda Biswanath, Joshua S. He
Page 375 and 376:
given in Chapter 6 helps you unders
Page 377 and 378:
the parameter being ignored, which
Page 379 and 380:
makes it possible to assign differe
Page 381 and 382:
Figure 7.1 Wine taste prediction pe
Page 383 and 384:
xTrain, xTest, yTrain, yTest = trai
Page 385 and 386:
Figure 7.2 Relative importance of v
Page 387 and 388:
ls Least mean squared error.lad Lea
Page 389 and 390:
If the type of max_features is floa
Page 391 and 392:
4. Once the oos performance curve i
Page 393 and 394:
test_size=0.30,random_state=531)# T
Page 395 and 396:
might want to try them both to make
Page 397 and 398:
LISTING 7-3: BUILDING A REGRESSIONM
Page 399 and 400:
latestPrediction = modelList[-1].pr
Page 401 and 402:
Figure 7.5 Wine taste error for Bag
Page 403 and 404:
Predictive Models Using Penalized L
Page 405 and 406:
#list of names forabaloneNames = nu
Page 407 and 408:
ASSESSING PERFORMANCE AND THEIMPORT
Page 409 and 410:
LISTING 7-5: PREDICTING ABALONE AGE
Page 411 and 412:
#plot training and test errors vs n
Page 413 and 414:
Figure 7.9 Abalone age prediction e
Page 415 and 416:
Figure 7.11 Abalone age prediction
Page 417 and 418:
outcomes might be “clicked on the
Page 419 and 420:
classification the labels are 0 or
Page 421 and 422:
LISTING 7-7: CLASSIFYING SONARRETUR
Page 423 and 424:
# Plot feature importancefeatureImp
Page 425 and 426:
# ('Threshold Value = ', 0.46564102
Page 427 and 428:
Figure 7.14 Variable importance for
Page 429 and 430:
This function predicts class probab
Page 431 and 432:
else:labels.append(0)attrRow = [flo
Page 433 and 434:
ctClass = [i*0.01 for i in range(10
Page 435 and 436:
# ('Threshold Value = ', 2.02817801
Page 437 and 438:
Figure 7.16 AUC versus ensemble siz
Page 439 and 440:
Figure 7.18 Mine detection ROC curv
Page 441 and 442:
Figure 7.20 Variable importance for
Page 443 and 444:
Solving Multiclass ClassificationPr
Page 445 and 446:
ncols = len(xNum[1])#Labels are int
Page 447 and 448:
featureImportance = featureImportan
Page 449 and 450:
Figure 7.23 is a bar chart showing
Page 451 and 452:
nrows = len(xNum)ncols = len(xNum[1
Page 453 and 454:
print("Best Missclassification Erro
Page 455 and 456:
As before, the Gradient Boosting ve
Page 457 and 458:
Figure 7.25 Glass classifier built
Page 459 and 460:
Figure 7.27 Glass classifier built
Page 461 and 462:
Table 7.1 Performance and Training
Page 463 and 464:
The chapter demonstrated the use of
Page 468 and 469:
®Machine Learning in Python : Esse
Page 470 and 471:
To my children, Scott, Seth, and Ca
Page 472 and 473:
About the Technical EditorDaniel Po
Page 474 and 475:
IndexerJohnna VanHoose DinseCover D
Page 476:
WILEY END USER LICENSEAGREEMENTGo t
show all

Machine Learning in Python Essential Techniques for Predictive Analysis by Michael Bowles (z-lib.org).epub

Create successful ePaper yourself

Delete template?

Save as template?