Pattern Classification

More documents

Recommendations

Info

Practical Considerations• Momentum• Prevents the algorithm from getting stuck at plateaus and localminima• Weight decay• Avoid overfitting by imposing the condition that weights must besmall• After each update, weights are decayed by some factor• Related to regularization (also used in SVM)• Hints• Additional input nodes added to NN that are onlyused during training. Help learn better featurerepresentation.
Practical Considerations44• Training setup• Online, stochastic, batch-mode• Stop training• Halt when validation error reaches (first) minimum• Number of hidden layers• More layers -> more complex• Networks with more hidden layers are more prone toget caught in local minima• Smaller the better (KISS)• Criterion function• We talked about squared error, but there are others<strong>Pattern</strong> <strong>Classification</strong>, Chapter 6
Page 1: PatternClassificationAll materials
Page 4 and 5: Natural Neural Net Models• Human
Page 6 and 7: Linear Discriminant Functions• A
Page 8 and 9: 7• There is no automatic method f
Page 10 and 11: 9Pattern Classification, Chapter 6
Page 12 and 13: 11• A single “bias unit” is c
Page 14 and 15: 13• The output units are referred
Page 16 and 17: • General Feedforward Operation -
Page 18 and 19: 17• Eq (8) can be expressed in ne
Page 20 and 21: 19Pattern Classification, Chapter 6
Page 22 and 23: Backpropagation Algorithm21• Any
Page 24 and 25: • Network has two modes of operat
Page 26 and 27: Network Learning• Start with an u
Page 28 and 29: 27Since net k = w kt .y, therefore:
Page 30 and 31: Sensitivity at Hidden Node
Page 32 and 33: • Training protocols:• Stochast
Page 34 and 35: • Learning Curves33• Before tra
Page 36 and 37: Representation at the Hidden Layer
Page 38 and 39: Practical Techniques for ImprovingB
Page 40 and 41: Practical Considerations• Scaling
Page 42 and 43: Total no. of WeightsError per patte

Pattern Classification

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?