Decision Trees from large Databases: SLIQ

More documents

Recommendations

Info

Pruning – MDL Example I • Assume: 16 binary attributes and 3 classes • cost M, D = cost D M + cost M Cost for the encoding of an internal node: log 2 (m) = log 2 16 = 4 Cost for the encoding of a leaf: log 2 (k) = log 2 (3) = 2 Cost for the encoding of a classification error for n training data points: log 2 (n) 7 Errors C 1 C 2 C 3 Sawade/Landwehr/Prasse/Scheffer, Maschinelles Lernen 56
Pruning – MDL Example II • Which tree is better? 7 Errors Tree 1: Tree 2: C 1 C 1 C 2 C 3 Internal nodes Leaves Errors 4 Errors C 2 C 1 C 2 • Cost for tree 1: 2 ∗ 4 + 3 ∗ 2 + 7 ∗ log 2 n = 14 + 7 log 2 n • Cost for tree 2: 4 ∗ 4 + 5 ∗ 2 + 4 ∗ log 2 n = 26 + 4 log 2 n • If n < 16, then tree 1 is better. • If n > 16, then tree 2 is better. C 3 Sawade/Landwehr/Prasse/Scheffer, Maschinelles Lernen 57
Page 1 and 2: Decision Trees from large Databases
Page 3 and 4: Compare: Information Gain vs. Gini
Page 5 and 6: SLIQ (1st Substep) • Pre-sorting
Page 7 and 8: SLIQ: Example Sawade/Landwehr/Prass
Page 9 and 10: SLIQ (2nd Substep) • Evaluation o
Page 11 and 12: SLIQ - Algorithm 1. Start Pre-sorti
Page 13 and 14: SLIQ: Example Sawade/Landwehr/Prass
Page 15: SLIQ- Pruning • Minimum Descripti
Page 19 and 20: Regression Trees • ID3, C4.5, SLI
Page 21 and 22: Regression Trees • Goal: small qu
Page 23 and 24: Linear Regression • Regression tr
Page 25 and 26: Model Trees • Decision trees but
Page 27 and 28: Model Trees: Creation of new Test N
Page 29 and 30: Attributes with Costs • Example:
Page 31 and 32: Bootstrapping - Example x = 3.12 0
Page 33 and 34: Random Forests • Repeat: Draw ra
Page 35 and 36: Decision Tree - Advantages • Easy
Page 37: Summary of Decision Trees • Class

Decision Trees from large Databases: SLIQ

Create successful ePaper yourself

Delete template?

Save as template?