Decision Trees from large Databases: SLIQ

More documents

Recommendations

Info

Regression Trees • Input: L = x 1 , y 1 , … , x n , y n , continuous y. • Desired result: f: X → Y • Algorithm CART It was developed simultaneously & independently from C4.5. Terminal nodes incorporate continuous values. Algorithm like C4.5. For classification, there are slightly different criteria (Gini instead of IG), but otherwise little difference. Information Gain (& Gini) only work for classification. Question: What is a split criterion for Regression? • Original reference: L. Breiman et. al.: ”Classification and Regression Trees”. 1984 Sawade/Landwehr/Prasse/Scheffer, Maschinelles Lernen 60
Regression Trees • Goal: small quadratic error (SSE) + small tree. • Examples L arrive at the current node. • SSE at the current node: SSE L = x,y ∈L y − y 2 where y = 1 y L x,y ∈L . • With which test should the data be split? • Criterion for test nodes x ≤ v : SSE − Red L, x ≤ v = SSE L − SSE L x≤v − SSE L x>v SSE-Reduction through the test. • Stop Criterion: Do not make a new split unless the SSE will be reduced by at least some minimum threshold. Sawade/Landwehr/Prasse/Scheffer, Maschinelles Lernen If so, then create a terminal node with mean m of L. 61
Page 1 and 2: Decision Trees from large Databases
Page 3 and 4: Compare: Information Gain vs. Gini
Page 5 and 6: SLIQ (1st Substep) • Pre-sorting
Page 7 and 8: SLIQ: Example Sawade/Landwehr/Prass
Page 9 and 10: SLIQ (2nd Substep) • Evaluation o
Page 11 and 12: SLIQ - Algorithm 1. Start Pre-sorti
Page 13 and 14: SLIQ: Example Sawade/Landwehr/Prass
Page 15 and 16: SLIQ- Pruning • Minimum Descripti
Page 17 and 18: Pruning - MDL Example II • Which
Page 19: Regression Trees • ID3, C4.5, SLI
Page 23 and 24: Linear Regression • Regression tr
Page 25 and 26: Model Trees • Decision trees but
Page 27 and 28: Model Trees: Creation of new Test N
Page 29 and 30: Attributes with Costs • Example:
Page 31 and 32: Bootstrapping - Example x = 3.12 0
Page 33 and 34: Random Forests • Repeat: Draw ra
Page 35 and 36: Decision Tree - Advantages • Easy
Page 37: Summary of Decision Trees • Class

Decision Trees from large Databases: SLIQ

Create successful ePaper yourself

Delete template?

Save as template?