Optimizing Abstaining Classifiers using ROC Analysis

More documents

Recommendations

Info

Outline 1. ROC Background 2. Tri-State Classifier 1. Cost-Based Model 2. Bounded-Abstention Model 3. Bounded-Improvement Model 3. Experiments, Results 4. Summary 4 August 9, 2005 ICML2005
Notation ! Binary classifier C is a function : i α {+,-}, where i ∈ I is an instance ! Ranker R (a.k.a scoring classifier) is a function attaching rank to an instance i α R, can be converted to a binary classifier C τ using ∀ i : C τ (i) = + ⇔ R(i) ≥ τ 1. ROC Background 2. Abstaining Classifier Cost-Based Bounded-Abstention Bounded-Improvement 3. Experiments, Results 4. Summary ! Abstaining binary classifier A is a classifier that in certain case can refrain from classification. We denote it as attaching a third class “”. 5 August 9, 2005 ICML2005
Page 1 and 2: IBM Zurich Research Laboratory, GSA
Page 3: Motivation ! Abstaining classifiers
Page 7 and 8: ROC Background ! ROC Convex Hull 1.
Page 9 and 10: Cost Minimizing Criteria for One Cl
Page 11 and 12: Metaclassifier A α,β 1. ROC Backg
Page 13 and 14: “Optimal” Metaclassifier A α,
Page 15 and 16: Selecting the Optimal Classifier 1.
Page 17 and 18: Understanding Cost Matrices 1. ROC
Page 19 and 20: Bounded Models ! Problem: 2x3 cost
Page 21 and 22: Bounded-Abstention Model 1. ROC Bac
Page 23 and 24: Bounded-Improvement Model 1. ROC Ba
Page 25 and 26: Experiments ! Tested with 15 UCI KD
Page 27 and 28: Results - Cost-Based Model 1. ROC B
Page 29 and 30: Results - Cost-Based Model ionosphe
Page 31 and 32: IBM Zurich Research Laboratory, GSA
Page 33 and 34: Bibliography (2) ! [NedlerMead65] N
Page 35 and 36: Optimal Classifier Path Optimal cla
Page 37 and 38: k Algorithm - Bounded-Improvement M
Page 39: Cost Matrices ! Theorem. If (*) is

Optimizing Abstaining Classifiers using ROC Analysis

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?