Accuracy assessment methods and current challenges

More documents

Recommendations

Info

Comparison Very common v. target e.g. or classifier evaluation e.g. z = ˆ κ − ˆ κ ˆ 1 2 2 2 σ κ + σ κ 1 ˆ 2 BUT often inappropriate & pessimistically biased
Comparative analysis Comparative analyses often based on hypothesis testing. e.g. H o – no difference in accuracy H 1 – the accuracy values differ Two types of error: Type I – when H 1 is incorrectly accepted (declare a difference as being significant when it is not). Type II – when H o is incorrectly accepted (fail to detect a meaningful difference that does exist).
Page 1 and 2: Accuracy assessment methods and cha
Page 3 and 4: Challenges include • Class defini
Page 5 and 6: Popular measures e.g. Sensitivity =
Page 7 and 8: NO, because the matrix might look l
Page 9 and 10: Impact on estimation Real accuracy
Page 11 and 12: PART 2: Comparisons Often compare (
Page 13: BUT Aim is often not to estimate ac
Page 17 and 18: Type II error H o is incorrectly ac
Page 19 and 20: Estimating sample size To determine
Page 21 and 22: e.g. common scenario v another accu
Page 23 and 24: So what Remember, important to use
Page 25 and 26: v another accuracy

Accuracy assessment methods and current challenges

Create successful ePaper yourself

Delete template?

Save as template?