Applied Statistics Using SPSS, STATISTICA, MATLAB and R

244 6 Statistical Classification
Figure 6.14. Classification results obtained with STATISTICA, of two classes of
cork stoppers using: (a) Ten features; (b) Four features.
Let us denote:
Pe – Probability of error of a given classifier;
Pe *
– Probability of error of the optimum Bayesian classifier;
Ped(n) – Training (design) set estimate of Pe based on a classifier
designed on n cases;
Pet(n) – Test set estimate of Pe based on a set of n test cases.
The quantity Ped(n) represents an estimate of Pe influenced only by the finite
size of the design set, i.e., the classifier error is measured exactly, and its deviation
from Pe is due solely to the finiteness of the design set. The quantity Pet(n)
represents an estimate of Pe influenced only by the finite size of the test set, i.e., it
is the expected error of the classifier when evaluated using n-sized test sets. These
quantities verify Ped(∞) = Pe and Pet(∞) = Pe, i.e., they converge to the theoretical
value Pe with increasing values of n. If the classifier happens to be designed as an
optimum Bayesian classifier Ped and Pet converge to Pe * .
In normal practice, these error probabilities are not known exactly. Instead, we
compute estimates of these probabilities, Pˆ
ed
and Pˆ
et
, as percentages of
misclassified cases, in exactly the same way as we have done in the classification
matrices presented so far. The probability of obtaining k misclassified cases out of
n for a classifier with a theoretical error Pe, is given by the binomial law:
⎛ n⎞
k n−k
P(
k)
= ⎜ Pe − Pe
k ⎟ ( 1 ) . 6.26
⎝ ⎠
The maximum likelihood estimation of Pe under this binomial law is precisely
(see Appendix C):
P ˆ e = k / n , 6.27
with standard deviation:
Pe ( 1−
Pe)
σ =
. 6.28
n