Combining Pattern Classifiers

32 FUNDAMENTALS OF PATTERN RECOGNITION
Sometimes more than two discriminant function might tie at the boundaries. Ties
are resolved randomly.
1.5.5 Bayes Error
Let D be a classifier that always assigns the class label with the largest posterior
probability. Since for every x we can only be correct with probability
P(v i jx) ¼ max
i¼1,...,c {P(v ijx)} (1:44)
there is some inevitable error. The overall probability of error of D is the sum of the
errors of the individual xs weighted by their likelihood values, p(x); that is,
ð
P e (D ) ¼ ½1 P(v i jx)Š p(x) dx (1:45)
R n
It is convenient to split the integral into c integrals, one on each classification
region. For this case class v i will be specified by the region’s label. Then
P e (D ) ¼ Xc
i¼1
ð
R i
½1 P(v i jx)Š p(x) dx (1:46)
S
where R i is the classification region for class v i , R i > R j ¼;for any j = i and
c
i¼1 R i ¼ R n . Substituting Eq. (1.31) into Eq. (1.46) and taking into account
that P c
Ð
i¼1 ¼ Ð R , n
R i
P e (D ) ¼ Xc
i¼1
ð
R i
1
ð
¼ p(x) dx
R n
¼ 1
X c
i¼1
ð
R i
P(v i )p(xjv i )
p(x)
X c
i¼1
ð
R i
p(x) dx (1:47)
P(v i )p(xjv i ) dx (1:48)
P(v i )p(xjv i ) dx (1:49)
Note that P e (D ) ¼ 1 P c (D ), where P c (D ) is the overall probability of correct
classification of D .
Consider a different classifier, D, which produces classification regions
R 1 , ...,R c , R i > R j ¼;for any j = i and S c
i¼1 R i ¼ R n . Regardless of the way
the regions are formed, the error of D is
P e (D) ¼ Xc
i¼1
ð
R i
½1 P(v i jx)Š p(x) dx (1:50)