Combining Pattern Classifiers

6 FUNDAMENTALS OF PATTERN RECOGNITION
Fig. 1.4 Canonical model of a classifier. The double arrows denote the n-dimensional input
vector x, the output of the boxes are the discriminant function values, g i (x )(scalars), and the
output of the maximum selector is the class label v k [ V assigned according to the maximum
membership rule.
maximum membership rule, that is,
D(x) ¼ v i
[ V , g i (x) ¼ max
i¼1,...,c {g i(x)} (1:3)
Ties are broken randomly, that is, x is assigned randomly to one of the tied classes.
The discriminant functions partition the feature space R n into c (not necessarily
compact) decision regions or classification regions denoted by R 1 , ..., R c
R i ¼ x
x [ Rn , g i (x) ¼ max g k(x) , i ¼ 1, ..., c (1:4)
k¼1,...,c
The decision region for class v i is the set of points for which the ith discriminant
function has the highest score. According to the maximum membership rule (1.3),
all points in decision region R i are assigned in class v i . The decision regions are
specified by the classifier D, or, equivalently, by the discriminant functions G.
The boundaries of the decision regions are called classification boundaries, and contain
the points for which the highest discriminant function votes tie. A point on the
boundary can be assigned to any of the bordering classes. If a decision region R i
contains data points from the labeled set Z with true class label v j , j = i, the classes
v i and v j are called overlapping. Note that overlapping classes for a particular partition
of the feature space (defined by a certain classifier D) can be nonoverlapping if
the feature space was partitioned in another way. If in Z there are no identical
points with different class labels, we can always partition the feature space into