Combining Pattern Classifiers

4 FUNDAMENTALS OF PATTERN RECOGNITION
Fig. 1.2
Types of features.
Figure 1.2. Discrete features with a large number of possible values are treated as
quantitative. Qualitative (categorical) features are these with small number of possible
values, either with or without gradations. A branch of pattern recognition, called
syntactic pattern recognition (as opposed to statistical pattern recognition) deals
exclusively with qualitative features [3].
Statistical pattern recognition operates with numerical features. These include,
for example, systolic blood pressure, speed of the wind, company’s net profit in
the past 12 months, gray-level intensity of a pixel. The feature values for a given
object are arranged as an n-dimensional vector x ¼½x 1 , ..., x n Š T [ R n . The real
space R n is called the feature space, each axis corresponding to a physical feature.
Real-number representation (x [ R n ) requires a methodology to convert qualitative
features into quantitative. Typically, such methodologies are highly subjective and
heuristic. For example, sitting an exam is a methodology to quantify students’ learning
progress. There are also unmeasurable features that we, humans, can assess
intuitively but hardly explain. These include sense of humor, intelligence, and
beauty. For the purposes of this book, we shall assume that all features have numerical
expressions.
Sometimes an object can be represented by multiple subsets of features. For
example, in identity verification, three different sensing modalities can be used
[11]: frontal face, face profile, and voice. Specific feature subsets are measured
for each modality and then the feature vector is composed by three subvectors,
x ¼½x (1) , x (2) , x (3) Š T . We call this distinct pattern representation after Kittler et al.
[11]. As we shall see later, an ensemble of classifiers can be built using distinct pattern
representation, one classifier on each feature subset.