Combining Pattern Classifiers

26 FUNDAMENTALS OF PATTERN RECOGNITION
Equation (1.31) gives the probability mass function of the class label variable v
for the observed x. The decision for that particular x should be made with respect to
the posterior probability. Choosing the class with the highest posterior probability
will lead to the smallest possible mistake when classifying x.
The probability model described above is valid for the discrete case as well. Let x
be a discrete variable with possible values in V ¼ {v 1 , ..., v s }. The only difference
from the continuous-valued case is that instead of class-conditional pdf, we use
class-conditional probability mass functions (pmf), P(xjv i ), giving the probability
that a particular value from V occurs if we draw at random an object from class
v i . For all pmfs,
0 P(xjv i ) 1, 8x [ V, and
X s
j¼1
P(v j jv i ) ¼ 1 (1:32)
1.5.2 Normal Distribution
An important example of class-conditional pdf is the normal distribution denoted
p(xjv i ) N(m i ,S i ), where m i [ R n , and S i are the parameters of the distribution.
m i is the mean of class v i , and S i is an n n covariance matrix. The classconditional
pdf is calculated as
1
p(xjv i ) ¼ pffiffiffiffiffiffiffi
exp
(2p) n=2 jS i j
1
2 (x m i) T S 1
i (x m i )
(1:33)
where jS i j is the determinant of S i . For the one-dimensional case, x and m i are
scalars, and S i reduces to the variance of x for class v i , denoted s 2 i . Equation
(1.33) simplifies to
"
p(xjv i ) ¼ pffiffiffiffiffiffi
1
#
1 x m 2
exp
i
2p si 2 s i
(1:34)
The normal distribution (or also Gaussian distribution) is the most natural
assumption reflecting the following situation: there is an “ideal prototype” of
class v i (a point in R n ) and all class members are distorted versions of it. Small distortions
are more likely to occur than large distortions, causing more objects to be
located in the close vicinity of the ideal prototype than far away from it. The prototype
is represented by the population mean m i and the scatter of the points around it
is associated with the covariance matrix S i .
Example: Data Cloud Shapes and the Corresponding Covariance
Matrices. Figure 1.8 shows four two-dimensional data sets generated from normal
distributions with different covariance matrices as displayed underneath the respective
scatterplot.