Applied Statistics Using SPSS, STATISTICA, MATLAB and R

224 6 Statistical Classification
(categorises)
2
ℜ into two decision regions: the upper half plane corresponding to
d(x) > 0 where feature vectors are assigned to ω1; the lower half plane
corresponding to d(x) < 0 where feature vectors are assigned to ω2. The
classification is arbitrary for d(x) = 0.
x 2
o o
o oo
o o o o
o
o o
ω 2
ω 1
+
-
x x
x x
x
x x
x x x
Figure 6.1. Two classes of cases described by two-dimensional feature vectors
(random variables X1 and X2). The black dots are class means.
The generalisation of the linear decision function for a d-dimensional feature
space in
d
ℜ is straightforward:
d ( x)
= w’
x + w , 6.2
0
where w x represents the dot product 1
’
of the weight vector and the d-dimensional
feature vector.
The root set of d(x) = 0, the decision surface, or discriminant, is now a linear
d-dimensional surface called a linear discriminant or hyperplane.
Besides the simple linear discriminants, one can also consider using more
complex decision functions. For instance, Figure 6.2 illustrates an example of
two-dimensional classes separated by a decision boundary obtained with a
quadratic decision function:
2 2
( 5 1 4 2 3 1 2 2 2 1 1 0
d x ) = w x + w x + w x x + w x + w x + w .
6.3
Linear decision functions are quite popular, as they are easier to compute and
have simpler statistical analysis. For this reason in the following we will only deal
with linear discriminants.
1
The dot product x’ y is obtained by adding the products of corresponding elements of the
two vectors x and y.
x 1