Pattern Recognition 1: Introduction - KTH

Classification
Summary
Pattern Recognition 1: Introduction
Arne Leijon
KTH Sound and Image Processing
Aug 27, 2012
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
APattern-ClassificationSystem
Input
signal
Transducer
Feature
Extractor
Classifier
Output
decision
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Source Categories
Noise
Signal Source
Sub-source #1
Sub-source #2
Noise
Noise
Noise
Transducer
Sub-source # N s
Noise
S
Random State
Switch
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Optimal Classification Mechanism
Classifier
Feature Extraction
#1
Discriminant
Function
g 1 ( x)
x
Select
index
of
largest
#K
Discriminant
Function
g N d
( x)
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
First Classifier Example: Man or Woman?
100
90
x 2
= Weight / kg
80
70
60
50
150 160 170 180 190 200
x = Height / cm
1
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Two Gaussian Feature Probability Density Functions
Women
Men
x 10 −3
5
x 10 −3
5
4
4
3
3
2
2
100 0 1
80
x 2
= Weight / kg
60
150
160
190
180
170
x 1
= Height / cm
200
100 0 1
80
x 2
= Weight / kg
60
150
160
190
180
170
x 1
= Height / cm
200
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Contours of Equal Probability Density
Women
100
Men
100
90
90
x 2
= Weight / kg
80
70
x 2
= Weight / kg
80
70
60
60
50
150 160 170 180 190 200
x = Height / cm
1
50
150 160 170 180 190 200
x = Height / cm
1
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Decision Boundary and Decision Regions
100
90
x 2
= Weight / kg
80
70
60
50
150 160 170 180 190 200
x 1
= Height / cm
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Gaussian Probability Density (2-dim)
5
4
3
x 2
2
1
0
−1
−2
−3
−4
−5
−5 −4 −3 −2 −1 0 1 2 3 4 5
x 1
✓ ◆ 3
2
0
cov [X ]=
0 1 2
✓ ◆ 9 0
=
0 1
=
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Gaussian Probability Density (2-dim)
5
4
3
x 2
2
1
0
−1
−2
−3
−4
−5
−5 −4 −3 −2 −1 0 1 2 3 4 5
x 1
✓ ◆ 1
2
0
cov [X ]=
0 3 2
✓ ◆ 1 0
=
0 9
=
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Gaussian Probability Density (2-dim)
5
x 2
4
3
2
1
0
−1
−2
−3
−4
−5
−5 −4 −3 −2 −1 0 1 2 3 4 5
x 1
✓ ◆ 3
2
0
cov [X ]=P
0 1 2 P T =
✓ ◆ 5 4
=
4 5
where
P = p 1 ✓ ◆ 1 1
2 1 1
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
Gaussian Probability Density (K-dim)
f X (x) =
1
(2⇡) K/2p det C e 1 2 (x µ)T C 1 (x µ)
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
General Probability Density: Gaussian Mixture (1-dim)
0.4
0.35
0.3
Prob. Density
0.25
0.2
0.15
0.1
0.05
0
−2 0 2 4 6
x
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
General Probability Density: Gaussian Mixture (2-dim)
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Intro
Example
Probability Density
General Probability Density: Gaussian Mixture (K-dim)
f X (x) =
MX
m=1
w m
1
(2⇡) K/2p det C m
e 1 2 (x µ m )T C 1
m (x µ m )
Arne Leijon
Pattern Recognition 1: Introduction

Classification
Summary
Summary: Important Concepts
Feature Vector: vector x containing all input data to the classifier
Observation Space: set of all possible feature-vector values
Decision Function: d(x) mapsfeaturevectorx into discrete
decision output, coded as integer
Decision Region: set of feature-vector values giving same decision:
⌦ i = {x : d(x) =i}
Discriminant Function: g i (x) mapsanyfeaturevectorx into real
number, used for classification
General Optimal Classifier: d(x) = argmax k g k (x)
Two-category Classifier: can be implemented using a single
Discriminant Function: d(x) = sgn g(x)
Gaussian Mixture Model: can approximate any distribution
Arne Leijon
Pattern Recognition 1: Introduction