Support Vector Machines - The Auton Lab

Maximize
QP with Quintic basis functions
We must do R 2 /2 dot products R R to get this
matrix ready.
∑αk
+ ∑∑αkαlQ
where Q ( ( ). ( ))
kl
kl
= yk
yl
Φ xk
Φ xl
In 100-d, each k = 1dot product k = 1now l=
1 needs 103
operations instead of 75 million
R
But there are still worrying things lurking away.
Subject to these
What are they? 0 ≤ α k
≤ C ∀k
∑ α k
y k
= 0
constraints:
•The fear of overfitting kwith = 1 this enormous
number of terms
Then define:
•The evaluation phase (doing a set of
predictions on a test set) will be very
expensive (why?)
w
=
k s.t.
∑
α k
α
y k k
> 0
Φ ( x
b = y
K
(1 − ε
K
) − x
K
. w
where K = arg max α
k
k
)
K
k
Then classify with:
f(x,w,b) = sign(w. φ(x) -b)
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 55
Maximize
QP with Quintic basis functions
We must do R 2 /2 dot products R R to get this
matrix ready.
∑αk
+ ∑∑αkαlQ
where Q ( ( ). ( ))
kl
kl
= yk
yl
Φ xk
Φ xl
In 100-d, each k = 1dot product k = 1now l=
1 needs 103
The use of Maximum Margin
operations instead of 75 million
magically makes this not a
problem R
But there are still worrying things lurking away.
Subject to these
What are they? 0 ≤ α k
≤ C ∀k
∑ α k
y k
= 0
constraints:
•The fear of overfitting kwith = 1 this enormous
number of terms
Then define:
•The evaluation phase (doing a set of
predictions on a test set) will be very
expensive (why?)
w
=
k s.t.
∑
α k
α
y k k
> 0
Φ ( x
b = y
K
(1 − ε
K
) − x
K
. w
where K = arg max α
k
k
)
K
k
Because each w. φ(x) (see below)
needs 75 million operations. What
can be done?
Then classify with:
f(x,w,b) = sign(w. φ(x) -b)
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 56
28