Support Vector Machines - The Auton Lab

Find
arg max
u
Quadratic Programming
T
T u Ru
c + d u +
2
Quadratic criterion
Subject to
And subject to
a
a
a
a
a
a
11
21
u + a
u
1
1
+ a
u + a
n1
1
( n+
1)1
( n+
2)1
( n+
e)1
u
1
1
1
12
22
n2
u + a
u + a
+ a
u
u
u
2
2
2
( n+
1)2
( n+
2)2
( n+
e)2
+ ... + a
+ ... + a
:
+ ... + a
u
u
u
2
2
2
:
1m
2m
nm
u
u
u
m
+ ... + a
+ ... + a
+ ... + a
m
m
≤ b
1
≤ b
2
≤ b
( n+
1) m
( n+
e)
m
n
( n+
2) m
u
u
u
m
m
m
= b
= b
= b
n additional linear
inequality
constraints
( n+
1)
( n+
2)
( n+
e)
e additional linear
equality
constraints
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 23
Find
arg max
u
Quadratic Programming
T
T u Ru
c + d u +
2
Quadratic criterion
Subject to
And subject to
a
a
a
a
a
a
11
21
u + a
u
1
1
+ a
u + a
n1
1
( n+
1)1
( n+
2)1
( n+
e)1
1
1
1
12
22
n2
u + a
u + a
u
+ a
u
u
u
2
2
2
( n+
1)2
( n+
2)2
( n+
e)2
+ ... + a
+ ... + a
:
+ ... + a
u
u
u
2
2
2
:
1m
2m
nm
u
u
u
m
+ ... + a
+ ... + a
+ ... + a
m
m
≤ b
1
≤ b
There exist algorithms for finding
such constrained quadratic
optima much more efficiently
and reliably than gradient
ascent.
2
≤ b
(But they are very fiddly…you
probably don’t want to write
one yourself)
( n+
1) m
( n+
e)
m
n
( n+
2) m
u
u
u
m
m
m
= b
= b
= b
n additional linear
inequality
constraints
( n+
1)
( n+
2)
( n+
e)
e additional linear
equality
constraints
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 24
12