Support Vector Machines - The Auton Lab

More documents

Recommendations

Info

Find arg max u Quadratic Programming T T u Ru c + d u + 2 Quadratic criterion Subject to And subject to a a a a a a 11 21 u + a u 1 1 + a u + a n1 1 ( n+ 1)1 ( n+ 2)1 ( n+ e)1 u 1 1 1 12 22 n2 u + a u + a + a u u u 2 2 2 ( n+ 1)2 ( n+ 2)2 ( n+ e)2 + ... + a + ... + a : + ... + a u u u 2 2 2 : 1m 2m nm u u u m + ... + a + ... + a + ... + a m m ≤ b 1 ≤ b 2 ≤ b ( n+ 1) m ( n+ e) m n ( n+ 2) m u u u m m m = b = b = b n additional linear inequality constraints ( n+ 1) ( n+ 2) ( n+ e) e additional linear equality constraints Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 23 Find arg max u Quadratic Programming T T u Ru c + d u + 2 Quadratic criterion Subject to And subject to a a a a a a 11 21 u + a u 1 1 + a u + a n1 1 ( n+ 1)1 ( n+ 2)1 ( n+ e)1 1 1 1 12 22 n2 u + a u + a u + a u u u 2 2 2 ( n+ 1)2 ( n+ 2)2 ( n+ e)2 + ... + a + ... + a : + ... + a u u u 2 2 2 : 1m 2m nm u u u m + ... + a + ... + a + ... + a m m ≤ b 1 ≤ b There exist algorithms for finding such constrained quadratic optima much more efficiently and reliably than gradient ascent. 2 ≤ b (But they are very fiddly…you probably don’t want to write one yourself) ( n+ 1) m ( n+ e) m n ( n+ 2) m u u u m m m = b = b = b n additional linear inequality constraints ( n+ 1) ( n+ 2) ( n+ e) e additional linear equality constraints Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 24 12
Learning the Maximum Margin Classifier “Predict Class = +1” zone wx+b=1 wx+b=0 wx+b=-1 “Predict Class = -1” zone M = 2 w.w Given guess of w , b we can • Compute whether all data points are in the correct half-planes • Compute the margin width Assume R datapoints, each (x k ,y k ) where y k = +/- 1 What should our quadratic optimization criterion be? How many constraints will we have? What should they be? Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 25 Learning the Maximum Margin Classifier “Predict Class = +1” zone wx+b=1 wx+b=0 wx+b=-1 “Predict Class = -1” zone M = 2 w.w Given guess of w , b we can • Compute whether all data points are in the correct half-planes • Compute the margin width Assume R datapoints, each (x k ,y k ) where y k = +/- 1 What should our quadratic optimization criterion be? Minimize w.w How many constraints will we have? R What should they be? w . x k + b >= 1 if y k = 1 w . x k + b
Page 1 and 2: Support Vector Machines Note to oth
Page 3 and 4: Linear Classifiers x α f y est den
Page 5 and 6: Maximum Margin x α f y est denotes
Page 7 and 8: Computing the margin width “Predi
Page 9 and 10: Computing the margin width The line
Page 11: Learning the Maximum Margin Classif
Page 15 and 16: denotes +1 denotes -1 Uh-oh! This i
Page 17 and 18: Learning Maximum Margin with Noise
Page 19 and 20: An Equivalent QP Maximize R ∑ α
Page 21 and 22: Suppose we’re in 1-dimension Not
Page 23 and 24: Common SVM basis functions z k = (
Page 25 and 26: Quadratic Dot Products ⎛ ⎜ ⎜
Page 27 and 28: Higher Order Polynomials Polynomial
Page 29 and 30: Maximize QP with Quintic basis func
Page 31 and 32: VC-dimension of an SVM • Very ver
Page 33: What You Should Know • Linear SVM

Support Vector Machines - The Auton Lab

Create successful ePaper yourself

Delete template?

Save as template?