Support Vector Machines - The Auton Lab

More documents

Recommendations

Info

Computing the margin width What we know: • w . x + + b = +1 • w . x - + b = -1 • x + = x - + λ w • |x + - x - | = M It’s now easy to get M in terms of w and b “Predict Class = +1” zone wx+b=1 wx+b=0 wx+b=-1 x + “Predict Class = -1” zone M = Margin Width w . (x - + λ w) + b = 1 => w . x - + b + λ w .w = 1 => -1 + λ w .w = 1 Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 19 x - => λ = 2 w.w Computing the margin width What we know: • w . x + + b = +1 • w . x - + b = -1 • x + = x - + λ w • |x + - x - | = M • 2 λ = w.w “Predict Class = +1” zone wx+b=1 wx+b=0 wx+b=-1 x + “Predict Class = -1” zone M = Margin Width = M = |x + - x - | =| λ w |= Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 20 x - = λ | w | = λ w. w 2 w. w = = w. w 2 w. w 2 w.w 10
Learning the Maximum Margin Classifier “Predict Class = +1” zone wx+b=1 wx+b=0 wx+b=-1 “Predict Class = -1” zone Given a guess of w and b we can • Compute whether all data points in the correct half-planes • Compute the width of the margin So now we just need to write a program to search the space of w’s and b’s to find the widest margin that matches all the datapoints. How? Gradient descent? Simulated Annealing? Matrix Inversion? EM? Newton’s Method? x + M = Margin Width = Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 21 x - 2 w. w Learning via Quadratic Programming • QP is a well-studied class of optimization algorithms to maximize a quadratic function of some real-valued variables subject to linear constraints. Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 22 11
Page 1 and 2: Support Vector Machines Note to oth
Page 3 and 4: Linear Classifiers x α f y est den
Page 5 and 6: Maximum Margin x α f y est denotes
Page 7 and 8: Computing the margin width “Predi
Page 9: Computing the margin width The line
Page 13 and 14: Learning the Maximum Margin Classif
Page 15 and 16: denotes +1 denotes -1 Uh-oh! This i
Page 17 and 18: Learning Maximum Margin with Noise
Page 19 and 20: An Equivalent QP Maximize R ∑ α
Page 21 and 22: Suppose we’re in 1-dimension Not
Page 23 and 24: Common SVM basis functions z k = (
Page 25 and 26: Quadratic Dot Products ⎛ ⎜ ⎜
Page 27 and 28: Higher Order Polynomials Polynomial
Page 29 and 30: Maximize QP with Quintic basis func
Page 31 and 32: VC-dimension of an SVM • Very ver
Page 33: What You Should Know • Linear SVM

Support Vector Machines - The Auton Lab

Create successful ePaper yourself

Delete template?

Save as template?