Support Vector Machines - The Auton Lab

More documents

Recommendations

Info

Doing multi-class classification • SVMs can only handle two-class outputs (i.e. a categorical output variable with arity 2). • What can be done? • Answer: with output arity N, learn N SVM’s • SVM 1 learns “Output==1” vs “Output != 1” • SVM 2 learns “Output==2” vs “Output != 2” • : • SVM N learns “Output==N” vs “Output != N” • Then to predict the output for a new input, just predict with each SVM and find out which one puts the prediction the furthest into the positive region. Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 63 References • An excellent tutorial on VC-dimension and Support Vector Machines: C.J.C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2):955-974, 1998. http://citeseer.nj.nec.com/burges98tutorial.html • The VC/SRM/SVM Bible: Statistical Learning Theory by Vladimir Vapnik, Wiley- Interscience; 1998 Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 64 32
What You Should Know • Linear SVMs • The definition of a maximum margin classifier • What QP can do for you (but, for this class, you don’t need to know how it does it) • How Maximum Margin can be turned into a QP problem • How we deal with noisy (non-separable) data • How we permit non-linear boundaries • How SVM Kernel functions permit us to pretend we’re working with ultra-high-dimensional basisfunction terms Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 65 33
Page 1 and 2: Support Vector Machines Note to oth
Page 3 and 4: Linear Classifiers x α f y est den
Page 5 and 6: Maximum Margin x α f y est denotes
Page 7 and 8: Computing the margin width “Predi
Page 9 and 10: Computing the margin width The line
Page 11 and 12: Learning the Maximum Margin Classif
Page 13 and 14: Learning the Maximum Margin Classif
Page 15 and 16: denotes +1 denotes -1 Uh-oh! This i
Page 17 and 18: Learning Maximum Margin with Noise
Page 19 and 20: An Equivalent QP Maximize R ∑ α
Page 21 and 22: Suppose we’re in 1-dimension Not
Page 23 and 24: Common SVM basis functions z k = (
Page 25 and 26: Quadratic Dot Products ⎛ ⎜ ⎜
Page 27 and 28: Higher Order Polynomials Polynomial
Page 29 and 30: Maximize QP with Quintic basis func
Page 31: VC-dimension of an SVM • Very ver

Support Vector Machines - The Auton Lab

Create successful ePaper yourself

Delete template?

Save as template?