A Tutorial on Support Vector Machines for Pattern Recognition

Recommendations

Info

14x i w + b +1 ; i for y i =+1 (40)x i w + b ;1+ i for y i = ;1 (41) i 0 8i: (42)P Thus, for an error to occur, the corresponding i must exceed unity, soi i is an upperbound on the number of training errors. Hence a natural way to assign an extra costfor P errors is to change the objective function to be minimized from kwk 2 =2tokwk 2 =2+C (i i) k , where C is a parameter to be chosen by the user, a larger C corresponding toassigning a higher penalty to errors. As it stands, this is a convex programming problemfor any positive integer k for k = 2 and k = 1 it is also a quadratic programming problem,and the choice k = 1 has the further advantage that neither the i , nor their Lagrangemultipliers, appear in the Wolfe dual problem, which becomes:Maximize:L D X isubject to: i ; 1 2Xij i j y i y j x i x j (43)0 i C (44)Xi i y i =0: (45)The solution is again given byw =XN Si=1 i y i x i : (46)where N S is the number of support vectors. Thus the only dierence from the optimalhyperplane case is that the i now have an upper bound of C. The situation is summarizedschematically in Figure 6.We will need the Karush-Kuhn-Tucker conditions for the primal problem. The primalLagrangian isL P = 1 2 kwk2 + C X i i ; X i i fy i (x i w + b) ; 1+ i g; X i i i (47)where the i are the Lagrange multipliers introduced to enforce positivity of the i . TheKKT conditions for the primal problem are therefore (note i runs from 1 to the number oftraining points, and from 1 to the dimension of the data)@L P@w = w ; X i i y i x i =0 (48)@L P@b= ;X i i y i =0 (49)
15@L P@ i= C ; i ; i =0 (50)y i (x i w + b) ; 1+ i 0 (51) i 0 (52) i 0 (53) i 0 (54) i fy i (x i w + b) ; 1+ i g =0 (55) i i =0 (56)As before, we can use the KKT complementarity conditions, Eqs. (55) and (56), todetermine the threshold b. Note that Eq. (50) combined with Eq. (56) shows that i =0if i
Page 2: 2and Smola, 1996). In most of these
Page 7 and 8: 7is not valid, nearest neighbour cl
Page 9 and 10: 9Origin-b|w|wH 1H 2MarginFigure 5.L
Page 11 and 12: 11which i 6= 0 and computing b (no
Page 13: 13Hencekwk 2 ==n+1Xij=1n+1Xi=1 i j
Page 17 and 18: 17K(x i x j )=e ;kxi;xjk2 =2 2 : (
Page 19 and 20: 19if and only if, for any g(x) such
Page 21 and 22: 214.3. Some Examples of Nonlinear S
Page 23 and 24: 23y i (w x i + b ) = y i ((1 ; )
Page 25 and 26: 25We also use a \sticky faces" algo
Page 27 and 28: 27Proof: If the minimal embedding s
Page 29 and 30: 29Figure 11. Gaussian RBF SVMs of s
Page 31 and 32: 31Thus for n even the maximum numbe
Page 33 and 34: 33with solution given byC = X i i (
Page 35 and 36: 358. LimitationsPerhaps the biggest
Page 37 and 38: 37Pi =1 0 i 1. Then w x+b = P i
Page 39 and 40: 39from the set can be found which a
Page 41 and 42: 414. Given the name \test set," per
Page 43: 43Edgar Osuna and Federico Girosi.

A Tutorial on Support Vector Machines for Pattern Recognition

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?