Methods for Constrained Optimization - MIT Certificate Error

More documents

Recommendations

Info

5.1.1 Decision FunctionGiven the training data, the algorithm must derive a decision function (i.e. classifier)D( x)to divide the two classes. The decision function may or may not be linear in xdepending on the properties of the corpus and what kind of kernel is chosen. For ourproblem formulation, a decision value of D ( x) = 0 would be on the boundary betweenthe classes. Any new, unknown vector x can be classified by determining its decisionvalue: a positive decision value places x in class A, and a negative value places x in classB.The decision function is a multidimensional surface that maximizes the minimum marginof the data in the two classes to the decision function. In our simple pedagogical example,D is some linear function of x = (x, y) that divides the two classes by maximizing thedistance between itself and the nearest points in the corpus, as shown in Figure 7.D = f ( x,y)max dmax dmax dFigure 7. Decision function for the pedagogical exampleThe decision function of an unknown input vector x can be described in either primalspace or dual space. In primal space, the representation of the decision function is:p( ) w ( x) bD x = ∑ kϕk+ ,k =1where theϕ iare previously defined functions of x, and w iand b are adjustable weightparameters of D. Here, b is the bias, i.e. offset of D. In dual space, the decision functionis represented by:p( ) K( x ,x) bD x = ∑α k k+ ,k =1where the akand b are adjustable parameters, and xk are the training patterns. The primaland dual space representations of D, i.e. Equations 2 and 3, are related to each other viathe following relation:
w i=p∑k = 1k( x )α ϕ .ikK is a predefined kernel function of the following form for polynomial decision functions:T( , x') = ( x ⋅ x'+ ) dK x 1 .5.1.2 1.1.3 Maximizing the MarginAssume the training data is separable. Given n-dimensional vector w and biasb, D( x) w is the distance between the hypersurface D and pattern vector x. Define M asthe margin between data patterns and the decision boundary:lkDw( x )k≥ MMaximizing the margin M produces the following minimax problem to derive the mostoptimal decision function:max min lD( x )k kw, w = 1 k.Figure 8 shows a visual representation of obtaining the optimal decision function.wM * M *ϕ ( x ) = x0D( x) < 0D( x ) = w ⋅ x + b = 0D( x) > 0Figure 8. Finding the decision function. The decision function is obtained by determining themaximum margin M*. Encircled are the three support vectors.
Page 6: so that( x) CT λu = ∇f+ ,x min=
Page 10 and 11: Figure 3. Results of different meth
Page 12 and 13: Figure 5. Results of penalty functi
Page 16 and 17: The following Lagrangian in the pri
Page 18 and 19: x 1 x 2 l α3 5 +1 04 6 +1 02 2 +1
Page 20 and 21: Figure 13 compares the run-times of
Page 22 and 23: Figure 15. Percent of misclassified

Methods for Constrained Optimization - MIT Certificate Error

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?