Methods for Constrained Optimization - MIT Certificate Error

More documents

Recommendations

Info

The following Lagrangian in the primal space can be used to solve the optimization:122minimize L( w, b,α) = w − α [ l D( x ) −1]subject toαα∑k = 1k≥ 0,k = 1, 2, K,[ l D( x ) −1] = 0, k = 1, 2, K pk k k,where the αkare the Lagrange multipliers. Taking the partial derivatives of theLagrangian with respect to w and b produces two equations:ppkkkandw =p∑k = 1αp∑k = 1k l kαk l k= 0ϕkSubstituting these two equations back into the Lagrangian function gives the problemformulation in the dual space:minimize J ( α) α − α ⋅ H ⋅αsubject toαp= ∑1kk = 1 2k≥ 0,k = 1, 2, K,p∑k = 1αk l k= 0pThe kernel function K is contained in thep × p matrix H, which is defined byHkm= lklmK( x , x )kmThe optimal parameters α* can be found by maximizing the Lagrangian in the dual spaceusing either the penalty function method or quadratic programming. The training datathat corresponds to non-zero values in α* are the support vectors. There are usually fewersupport vectors than the total number of data points. The equality constraint in the dualproblem requires at least one support vector in each class. Assuming two arbitrarysupport vectors x ∈ class A and x ∈ class B , the bias b can be found:ABb∗= −12p∑kk = 1∗k[ K( x , x ) + K( x , x )]l α .AkBkThe decision function is therefore:
Each datum x is classified according top∗∗( ) = ∑lK( x , x) + bD x α .kk = 1[ D( x)]sgn .kk6 Performance of SVM for Sample 2-D DataClassification ExampleSince the optimization problem that we are solving has both equality and inequalityconstraints, we can use the methods discussed in section 4.3, i.e. the penalty function andquadratic programming methods. I first demonstrate that these methods work on a simpleset of fabricated 2-D data in Figure 9. I also include the results from MATLAB’s built-inquadratic programming function for comparison in Figure 10.x 1 x 2 l α3 5 +1 04 6 +1 02 2 +1 0.0657 4 +1 09 8 +1 0-5 -2 –1 0-8 2 –1 0.015-2 -9 –1 01 -4 –1 0.05x 1 x 2 l α3 5 +1 04 6 +1 02 2 +1 0.0657 4 +1 09 8 +1 0-5 -2 –1 0-8 2 –1 0.015-2 -9 –1 01 -4 –1 0.05Figure 9. The linear decision function for example data is shown in green. Encircled are the supportvectors. The data point values, labels, and optimal parameters α are shown on the right.
Page 6: so that( x) CT λu = ∇f+ ,x min=
Page 10 and 11: Figure 3. Results of different meth
Page 12 and 13: Figure 5. Results of penalty functi
Page 14 and 15: 5.1.1 Decision FunctionGiven the tr
Page 18 and 19: x 1 x 2 l α3 5 +1 04 6 +1 02 2 +1
Page 20 and 21: Figure 13 compares the run-times of
Page 22 and 23: Figure 15. Percent of misclassified

Methods for Constrained Optimization - MIT Certificate Error

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?