The error rate of learning halfspaces using kernel-SVM

Lemma 5.11 (John’s Lemma) (Matousek, 2002) Let V be an m-dimensional real vectorspace and let K be a full-dimensional compact convex set. There exists an inner product onV so that K is contained in a unit ball and contains a ball of radius 1 , both are centered atm(the same) x ∈ K. Moreover, if K is 0-symmetric it is possible to take x = 0 and the ratiobetween the radiuses can be improved to √ m.Lemma 5.12 Let l be a convex surrogate, let V an m-dimensional vector space and letX ⊂ V be a bounded subset that spans V as an affine space. There exists an inner product〈·, ·〉 on V and a probability measure µ N such that• For every w ∈ V, b ∈ R, ||w|| ≤ 4m 2 Err µN ,hinge(Λ w,b )• X is contained in a unit ball.Proof Let us apply John’s Lemma to K = conv(X ). It yields an inner product on V withK contained in a unit ball and containing the ball with radius 1 both centered at the samemx ∈ V . It remains to prove the existence of the measure µ N . W.l.o.g., we assume that x = 0.Let e 1 , . . . , e m ∈ V be an orthonormal basis. For every i ∈ [m], represent both 1 e m i and− 1 e m i as a convex combination of m + 1 elements from X :Now, definem+11m e ∑i =j=1λ j i xj i , − 1 m e i =m+1∑j=1ρ j i zj i .µ N (x j i , 1) = µ N(x j i , −1) = λj i4m , µ N(z j i , 1) = µ N(z j i , −1) = ρj i4m .20