The error rate of learning halfspaces using kernel-SVM

( )that in this case Err µN ,hinge(Λ w,b ) ≤ l(0)+1+ 2 ∂ +m. Indeed,l(0)Err µN ,hinge(Λ w,b ) = E(x,y)∼µ Nl hinge (yΛ w,b (x))= E(x,y)∼D Nl hinge (yΛ w,b (P ˜Mx))= E(x,y)∼D Nl hinge (yΛ w,b (x + (x − P ˜Mx)))≤≤≤E l hinge (yΛ w,b (x)) + ‖w‖ H · d(x, P ˜Mx)(x,y)∼D NErr DN ,hinge(Λ w,b ) + C · mC1|∂ + l(0)| Err D N ,l(Λ w,b ) + 1 + m≤ l(0) + 1∂ + l(0) m + 2mBy the properties of µ N , it follows that ‖Λ P ˜M w,b‖ 1 = O (m 3 ) (here, the constant in the big-Onotation depends only on l). Finally, we have,‖Λ w ‖ 2 s = ‖Λ P ˜Mw‖ 2 s + ‖Λ P ˜M⊥w‖ 2 s=≤2‖Λ P ˜Mw‖ 2 1 + 2m2C ‖Λ 2 P ˜M⊥w‖ 2 HO ( m 6) + 2m 2 ≤ O ( m 6) ✷5.3 Symmetric Kernels and SymmetrizationIn this section we concern symmetric kernels. Fix d ≥ 2 and let k : S d−1 × S d−1 → R be acontinuous positive definite kernel. We say that k is symmetric if∀A ∈ O(d), x, y ∈ S d−1 , k(Ax, Ay) = k(x, y)In other words, k(x, y) depends only on 〈x, y〉 R d. A RKHS is called symmetric if its reproducingkernel is symmetric. The next theorem characterize symmetric RKHSs. We notethat Theorems of the same spirit have already been proved (e.g. (Schoenberg, 1942)).Theorem 5.17 Let k : S d−1 ×S d−1 → R be a normalized, symmetric and continuous kernel.Then,1. The group O(d) acts on H k . That is, for every A ∈ O(d) and every f ∈ H k if holdsthat f ◦ A ∈ H k and ||f|| Hk = ||f ◦ A|| Hk .2. The mapping ρ : O(d) → U(H k ) defined by ρ(A)f = f ◦A −1 is a unitary representation.3. The space H k consists of continuous functions.24