A Tutorial on Support Vector Machines for Pattern Recognition

19if and only if, for any g(x) suchthatZg(x) 2 dx is nite (66)thenZK(x y)g(x)g(y)dxdy 0: (67)Note that for specic cases, it may not be easy to check whether Mercer's condition issatised. Eq. (67) must hold for every g with nite L 2 norm (i.e. which satises Eq. (66)).However, we can easily prove that the condition is satised for positive integral powers ofthe dot product: K(x y) =(x y) p . We must show thatZ(dXi=1x i y i ) p g(x)g(y)dxdy 0: (68)The typical term in the multinomial expansion of ( P di=1 x iy i ) p contributes a term of theformZp!r 1 !r 2 ! (p ; r 1 ; r 2 )!x r11 xr2 2 yr1 1 yr2 2g(x)g(y)dxdy (69)to the left hand side of Eq. (67), which factorizes:Zp!=r 1 !r 2 ! (p ; r 1 ; r 2 )! ( x r11 xr2 2 g(x)dx)2 0: (70)One simple consequence is that anykernel which can be expressed as K(x y) = P 1p=0 c p(xy) p , where the c p are positive real coecients and the series is uniformly convergent, satisesMercer's condition, a fact also noted in (Smola, Scholkopf and Muller, 1998b).Finally, what happens if one uses a kernel which does not satisfy Mercer's condition? Ingeneral, there may exist data such that the Hessian is indenite, and for which the quadraticprogramming problem will have no solution (the dual objective function can become arbitrarilylarge). However, even for kernels that do not satisfy Mercer's condition, one mightstill nd that a given training set results in a positive semidenite Hessian, in which case thetraining will converge perfectly well. In this case, however, the geometrical interpretationdescribed above is lacking.4.2. Some Notes on and HMercer's condition tells us whether or not a prospective kernel is actually a dot productin some space, but it does not tell us how to construct or even what H is. However, aswith the homogeneous (that is, homogeneous in the dot product in L) quadratic polynomialkernel discussed above, we can explicitly construct the mapping for some kernels. In Section6.1 we show howEq. (62) can be extended to arbitrary homogeneous polynomial kernels,. Thus forand that the corresponding space H is a Euclidean space of dimension ; d+p;1pexample, for a degree p = 4 polynomial, and for data consisting of 16 by 16 images (d=256),dim(H) is 183,181,376.Usually, mapping your data to a \feature space" with an enormous number of dimensionswould bode ill for the generalization performance of the resulting machine. After all, the