Shared Gaussian Process Latent Variables Models - Oxford Brookes ...

2.5. NON-LINEAR 27
resentation. This is the fundamental background to the Kernel Trick which
is a way of non-linearizing algorithms that depend only on the inner product
between data-points. Even though an accepted term, it is not clear where the
term was initially suggested. The Kernel Trick is based on that rather than
finding a specific mapping Φ that takes the data to the feature space F we
specify a function k(yi,yj), called the kernel function that parameterizes the
inner product between Φ(yi) and Φ(yj),
k(yi,yj) = Φ(yi) T Φ(yj). (2.32)
Evaluated between each pair of points in the data-set the kernel function k
specifies the kernel matrix K(Y,Y) which specifies the Gram matrix in the
feature space F . From Eq. 2.17 we know that the Gram matrix and a dis-
tance matrix is interchangeable representation for centered data. Therefore
as long as the kernel function k specifies a valid Gram matrix K there is an
underlying geometrical representation of data in F . The class of kernel func-
tions that specifies geometrically representable feature spaces are known as
Mercer Kernels [41, 50]. Mercer Kernels are positive semidefinite, i.e in the
spectral decomposition of the resulting kernel matrix K all eigenvalues are
non-negative. Intuitively this can be understood through Eq. 2.17, if the
eigenvalues where to be negative then by adding basis vectors the distance
between two points would be reduced, which is not possible in a Euclidean
space. Using a kernel function to represent the data the feature space F is
know as a kernel induced feature space.
One advantage using kernel induced feature space is that if we aim to