Shared Gaussian Process Latent Variables Models - Oxford Brookes ...

2.2. DIMENSIONALITY REDUCTION 17
2.2 Dimensionality Reduction
In most data-sets that the number of degrees of freedom of the representation is
much higher then that of the intrinsic representation. This is due to the fact that
rather then representing the degrees of freedom of the data the representation is a
reflection of the degrees of freedom in the collection process of the data.
How complex or simple a structure is depends critically upon the way
we describe it. Most of the complex structures found in the world are
enormously redundant, and we can use this redundancy to simplify
their description. But to use it, to achieve the simplication, we must
find the right representation [56].
One example of this is the parameterization of a natural image as a matrix
of real values (pixels). The matrix being captured by a camera, each pixel cor-
responds to a single light sensor which are allowed to vary independently of the
other sensors on the lens. However, this does not correspond well to natural im-
ages as neighboring pixels are strongly correlated [8]. This implies that natural
images have a significantly different intrinsic representation , and degrees of free-
dom, than the image representation of the camera. The correlation between pixels
will, in the vector space spanned by the pixel values, manifest itself as a low-
dimensional manifold. Parameterizing this manifold are the degrees of freedom
for natural images. This example is a simplification as it assumes that the camera
is capable of capturing the full variability or all the degrees of freedom in a nat-
ural image, i.e. that the intrinsic representation can be found as a mapping from
the observed representation. A more general view of the problem that avoids this
assumption is to view the collection process as a mapping from the data’s intrinsic