Shared Gaussian Process Latent Variables Models - Oxford Brookes ...

2.6. GENERATIVE DIMENSIONALITY REDUCTION 37
parameters W of the generative mapping the joint probability of the full model
can be formulated. Formulating an error function of the models means that the
gradients of the unknown latent locations and the parameters of the generative
mapping can be formulated. However, these gradients involve integrals over X
and W which needs to be evaluated using sampling based methods such as Monte
Carlo sampling. Optimizing the parameters W a density over the input space can
be found.
Tipping and Bishop [65] formulated probabilistic PCA (PPCA) by making the
assumption that the observed data was related to the latent locations as a linear
mapping yi = Wxi + ǫ, where ǫ ∼ N(0, β −1 I). Placing a spherical Gaussian
prior over the latent locations leads to the marginal likelihood,
p(y|W, β −1 I) =
p(y|W, β −1 )p(x)dx (2.40)
p(x) = N(0, β −1 I). (2.41)
The parameters of the mapping W can be found by maximum likelihood.
Assuming a linear mapping severely restricts the classes of data-sets that can
be modeled. But the prior over the latent locations has to be propagated through
the generating mapping to form the marginal likelihood. For the linear mapping
Eq 2.40 is solvable. However, when considering mappings of more general form
it is not clear how to propagate the latent prior through the mapping to make the
the integral Eq. 2.40 analytically tractable.
Bishop [9] suggested a specific prior over the latent space making marginal-
ization over more general mappings feasible, the model is referred to as The Gen-
erative Topographic Map (GTM). By discritizing the latent space into regular grid,