Shared Gaussian Process Latent Variables Models - Oxford Brookes ...

2.7. GAUSSIAN PROCESSES 45
2.7.3 Relevance Vector Machine
In this thesis our main use of Gaussian Processes will be as a tool to model
functions. A different regression model is the Relevance Vector Machine
(RVM) [63, 64]. In the RVM the mapping yi = f(xi) is modeled as a lin-
ear combination of a the response to a kernel function of the training data,
f(xi) =
N
wjc(xi,xj) + w0, (2.51)
j=1
where w = [w0, . . .,wN] are the model weights and c(·,xj) the kernel basis
functions. One approach to find the weights of the model would be to min-
imizes a reconstruction error of the training data. However, this is likely to
lead to sever over-fitting as we are trying to estimate N + 1 parameters from
given N inputs. Further, predications would only be point-estimates with no
associated uncertainty.
The RVM was suggested as a model to tackle the above issues. The model
specifies a likelihood model of the data through which the parameters can
be found associating each prediction with an uncertainty. Further, to avoid
over-fitting of the data, a prior is specified over the weights w. This prior en-
courages the model to push as many weights wi towards 0 making the linear
combination in Eq. 2.51 depend on as few basis functions k(·,xj) as possible.
Assuming additive Gaussian noise the likelihood of the model is formu-
lated as,
p(y|w, σ 2 ) =
1
2πσ2 2 1
(−
e 2σ2 ||y−˜ Cx|| 2 ) , (2.52)