Bernal S D_2010.pdf - University of Plymouth

3.4. EXISTING MODELS
e.g. 8^/(^2^') = P(x2\^]' ',y-(..-.,-t'n)- ^^ second algorithm provides a less computa­
tionally demanding alternative, by choosing a value at random for each variable of the
distribulion and ihcn calculating the acceptance probahiliiy of the new distribution. Both
methods applied lo graphical models yield a message-passing algorithm similar to belief
propagation.
In importance sampling (also called panicle filtering), on the other hand, samples are
chosen from a similar but simpler distribulion than the original joint probability disiri-
buiion. This simpler distribulion can be obtained by simplifying the original graph, for
example, by deleting edges. The .samples arc then re-weighted appropriately.
• Variational approximation: Variational methods, such as the mean fie hi approximation,
convert the probabilistic inference problem into an optimization problem. The basic ap­
proach is to choose from a family of approximate distributions by introducing a new
parameter for each node, called a variational parameter. These variational parameters
are updated iteratively as to minimize the variational free energy of the system, which is
equivalent to the cross-entropy (Kullback-I.eibler divergence) between the approximate
and the true probability distributions. When the variational free energy is minimum, the
approximate and Ihe true probabihty distributions are equivaleni. More elaborate approx­
imations lo the free energy, such as the Bethe free energy, provide better approximate
marginal probabilities (Jordan and Weiss 2002. Murphy 2001, Winn and Bishop 2005).
This method has become more popular in recent years due to the high computational
cost of sampling methods, li is currently being used by several research groups to model
complex systems such as the visual system (Frision and Kiebel 2(XJ9, Hinion et al. 2006).
Section 3.4.2 describes some of these models.
3.4 Existing models
Bayesian inference has been employed extensively to model different aspects of conical pro­
cessing, from singie neuron spikes (Deneve 2005) to higher-level functions, such as object per­
ception (Kersten et al. 2004) and decision making (C:hater et al. 2006). In this section, the focus
11.3