Bernal S D_2010.pdf - University of Plymouth

3.4. EXISTING MODELS
aleni lo finding the means of the unknown causes of sensory data given the generative mtxlel.
The specific form of the generative model is given by the equations of a hierarchical dynamic
model which impose slructural and dynamical constraints on the inference process. Solving
these equations implies implementing a message-passing algorithm reminiscent of the predic­
tive coding scheme.
Frislon (2005) then reviews anatomical and physiological data from the brain, suggesting the
proposed hierarchical dynamical system and message-passing scheme could be implenienied by
the cortex. At the same time brain responses related lo perception and action can be understood
in teims of the proposed model. However, the model remains in a relatively theoretical fomi
and is only applied practically to two simple scenarios: a birdsong recognition problem, and a
4-pixel image recognition. The second example, more relevant for this section, comprises a a
2-layer network which illustrates the dynamics of the free-energy model and how the prediction
error is reduced after the parameters are gradually learned.
A similar approach was previously implemented by Rao and Ballard (1999) using the Kalman
filter, which is derived from ihc Minimum Description Length principle, similar in flavour lo
free-energy minimization. The model could have been included in this section as it employs a
variational approximation, but was previously described in Section 2.2.3, together with other
predictive coding models of the visual .system.
The model by Murray and Kreulz-Dclgado (2(H)7) also attempts to solve several visual percep­
tual tasks such as recognition or reconslruclion, formulating them as inference problems in a
stochastic generative model. The joint probability distribution is defined using the neighbour­
ing layer conditional probability (NLCP). which stales that the nodes of a layer only depend
on the nodes of its immediaie neighbouring layers (closely related to belief propagation in
Bayesian networks). The NLCl's can conveniently be formulated using Boltzmann-like distri­
butions. A variational approximation (factorial Bernoulli distribution) is employed to deal with
the intractable exact inference problem. This leads to the development of a simplified genera­
tive model which can be implemented using a hierarchical dynamic network with feedforward,
feedback and lateral connections. The model places a strong focus on overcomplete sparse
130