Bernal S D_2010.pdf - University of Plymouth

4.5. FEEDBACK PROCES.SING
4.5.3.2 Dynamics of loopy belief pri>pa(>ulion
Loopy belief propagation also requires all A and n messages to be initialized to a Hal disiri-
huiion. so that, even during the lirst iteration, all nodes of the network propagate upward and
downward messages. Except for the A me.ssage.s from the dummy nodes, which will contain
evidence from the image, and the n messages from the root nodes, which will propagate ihe
prior, the rest of Ihe messages will propagate Hal distributions during Ihe first time step. During
the following lime sleps the dummy nodes' evidence will propagate to ihe top layers, merging
with Ihe downward prior informalion and being modulated al each layer by the inherent param­
eters of the network contained in the CPTs, The dynamics of loopy belief propagation in ihe
propo.sed Bayesian network are illuslraled in Figure 4.18.
The computational cost of updating the beliefs of nodes at all layers at every lime step is very
high. An altemaiive approach to reduce this cost is to update only ihe belief of a given layer al
each time step as in tree-structured networks. For the majority of results present in this thesis
the model implemented an Hpwnrrf belief update as opposed to the complelc belief update. This
is illustrated in Figure 4.19. For purposes of clarity each simulation step, z,,,,,,, thus consists of
five loopy belief propagation steps. This simplification of loopy belief propagation is justified in
the sense thai evidence arrives from the lower layer dummy nodes and thus only the belief of the
nodes in ihe adjacen! layer will provide meaningful informalion. All the computation required
to calculate the K messages and beliefs in the upper layers during the first time-steps shown
in Figure 4.18 is now saved. Further, it means evidence propagated from the dummy nodes
will only be modulated at each layer by the initial flat top-down K mes.sages, thus increasing
the chance of a good recognition performance. The main disadvantage of this method is the
asymmetry between bottom-up and top-down propagation of evidence, as a belief update or
''Caption for Figure 4, |8, Dynamics of loopy bcliyt pnipagaiiun in ihc propo.sed nuxlel. Al 1=0 all messages
are inilializcJ in a Hat diMribution (symbolized wiih a / in ihe figure) encepi for Ihe A message from the dummy
nodes and the lup level Jr message or prior distrihuiion. Ai i^l, ouL-e the initial flat jz messages are multiplied by
Ihe correiiponding node CtTs they generate non-flai beliefs and subsequent non-tlai JT and A messages (see Figure
3.7 for a numeric example). The non-flat feedforward k messages, ^dumm\' *ill ^'so modulate Ihe belief at each
node and subsequent A messages generated. However, the A message generated by nodes wiih an incoming flat A
me.iisage will also generate Mat output A messagCfi, For this reason, il takes 4 time-steps (the diameter of the network)
to propagate the lower level evidence, k^umn^. to ihe top node Tlie bottom right image symbolically illusirates the
exisience of loops in the nelwork and how this leads to the recursive nature and doulile-counting of mes.sages in
loopy belief propagation.
184