Bernal S D_2010.pdf - University of Plymouth

3.3. DEFINITION AND MATHFMAnCAL FORMULATION
many common intermediate terms in the expressions for Ihe difTerent marginal probabilities,
which implies a high redundancy and thus low efficiency in the calculations. Additionally, when
new evidence arrives into the network, the effect.^ of the observed node modify the marginal
probabilities of all other nodes, requiring the whole marginalizaiion process to be repeated for
each variable.
Belief propagation, a message-passing algorithm, manages to perform inference in a Bayesian
network in a way that grows only linearly with the number of nodes, as it exploits Ihe common
inlermediaie terms that appear in the calculations. In belief propagation the effects of the obser­
vation are propagated throughout the network by passing messages between nodes. The Hnal
belief, or posterior probability, is computed locally at each node by combining all incoming
messages, i.e. evidence from higher and lower levels.
The belief propagation algorithm is not restricted to solving inference problems in Bayesian net­
works. In fact, a generalized version of the algorilhm, also called the sum-product algorithm,
can be shown to encompass a number of methods from different disciplines such as physics,
digital communications and artihcial intelligence. Some of the methods that can be considered
particular cases of belief propagation arc the forward-backward algorithm, the Viierbi algo­
rithm, decoding algorithms such as turbo-codes, the Kalman lilter and Ihe transfer-matrix in
physics {Yedidia et al. 2003. Kschischang et al. 2001).
To derive particular instantiations of the belief propagation algorithm it is necessary lo consider
maihcmatical scenario.s with very specific conditions in each case. For example, the Kalman fil­
ter is derived from applying ihc Ihe generalized belief propagaiion algorithm to a set of Gaussian
random variables thai follow certain discrete-lime dynamical equations. It is useful lo represent
the different problems using factor graphx, a graph-based language that allows us lo represent a
set Lif variables, together with a generic set of functions which relates different subsets of these
variables (Yedidia et al. 2003). It has been shown that factor graphs can capture a wide range of
malhematical systems, including Markov random fields and Bayesian networks. It is therefore
possible to convert any arbitrary Bayesian network into a precisely mathematically equivalent
factor graph (and vice versa) and apply the generalized belief propagation algorithm to solve
84