Bernal S D_2010.pdf - University of Plymouth

3.3. OmmnON AND MMOEMATICAL FORMULATION
3.3.5 Networks with loops and inference methods
A loop is a chain where at least one node is visiled more than once, as described in Section 3.3.2
and illustrated in Figure 3.3b, Loops arc very common in Bayesian networks which try to model
real-world data. The belief propagalion equations described for singly connected networks are
not correct for multiply connected networks (those with loops). The reason is thai the equations
are based on the a.ssumption that all parents of a node X are mutually independent as long
as none of iheir common descendants are instantiated. This assumption is no longer valid in
networks with loop.s, where some of the parents of JV will share a common ancestor.
Consider, for example, the network in 3.4, with nodes Surfing and Fishinf- having a common
child node Waier poUulion (which we assume can be caused by both fishing and surfing activ­
ity). The conditional independence of the parent nodes would not be satisfied, as they would
both share a common cause, i.e. IVflvcs. To illustrate ihe recursiveness of the loop, consider the
7t message from Surjin^ lo Water pollulion. It would convey top-down evidence from Waves,
which in turn would include evidence from its descendants Fishing and Walerpollution.
Several methods have been developed to deal with the problem of multiply-connected graphs.
Exact inference methods all have a complexity that is exponential to the width of the network.
Approximate inference methods are designed to reduce the processing complexily. although the
trade-off is reduced accuracy of the result. Most approximate inference methods yield message-
passing algorithms which can be implemented in a dislribuled manner, equivalent lo the original
belief propagation. Note these methods are used not only tor networks with loops, but also for
networks with other type of complexities, such as high fan-in or a large number of layers.
3.3.5.1 F.xact inference methods
• Clustering/junction tree algorithm: This method provides exact marginalization of multi­
ply connected Bayesian networks. Ii entails performing belief propagalion on a modified
version of the Bayesian network called a junction tree. The junction tree is an undi­
rected graph in which groups of nodes are clustered logelher into single nodes in order to
eliminate the cycles. The algorithm can be very computationally expensive, specially for
111