Foundations of Data Science

A
B
C
(a) A graph with a single cycle
A B C A B C A B C
(b) Segment of unrolled graph
Figure 9.5: Unwrapping a graph with a single cycle
value of J ′ through that Ψ. Since the algorithm maximizes J k = kJ (A, B, C) + J ′ in the
unwrapped network for all k, it must maximize J (A, B, C). To see this, set the variables
A, B, C, so that J k is maximized. If J (A, B, C) is not maximized, then change A, B, and
C to maximize J (A, B, C). This increases J k by some quantity that is proportional to
k. However, two of the variables that appear in copies of J (A, B, C) also appear in J ′
and thus J ′ might decrease in value. As long as J ′ decreases by some finite amount, we
can increase J k by increasing k sufficiently. As long as all Ψ’s are nonzero, J ′ which is
proportional to log Ψ, can change by at most some finite amount. Hence, for a network
with a single loop, assuming that the message passing algorithm converges, it converges
to the maximum a posteriori assignment.
9.10 Belief Update in Networks with a Single Loop
In the previous section, we showed that when the message passing algorithm converges,
it correctly solves the MAP problem for graphs with a single loop. The message passing
algorithm can also be used to obtain the correct answer for the marginalization problem.
Consider a network consisting of a single loop with variables x 1 , x 2 , . . . , x n and evidence
316