Foundations of Data Science

∑
x 2 ,x 3
x 1 (x 1 + x 2 + x 3 )(2 + x 3 ) = 10x 2 1 + 11x 1
x 1 ↑
x 1
(x 1 + x 2 + x 3 )(2 + x 3 ) ↑
x 1 x 1 + x 2 + x 3
1 ↑
∑
x 4 ,x 5
(x 3 + x 4 + x 5 )x 4 x 5
= 2 + x 3 ↑
x 2 x 3
(x 3 + x 4 + x 5 )x 4 x 5 ↑
x 3 + x 4 + x 5
x 4 ↑
x 5 ↑
x 4 x 5
x 4 ↑
x 5 ↑
x 4 x 5
Figure 9.4: Messages.
downwards for this pass. Each node waits until its parent has sent it a message before
sending messages to each of its children. The rules for messages are:
Rule 1 The message from a factor node v to a child x i , which is the variable node x i ,
is the product of all messages received by v in both passes from all nodes other than x i
times the factor at v itself.
Rule 2 The message from a variable node x i to a factor node child, v, is the product
of all messages received by x i in both passes from all nodes except v, with all variables
except x i summed out. The message is a function of x i alone.
At termination, when the graph is a tree, if we take the product of all messages received
in both passes by a variable node x i and sum out all variables except x i in this
product, what we get is precisely the entire function marginalized to x i . We do not give
the proof here. But the idea is simple. We know from the first pass that the product of
314