Bernal S D_2010.pdf - University of Plymouth

4.5. FEEDBACK PROCESSING
George and Hawkins 2009).
4.5.2 Multiple parents
As desciibed in Section 3.3.4 the number of elements of the CPT P{X\U],-- ,(/^) is exponen­
tial lo the number of parents, N, as it includes entries for all possible combinaiions of the states
in node X and its parent mxles. e.g. given kx — ku — A.N = 8, the number of parameters in
the CPT is 4 • 4** - 262,144, Additionally, the number of operations to compute the belief is
also exponential lolhe number of parents, more precisely it requires ^ sums and N •k'^ product
operations. The exponential growth to the number of parameters and operations resulting from
the comhinalion of multiple parents is illustrated in Figure 4.14.
4.5.2.1 Weighted .sum or compatible parental configurations
To solve the problem of Ihe large number of entries in the CPT we implemeni the weighted
sum of simpler CPTs based on the concept of compalihle parental configurations (Das 2004)
described in Section 3.3,4, This method ohiains a kx x *(/ CPT. P[X\Ui), between node X and
each of its N parent nodes, and assumes the rest of the parents. Vj, where j / i. are in compatible
states. The hnal CPT F(X|(/i,--.(/AT) is oblained as a weighted sum of the A" P{X\JJi) CPTs.
The total number of parameters required to be learned is therefore linear with the number of
parents, more precisely, kx -k-N • N. Using llie values of the previous example, the number of
elements is now 4 • 4 • 8 - 128, several orders of magnitude smaller than the previous result.
This is illustrated in Figure 4.15.
The Learning section (4.3) described 1) how to obtain the weight matrices between a parent
node and its children, and 2) how to conven these weight matrices into individual CPTs for
each of the child nodes. The resulting CT'T is precisely in the form required to implement ihe
weighted sum method, i.e. for each child node X there are N CPTs of the form /'(X It/,), one for
each of its parents. These can then be combined to form the [\nii\P{X\U\,--- ,UN).
4.5.2.2 Sampling fnmi parent nodes
To reduce the excessive number of operations required lo calculate the belief, only the i^niiK
states, with the highest values, from the NmoK ^ messages, with the higher variance, are used
177