Bernal S D_2010.pdf - University of Plymouth

4.4. F^DFORWARD PROCESSING
u
I
M,^/N
true vs. random
distributions (baseline)
Figure 4. J3: Kullback-Leibler divergence belween the true and Iheapproximale likelihood distrihuiion
for dilfereni values oi' M,,,,^. averaged over 5(X) trials. The Kullhack-
Leibler (K-l.) divergence, on the y-axis, measures the eross-correlation belween
an approximale disirihuiion and the true disirihution. This method cannot be
considered a distance measure, as ii is mil symmetric, bui has been used extensively
U) measure the goodness of fii between lw{) discrete probability distributions
(Frislon and Kiebel 2009, Winn and Bishop 2005, Hinlon el al, 2006) The
X-axis shows the coeflicieni Mm„,/A'. i.e. the percentage of A messages of the total
that are used in the approximation. Three dilierenl values of N are shown: 20
(blue lines), 50 (red lines) and 100 (green lines). The dotted horizontal line shows
ihe K-L divergence between the true and a random distribution, which serves as
a baseline to compare the goodness of fit of die approximate dislribufion.
175