Bernal S D_2010.pdf - University of Plymouth

4.4. FEEDFORWARD PROCFSSING
Figure 4.13 shows the K-L divergence between the true and approximate likelihood dis­
tributions, averaged over 500 trials, as a function of M^^y ^nd the total number of parenls
TV. The likelihood distributions are assumed to have AT = 100 slates. For comparison,
the K-L divergence between Ihe exact likelihood and a randomly generated likelihood
distribution is also plotted.
The results show that the coefficient M,„,i,//V increases as the goodness of fit between
the approximation and the exact solution increases. Also, as the total number of input
messages, N. increases, the goodness of fit decreases. The relative difference between
the K-L divergence of the approximaie and the random distributions suggests that for
values of M,„a:, above a given threshold the approximate distribution provides a good fit
to the exact solution. It is important to note that in the real model data, the input A
messages are correlated (due to the overlap in receptive fields) and are therefore likely to
present more similarities between them than the randomly generated A messages of the
statistical test. Additionally, a subset of Ihe discarded distributions w^iM typically present
near-llat distributions as they originate from blank regions of the image. Consequently,
the approximation in the model will constitute a belter (it to the exact distribution than
that suggested by this empirical test.
• The messages (probabihly distributions) are sum-normalized to 1 and then re-weighted
so thai the minimum value of the distribution is never below V, — ]/{\0- Kx). All
elements of the message that are below Vm,n are set to Vmin- The overall Increase in the
sum of the elements of the resulting distribution is ihen compensated by proportionally
decreasing Ihe remaining elements (those which were not set the to Vmin)- Consequently.
ihe rcsuliing distribution will still be sum-normalized to I, while having a minimum value
equal to V,„,„. The distribution will have a profile equivalent to that of the original one.
except for those elements that were originally below P^,,,, which will now exhibit higher
relative values.
This adjustment of the message probabilily distributions eliminales all values under V„i„,
thus allowing multiplicative combination of a greater number of input messages, i.e. Mmat
174