Bayesian Programming and Learning for Multi-Player Video Games ...

Table 7.4 shows the full results, the first line has to be interpreted as “without noise, the
average of the following measures are” (columns):
• d for k = 0:
– d(best, real) = 0.535, it means that the average distance from the most probable
(“best”) build tree to the real one is 0.535 building.
– d(bt, real) ∗ P(bt) = 0.870, it means that the average distance from each value bt of
the distribution on BT , weighted by its own probability (P(bt)), to the real one is
0.87 building.
• k for d = 1:
– best k = 1.193, it means that the average number of buildings ahead (of their construction)
that the model predicts at a fixed maximum error of 1 (d = 1) for the most
probable build tree (“best”) is 1.193 buildings.
– “mean” k = 3.991, it means that the average number of buildings ahead (of their
construction) that the model predicts at a fixed maximum error 1 (d = 1), summed
for bt in BT value and weighted by P(bt) is 3.991 5
• idem that the bullet above but k for d = 2 and d = 3.
The second line (“min”) is the minimum across the different match-ups* (as they are detailed for
noise = 10%), the third (“max”) is for the maximum across match-ups, both still at zero noise.
Subsequent sets of lines are for increasing values of noise (i.e. missing observations).
Predictive power
To test the predictive power of our model, we are interested at looking at how big k is (how
much “time” before we can predict a build tree) at fixed values of d. We used d = 1, 2, 3: with
d = 1 we have a very strong sense of what the opponent is doing or will be doing, with d = 3,
we may miss one key building or the opponent may have switched of tech path.
We can see in Table 7.4 that with d = 1 and without noise (first line), our model predicts
in average more than one building in advance (k > 1) what the opponent will build next if we
use only its best prediction. If we marginalize over BT (sum on BT weighted by P(bt)), we
can almost predict four buildings in advance. Of course, if we accept more error, the predictive
power (number of buildings ahead that our model is capable to predict) increases, up to 6.122
(in average) for d = 3 without noise.
Robustness to missing informations (“noise”)
The robustness of our algorithm is measured by the quality of the predictions of the build trees
for k = 0 (reconstruction, estimation) or k > 0 (prediction) with missing observations in:
P(BT t+k |T = t, O1:N = partial(o1:N), λ = 1)
5 This shows that when the most probable build tree is mistaken, there is 1) not much confidence in it
(otherwise we would have P(btbest) ≈ 1.0 and the “mean” k for fixed d values would equal the “best” one). 2)
much information in the distribution on BT , in the subsequent P(bt¬best) values.
133