Bayesian Programming and Learning for Multi-Player Video Games ...

parameters. By refining P(H|HP, opponent), we are learning the opponent’s inclination for particular
types of tactics according to what is available to them, or for us the effectiveness of our
attack types choices.
6.7.2 Possible improvements
There are several main research directions for possible improvements:
• improving the underlying heuristics: the heuristics presented here are quite simple but
they may be changed, and even removed or added, for another RTS or FPS, or for more
performance. In particular, our “defense against invisible” heuristic could take detector
positioning/coverage into account. Our heuristic on tactical values can also be reworked
to take terrain tactical values into account (chokes and elevation in StarCraft). We now
detail two particular improvements which could increase the performance significantly:
– To estimate T1:n (tactical value of the defender) when we attack, or T A1:n (tactical
values of the attacker) when we defend, is the most tricky of all because it may
be changing fast. For that we use a units filter which just decays probability mass
of seen units. An improvement would be to use a particle filter Weber et al. [2011],
additionally with a learned motion model, or a filtering model adapted to (and taking
advantage of) regions as presented in section 8.1.5.
– By looking at Table 6.2, we can see that our consistently bad prediction across types
is for air attacks. It is an attack type for which is particularly important to predict
to have ranged units (which can attack flying units) to defend, and because the
positioning is so quick (air units are more mobile). Perhaps we did not have enough
data, as our model fares well in ZvZ for which we have much more air attacks, but
they may also be more stereotyped. Clearly, our heuristics are missing information
about air defense positioning and coverage of the territory (this is a downside of region
discretization). Air raids work by trying to exploit fine weaknesses in static defense,
and they are not restrained (as ground attacks) to pass through concentrating chokes.
• improving the dynamic of the model: there is room to improve the dynamics of the model:
considering the prior probabilities to attack in regions given past attacks and/or considering
evolutions of the T ,T A,B,E values (derivatives) in time.
• The discretization that we used may show its limits, though if we want to use continuous
values, we need to setup a more complicated learning and inference process (Monte-Carlo
Markov chain (MCMC)* sampling).
• improving the model itself: finally, one of the strongest assumptions (which is a drawback
particularly for prediction) of our model is that the attacking player is always
considered to attack in this most probable regions. While this would be true if the
model was complete (with finer army positions inputs and a model of what the player
thinks), we believe such an assumption of completeness is far fetched. Instead we should
express that incompleteness in the model itself and have a “player decision” variable
D ∼ Multinomial(P(A1:n, H1:n), player).
113