Bayesian Programming and Learning for Multi-Player Video Games ...

This model can be used to specify the behavior of units in RTS games. Instead of relying
on a “units push each other” physics model for handling dynamic collision of units, this model
makes the units react themselves to collision in a more realistic fashion (a marine cannot push a
tank, the tank will move). More than adding realism to the game, this is a necessary condition
for efficient micro-management in StarCraft: Brood War, as we do not have control over the
game physics engine and it does not have this “flow-like” physics for units positioning.
5.6 Discussion
5.6.1 Perspectives
Adding a sensory input: height attraction
We make an example of adding a sensory input (that we sometimes use in our bot): height
attraction. From a tactical point of view, it is interesting for units to always try to have the
higher ground as lower ranged units have a high miss rate (almost 50%) on higher positioned
units. For each of the direction tiles Diri, we just have to introduce a new set of sensory variables:
H i∈�0...n� ∈ {normal, high, very_high, unknown} (unknown can be given by the game engine).
P(Hi|Diri) is just an additional factor in the decomposition: JD ← JD × Π n i=1 P(Hi|Diri). The
probability table looks like:
Even more realistic behaviors
Hi diri
normal 0.2
high 0.3
very_high 0.45
unknown 0.05
The realism of units movements can also be augmented with a simple-to-set P(Dir t−1 |Dir t )
steering parameter, although we do not use it in the competitive setup. We introduce
Dir t−1
i∈�1...n� ∈ {T rue, F alse}: the previous selected direction, Dirt−1
i = T iff the unit went
to the direction i, else F alse for a steering (smooth) behavior. The JD would then be
JD × Π n i=1 P(Dirt−1
i |Diri), with P(Dir t−1
i |Diri) the influence of the last direction, either a
dot product (with minimal threshold) as before for the objective and flocking, or a parametrized
Bell shape over all the i.
Reinforcement learning
Future work could consist in using reinforcement learning [Sutton and Barto, 1998] or evolutionary
algorithms [Smith et al., 2010] to learn the probability tables. It should enhance the
performance of our Bayesian units in specific setups. It implies making up challenging scenarii
and dealing with huge sampling spaces [Asmuth et al., 2009]. We learned the distribution of
P(Dmgi|Diri) through simulated annealing on a specific fight task (by running thousands of
games). Instead of having 4 parameters of the probability table to learn, we further restrained
P(Dmgi|Diri) to be a discrete exponential distribution and we optimized the λ parameter.
When discretized back to the probability table (for four values of Dmg), the parameters that
87