Bayesian Programming and Learning for Multi-Player Video Games ...

Finally, our approach is not exclusive to most of the techniques presented above, and it
could be interesting to combine it with Monte-Carlo planning, as Chung et al. [2005] did for
capture-the-flag tactics in Open RTS. Also, UCT* is a Monte-Carlo planning algorithm allowing
to build a sparse tree over the state tree (edges are actions, nodes are states) which had excellent
results in Go [Gelly and Wang, 2006, Gelly et al., 2012]. Balla and Fern [2009] showed that UCT
could be used in an RTS game (with multiple simultaneous actions) to generate tactical plans.
Another track (instead of UCT) would be to use metareasoning for MCTS* [Hay and Russell,
2011], which would incorporate domain knowledge and/or hyper-parameters (aggressiveness...).
6.7.3 Conclusion
We have presented a Bayesian tactical model for RTS AI, which allows both for opposing tactics
prediction and autonomous tactical decision-making. Being a probabilistic model, it deals with
uncertainty easily, and its design allows easy integration into multi-granularity (multi-scale) AI
systems as needed in RTS AI. The advantages are that 1) learning will adapt the model output
to its biased heuristic inputs 2) the updating process is the same for offline and online (in-game)
learning. Without any temporal dynamics, the position prediction is above a baseline heuristic
([32.8-40.9%] vs [20.5-25.2%]). Moreover, its exact prediction rate of the joint position and
tactical type is in [23-32.8]% (depending on the match-up), and considering the 4 most probable
regions it goes up to ≈ 70%. More importantly, it allows for tactical decision-making under
(technological) constraints and (state) uncertainty. It can be used in production thanks to its
low CPU and memory footprint.
114