Bayesian Programming and Learning for Multi-Player Video Games ...

Aha et al. [2005] used CBR* to perform dynamic plan retrieval extracted from domain
knowledge in Wargus (Warcraft II open source clone). Ontañón et al. [2008] base their real-time
case-based planning (CBP) system on a plan dependency graph which is learned from human
demonstration in Wargus. In [Ontañón et al., 2007b, Mishra et al., 2008b], they use CBR and
expert demonstrations on Wargus. They improve the speed of CPB by using a decision tree to
select relevant features. Hsieh and Sun [2008] based their work on [Aha et al., 2005] and used
StarCraft replays to construct states and building sequences. Strategies are choices of building
construction order in their model.
Schadd et al. [2007] describe opponent modeling through hierarchically structured models of
the opponent’s behavior and they applied their work to the Spring RTS game (Total Annihilation
open source clone). Hoang et al. [2005] use hierarchical task networks (HTN*) to model strategies
in a first person shooter with the goal to use HTN planners. Kabanza et al. [2010] improve the
probabilistic hostile agent task tracker (PHATT [Geib and Goldman, 2009], a simulated HMM*
for plan recognition) by encoding strategies as HTN.
Weber and Mateas [2009] presented “a data mining approach to strategy prediction” and
performed supervised learning (from buildings features) on labeled StarCraft replays. In this
chapter, for openings prediction, we worked with the same dataset as they did, using their
openings labels and comparing it to our own labeling method.
Dereszynski et al. [2011] presented their work at the same conference that we presented [Synnaeve
and Bessière, 2011]. They used an HMM which states are extracted from (unsupervised)
maximum likelihood on the dataset. The HMM parameters are learned from unit counts (both
buildings and military units) every 30 seconds and Viterbi inference is used to predict the most
likely next states from partial observations.
Jónsson [2012] augmented the C4.5 decision tree [Quinlan, 1993] and nearest neighbour with
generalized examplars [Martin, 1995], also used by Weber and Mateas [2009], with a Bayesian
network on the buildings (build tree*). Their results confirm ours: the predictive power is strictly
better and the resistance to noise far greater than without encoding probabilistic estimations of
the build tree*.
7.3 Perception and interaction
7.3.1 Buildings
From last chapter (section 6.3.3), we recall that the tech tree* is a directed acyclic graph which
contains the whole technological (buildings and upgrades) development of a player. Also, each
unit and building has a sight range that provides the player with a view of the map. Parts of
the map not in the sight range of the player’s units are under fog of war and the player ignores
what is and happens there. We also recall the notion of build order*: the timings at which the
first buildings are constructed.
In this chapter, we will use build trees* as a proxy for estimating some of the strategy. A
build tree is a little different than the tech tree*: it is the tech tree without upgrades and
researches (only buildings) and augmented of some duplications of buildings. For instance,
{P ylon∧Gateway} and {P ylon∧Gateway, Gateway2} gives the same tech tree but we consider
that they are two different build trees. Indeed, the second Gateway gives some units producing
120