Bayesian Programming and Learning for Multi-Player Video Games ...

This is an extension of the work on inverse programming for Unreal Tournament avatars
[Le Hy et al., 2004]. Combined, these contributions lead to real-time micro-management
behaviors for StarCraft, which achieved very satisfying results (ex-aequo with the best
micro-management AI of AIIDE 2010 competition), and were published in [Synnaeve and
Bessière, 2011a].
• By going up in the ladder of abstraction (strategy & tactics), we were able to exploit
what we previously called vertical continuity (see section 2.8.1) through hierarchical models.
About strategy (chapter 7), from low-level observations, we produced build trees*
(section 7.5, [Synnaeve and Bessière, 2011]), and built upon that to infer openings* (section
7.6, [Synnaeve and Bessière, 2011b]) and to constrain the inference on the opponent’s
army composition (section 7.7). Tactics (chapter 6) also make good use of the prediction
on the opponent’s tech tree*. When used in decision-making, our models are even more
constrained, because we have full knowledge of our state instead of distributions.
• We also took advantage of actions sequencing, previously called horizontal continuity (see
section 2.8.1), by assessing that, often, things which are done should not be undone (at least
not immediately) and that some strategic and tactical steps are prerequisite of intended
steps. At the strategic level, the distribution on build trees* is time-dependent: the
sequencing is encoded in the learned discrete Gaussian distributions (section 7.5, [Synnaeve
and Bessière, 2011]). The openings are filtered on previous inferences with a first order
Markovian assumption (section 7.6, [Synnaeve and Bessière, 2011b]), i.e. the value at
time t is dependent on the value at time t − 1. The army composition model (section 7.7)
makes use of temporal continuity to adapt the player’s army to the opponent’s future army
composition. Micro-management assumes an uncertain linear interpolation of future units
position resulting of trajectory continuity (instead of considering all possible positions), as
explained in subsection 5.3.1.
We used machine learning to help specifying our models, both used for prediction (opponent
modeling) and for decision-making. We exploited different datasets for mainly two separate
objectives:
• We produced relevant abstract models thanks to learning. For the labeling replays (allowing
for supervised learning of our full openings* prediction model), we used semi-supervised
(by selecting features and a scoring function) GMM* clustering of replays, as presented
in section 7.4 [Synnaeve and Bessière, 2011b]. In order to reason qualitatively and quantitatively
about armies composition with a tractable model, we applied GMM to armies
unit types percentages to find the different composing components (section 7.7). For tactics,
we used heuristics (see subsection 6.3.2) for the evaluation of the regions, whose bias
(incompleteness) the model was adapted to by learning.
• We learned the parameters of our models from human-played games datasets. At the
strategic level, we learned the time-dependent distributions on the build trees and the
co-occurrence of openings with build trees in sections 7.5 and 7.6, respectively published
as [Synnaeve and Bessière, 2011], [Synnaeve and Bessière, 2011b]. We were also able to
study the strengths and weaknesses of openings this way (subsection 7.6.2, and we looked
172