Bayesian Programming and Learning for Multi-Player Video Games ...

7.5 Build tree prediction
The work described in the next two sections can be classified as probabilistic plan recognition.
Strictly speaking, we present model-based machine learning used for prediction of plans, while
our model is not limited to prediction. We performed two levels of plan recognition, both are
learned from the replays: tech tree prediction (unsupervised, it does not need openings labeling,
this section) and opening prediction (semi-supervised or supervised depending on the labeling
method, next section).
7.5.1 Bayesian model
Variables
• BuildTree: BT ∈ {∅, {building1}, {building2}, {building1 ∧ building2}, . . . }: all the possible
building trees for the given race. For instance {pylon, gate} and {pylon, gate, core}
are two different BuildT rees.
• Observations: O i∈�1...N� ∈ {0, 1}, Ok is 1/true if we have seen (observed) the kth building
(it can have been destroyed, it will stay “seen”). Otherwise, it is 0/false.
• λ ∈ {0, 1}: coherence variable (restraining BuildT ree to possible values with regard to
O1:N)
• Time: T ∈ �1 . . . P �, time in the game (1 second resolution).
At first, we generated all the possible (according to the game rules) build trees* (BT values)
of StarCraft, and there are between ≈ 500 and 1600 depending on the race without even
counting buildings duplicates! We observed that a lot of possible build trees are too absurd to
be performed in a competitive match and were never seen during the learning. So, we restricted
BT to have its value in all the build trees encountered in our replays dataset and we added
multiple instances of the basic unit producing buildings (gateway, barracks), expansions and
supply buildings (depot, pylon, “overlord” as a building), as shown in Table 7.2. This way, there
are 810 build trees for Terran, 346 for Protoss and 261 for Zerg (learned from ≈ 3000 games for
each race, see Table 7.1). In a new game, if we encounter a build tree that we never saw, we
are in a unknown state. Anyway we would not have seen enough data (any at all) during the
learning to conclude anything. We could look at the proximity of the build tree to other known
build trees, see section 7.5.2 and the Discussion (7.5.3).
Decomposition
The joint distribution of our model is the following:
P(T, BT, O1 . . . ON, λ) = P(T |BT )P(BT )P(λ|BT, O1:N)
This can also be see as a plate diagram in Figure 7.7.
129
N�
P(Oi) (7.1)
i=1