Bayesian Programming and Learning for Multi-Player Video Games ...

Indeed, when evaluating two models π1 and π2, doing the ratio of evidences for the two
modeling assumption helps us choose:
P(π1|δ) P(π1)P(δ|π1)
=
P(π2|δ) P(π2|δ)P(δ|π2)
If our model is complicated, its expressiveness is higher, and so it can model more complex
causes of/explanations for the dataset. However, the probability mass will be spread thiner on
the space of possible datasets than for a simpler model. For instance, in the task of regression,
if π1 can only encode linear regression (y = Ax + b) and π2 can also encore quadratic regression
(y = Ax + Bx 2 + c), all the predictions for π1 are concentrated on linear (affine) functions, while
for π2 the probability mass of predictions is spread on more possible functions. If we only have
data δ of linear functions, the evidence for π1, P(π1|δ) will be higher. If δ contains some quadratic
functions, the evidence ratio will shift towards π2 as the second model has better predictions:
the shift to π2 happens when the additional model complexity of π2 and the prediction errors of
π1 balance each other.
3.3 Modeling of a Bayesian MMORPG player
3.3.1 Task
We will now present a model of a MMORPG* player with the Bayesian programming framework
[Synnaeve and Bessière, 2010]. A role-playing game (RPG) consist in the incarnation by the
human player of an avatar with specific skills, items, numbers of health points (HP) and stamina
or mana (magic energy) points (we already presented this gameplay in section 2.6). We want to
program a robotic player which we can substitute to a human player in a battle scenario. More
specifically here, we modeled the “druid” class, which is complex because it can cast spells to
deal damages or other negative effects as well as to heal allies or enhance their capacities (“buff”
them). The model described here deals only with a sub-task of a global AI for autonomous
NPC.
The problem that we try to solve is: how do we choose which skill to use, and on which target,
in a PvE* battle? The possible targets are all our allies and foes. There are also several skills
that we can use. We will elaborate a model taking all our perceptions into account and
giving back a distribution over possible target and skills. We can then pick the most
probable combination that is yet possible to achieve (enough energy/mana, no cooldown/reload
time) or simply sample in this distribution.
An example of a task that we have to solve is depicted in Figure 3.2, which shows two
different setups (states) of the same scenario: red characters are foes, green ones are allies, we
control the blue one. The “Lich” is inflicting damages to the “MT” (main tank), the “Add” is
hitting the “Tank”. In setup A, only the “Tank” is near death, while in setup B both the “Tank”
and the “Add” are almost dead. A first approach to reason about these kind of situations would
be to use the combination of simple rules and try to give them some priority.
• if an ally has less than X (threshold) HP then use a “heal” on him.
(3.4)
• if the enemy receiving most damage has resistances (possibly in a list), lower them if
possible.
47