Bayesian Programming and Learning for Multi-Player Video Games ...

Algorithm 4 UCB1 ExpandFrom
function ExpandFrom(node)
if node is terminal then
return node ⊲ terminal
end if
if ∃c ∈ node.children s.t. N(c) = 0 then
return c ⊲ grow
end if
return ExpandFrom(arg maxc∈node.children) end function
+r
+r
+r
�
Q(c) ln N(node)
N(c) + k
N(c) ) ⊲ select
Figure 2.4: An iteration of the while loop in MCTS, from left to right: ExpandFrom select,
ExpandFrom grow, Evaluate & Merge.
2.4 Games with uncertainty
An exhaustive list of games, or even of games genres, is beyond the scope/range of this thesis.
All uncertainty boils down to incompleteness of information, being it the physics of the dice
being thrown or the inability to measure what is happening in the opponent’s brain. However,
we will speak of 2 types (sources) of uncertainty: extensional uncertainty, which is due to
incompleteness in direct, measurable information, and intentional uncertainty, which is related
to randomness in the game or in (the opponent’s) decisions. Here are two extreme illustrations
of this: an agent acting without sensing is under full extensional uncertainty, while an agent
whose acts are the results of a perfect random generator is under full intentional uncertainty.
The uncertainty coming from the opponent’s mind/cognition lies in between, depending on the
simplicity to model the game as an optimization procedure. The harder the game is to model,
the harder it is to model the trains of thoughts our opponents can follow.
23