Bayesian Programming and Learning for Multi-Player Video Games ...

Recommendations

Info

5.4.1 Bayesian unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.5 Results on StarCraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.5.1 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.5.2 Our bot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.6.1 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.6.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6 Tactics 91 6.1 What are tactics? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.1.1 A tactical abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.1.2 Our approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.3 Perception and tactical goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3.1 Space representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3.2 Evaluating regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.3.3 Tech tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.3.4 Attack types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.4 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.4.1 Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.4.2 Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.4.3 Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.5 A Bayesian tactical model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.5.1 Tactical Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.6 Results on StarCraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.6.1 Learning and posterior analysis . . . . . . . . . . . . . . . . . . . . . . . . 105 6.6.2 Prediction performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 6.6.3 Predictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.6.4 Error analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.6.5 In-game decision-making . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.7.1 In-game learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.7.2 Possible improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.7.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 7 Strategy 117 7.1 What is strategy? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 7.3 Perception and interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.3.1 Buildings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.3.2 Openings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.3.3 Military units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.4 Replays labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 7.4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 7.4.2 Probabilistic labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6
7.5 Build tree prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 7.5.1 Bayesian model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 7.5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 7.5.3 Possible improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 7.6 Openings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7.6.1 Bayesian model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7.6.3 Possible uses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 7.7 Army composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 7.7.1 Bayesian model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 7.7.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 7.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 8 BroodwarBotQ 157 8.1 Code architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 8.1.1 Units control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 8.1.2 Tactical goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.1.3 Map and movements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.1.4 Technology estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.1.5 Enemy units filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 8.2 A game walk-through . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 8.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 9 Conclusion 171 9.1 Contributions summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 9.2 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 9.2.1 Micro-management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 9.2.2 Tactics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 9.2.3 Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 9.2.4 Inter-game Adaptation (Meta-game) . . . . . . . . . . . . . . . . . . . . . 176 Glossary 177 Bibliography 181 A Game AI 193 A.1 Negamax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 A.2 “Gamers’ survey” in section 2.9 page 39 . . . . . . . . . . . . . . . . . . . . . . . 193 B StarCraft AI 197 B.1 Micro-management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 B.2 Tactics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 B.2.1 Decision-making: soft evidences and coherence variables . . . . . . . . . . 198 B.3 Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 B.4 BroodwarBotQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 7
Page 1: THÈSE Pour obtenir le grade de DOC
Page 4 and 5: Contents Contents 4 1 Introduction
Page 8 and 9: Notations Symbols ← assignment of
Page 10 and 11: Complexity, real-time constraints a
Page 12 and 13: intuition of Bayesian modeling to r
Page 14 and 15: e played by humans, by opposition t
Page 16 and 17: As a first approach, programmers ca
Page 18 and 19: 3 2 1 1 Figure 2.1: A Tic-tac-toe b
Page 20 and 21: Algorithm 2 Alpha-beta algorithm fu
Page 22 and 23: ewards on all the runs through node
Page 24 and 25: 2.4.1 Monopoly In Monopoly, there i
Page 26 and 27: 2.4.3 Poker Poker 4 is a zero-sum (
Page 28 and 29: 2.5.2 State of the art FPS AI consi
Page 30 and 31: 2.6.2 State of the art Methods used
Page 32 and 33: are no generic and efficient approa
Page 34 and 35: Strategy Tactics Action Strategic d
Page 36 and 37: 2.8.4 Time constant(s) For novice t
Page 38 and 39: effects. In RTS games, there is a l
Page 41 and 42: Chapter 3 Bayesian modeling of mult
Page 43 and 44: programmer-specified states, the (m
Page 45 and 46: which derives the laws of probabili
Page 47 and 48: Indeed, when evaluating two models
Page 49 and 50: • energy/mana/stamina regenerator
Page 51 and 52: • The probability that the ith un
Page 53 and 54: 3.3.4 Example This model has been a
Page 55 and 56: and P(Ai = false|T = i) = 0.6. To m
Page 57 and 58:
Chapter 4 RTS AI: StarCraft: Broodw
Page 59 and 60:
eplay is shown in appendix in Table
Page 61 and 62:
Supply/Max supply Build Note (popul
Page 63 and 64:
Figure 4.3: Military moves from a S
Page 65 and 66:
Technology Strategy Army How? Econo
Page 67:
• Robotic player (bot): chapter 8
Page 70 and 71:
• Complexity: pspace-complete [Pa
Page 72 and 73:
educing the complexity (no communic
Page 74 and 75:
(grid-based) pathfinding was recent
Page 76 and 77:
U A E Figure 5.4: In both figures,
Page 78 and 79:
Fire Reload Figure 5.6: Fight FSM o
Page 80 and 81:
• Obj i∈�1...n� ∈ {T rue,
Page 82 and 83:
Identification Parameters and proba
Page 84 and 85:
⎧⎪ Bayesian program ⎨ ⎧⎪
Page 86 and 87:
36 units setup vs OAI. For Bayesian
Page 88 and 89:
we learned, by optimizing the effic
Page 90 and 91:
Probabilistic modality Finally, we
Page 92 and 93:
• Type: prediction is problem of
Page 94 and 95:
can possibly be in the future. In t
Page 96 and 97:
6.3.2 Evaluating regions Partial ob
Page 98 and 99:
Pylon Gate Core Range Gate Core Pyl
Page 100 and 101:
events (detected by a heuristic, se
Page 102 and 103:
• AD1:n ∈ {no, low, med, high}:
Page 104 and 105:
The model is highly modular, and so
Page 106 and 107:
attacked: P(ei�=r, ti�=r, tai
Page 108 and 109:
Figure 6.7: P(A) for varying values
Page 110 and 111:
Towards a baseline heuristic The me
Page 112 and 113:
Figure 6.9 displays the mean P(A, H
Page 114 and 115:
Finally, our approach is not exclus
Page 117 and 118:
Chapter 7 Strategy Strategy without
Page 119 and 120:
etween economy, technology and mili
Page 121 and 122:
power to the player (it allows for
Page 123 and 124:
7.4.2 Probabilistic labeling Instea
Page 125 and 126:
• Variables: - X i∈�1...n�
Page 127 and 128:
Figure 7.3: Protoss vs Terran distr
Page 129 and 130:
7.5 Build tree prediction The work
Page 131 and 132:
Questions The question that we will
Page 133 and 134:
Table 7.4 shows the full results, t
Page 135 and 136:
that this average “missed” (unp
Page 137 and 138:
7.6 Openings 7.6.1 Bayesian model W
Page 139 and 140:
Questions The question that we will
Page 141 and 142:
Figure 7.12: Evolution of P(Opening
Page 143 and 144:
Table 7.5: Prediction probabilities
Page 145 and 146:
7.6.3 Possible uses We recall that
Page 147 and 148:
• U t+1 ∈ ([0, 1] . . . [0, 1])
Page 149 and 150:
(tt) if it allows for building all
Page 151 and 152:
7.7.2 Results We did not evaluate d
Page 153 and 154:
forces scores PvP PvT PvZ TvT TvZ Z
Page 155 and 156:
shows a brutal transition from the
Page 157 and 158:
Chapter 8 BroodwarBotQ Dealing with
Page 159 and 160:
8.1.2 Tactical goals The decision t
Page 161 and 162:
• X t i∈�1...n� ∈ �r1 .
Page 163 and 164:
3 2 player 1 1 6 4 resources 8 play
Page 165 and 166:
the construction plan as our Produc
Page 167 and 168:
Figure 8.5: Crops of screenshots of
Page 169:
anking). We consider that this rati
Page 172 and 173:
This is an extension of the work on
Page 174 and 175:
• differences in situations: as t
Page 176 and 177:
9.2.4 Inter-game Adaptation (Meta-g
Page 178 and 179:
fog of war hiding of some of the fe
Page 180 and 181:
RTS Real-Time Strategy games are (m
Page 182 and 183:
Sander C. J. Bakkes, Pieter H. M. S
Page 184 and 185:
Julien Diard, Pierre Bessière, and
Page 186 and 187:
Damian Isla. Handling complexity in
Page 188 and 189:
Kinshuk Mishra, Santiago Ontañón,
Page 190 and 191:
Mark Riedl, Boyang Li, Hua Ai, and
Page 192 and 193:
Adrien Treuille, Seth Cooper, and Z
Page 194 and 195:
• 0 (not at all, or irrelevant)
Page 197 and 198:
Appendix B StarCraft AI B.1 Micro-m
Page 199 and 200:
1|B, B ′ ) = 1.0 iff B = B ′ ,
Page 201 and 202:
Figure B.2: Top: StarCraft’s Lost
Page 203 and 204:
0,1 B' [0..5] T [0..2] E' 0,1 λ B
Page 205 and 206:
C C A A C C A A 4 B B 4 3 4 B B 4 3
show all

Bayesian Programming and Learning for Multi-Player Video Games ...

Create successful ePaper yourself

Delete template?

Save as template?