RTS Real-Time Strategy games are (mainly) allocentric economic <strong>and</strong> military simulations from an operational tactical/strategist comm<strong>and</strong>er viewpoint, e.g. Comm<strong>and</strong> & Conquer, Age of Empires, StarCraft, Total Annihilation. 9, 10, 14, 35, 36 rush quick aggressive opening. 121 solved game a game whose outcome can be correctly predicted from any position when each side plays optimally. 17 StarCraft: Brood War a science fiction real-time strategy (RTS) game released in 1998 by Blizzard Entertainment. 14 supply cost in population (or supply) of a given unit, or current population count of a player. Originally, the Terran name <strong>for</strong> population/psi/control.. 59, 60, 100, 165 tech tree abbreviation <strong>for</strong> “technological tree”, state of the technology (buildings, researches, upgrades) which are unlocked/available to a given player.. 57, 59, 61, 66, 96–98, 111, 117–121, 146, 159, 165, 169, 172, 203 UCT Upper Confidence Bounds <strong>for</strong> Trees. 22, 114 zero-sum game a game in which the total score of each players, from one player’s point-ofview, <strong>for</strong> every possible strategies, adds up to zero; i.e. “a player benefits only at the expense of others”. 17 180
Bibliography David W. Aha, Matthew Molineaux, <strong>and</strong> Marc J. V. Ponsen. <strong>Learning</strong> to win: Case-based plan selection in a real-time strategy game. In Héctor Muñoz-Avila <strong>and</strong> Francesco Ricci, editors, ICCBR, volume 3620 of Lecture Notes in Computer Science, pages 5–20. Springer, 2005. ISBN 3-540-28174-6. 5 citations pages 63, 72, 94, 119, <strong>and</strong> 120 Srinivas M. Aji <strong>and</strong> Robert J. McEliece. The generalized distributive law. IEEE Transactions on In<strong>for</strong>mation Theory, 46(2):325–343, 2000. cited page 45 David W. Albrecht, Ingrid Zukerman, <strong>and</strong> Ann E. Nicholson. <strong>Bayesian</strong> models <strong>for</strong> keyhole plan recognition in an adventure game. User Modeling <strong>and</strong> User-Adapted Interaction, 8:5–47, January 1998. cited page 119 Victor L. Allis. Searching <strong>for</strong> Solutions in <strong>Games</strong> <strong>and</strong> Artificial Intelligence. PhD thesis, University of Limburg, 1994. URL http://fragrieu.free.fr/SearchingForSolutions.pdf. 2 citations pages 19 <strong>and</strong> 21 Christophe Andrieu, N<strong>and</strong>o De Freitas, Arnaud Doucet, <strong>and</strong> Michael I. Jordan. An introduction to mcmc <strong>for</strong> machine learning. Machine <strong>Learning</strong>, 50:5–43, 2003. cited page 46 M. Sanjeev Arulampalam, Simon Maskell, <strong>and</strong> Neil Gordon. A tutorial on particle filters <strong>for</strong> online nonlinear/non-gaussian bayesian tracking. IEEE Transactions on Signal Processing, 50:174–188, 2002. cited page 42 Robert B. Ash <strong>and</strong> Richard L. Bishop. Monopoly as a Markov process. Mathematics Magazine, (45):26–29, 1972. cited page 24 John Asmuth, Lihong Li, Michael Littman, Ali Nouri, <strong>and</strong> David Wingate. A bayesian sampling approach to exploration in rein<strong>for</strong>cement learning. In Uncertainty in Artificial Intelligence, UAI, pages 19–26. AUAI Press, 2009. 2 citations pages 82 <strong>and</strong> 87 Phillipa Avery, Sushil Louis, <strong>and</strong> Benjamin Avery. Evolving Coordinated Spatial Tactics <strong>for</strong> Autonomous Entities using Influence Maps. In Proceedings of the 5th international conference on Computational Intelligence <strong>and</strong> <strong>Games</strong>, CIG’09, pages 341–348, Piscataway, NJ, USA, 2009. IEEE Press. ISBN 978-1-4244-4814-2. URL http://dl.acm.org/citation.cfm?id= 1719293.1719350. 3 citations pages 73, 94, <strong>and</strong> 174 S<strong>and</strong>er Bakkes, Pieter Spronck, <strong>and</strong> Eric Postma. TEAM: The Team-Oriented Evolutionary Adaptability Mechanism. pages 273–282. 2004. URL http://www.springerlink.com/ content/hdru7u9pa7q3kg9b. 2 citations pages 16 <strong>and</strong> 72 181
- Page 1:
THÈSE Pour obtenir le grade de DOC
- Page 4 and 5:
Contents Contents 4 1 Introduction
- Page 6 and 7:
5.4.1 Bayesian unit . . . . . . . .
- Page 8 and 9:
Notations Symbols ← assignment of
- Page 10 and 11:
Complexity, real-time constraints a
- Page 12 and 13:
intuition of Bayesian modeling to r
- Page 14 and 15:
e played by humans, by opposition t
- Page 16 and 17:
As a first approach, programmers ca
- Page 18 and 19:
3 2 1 1 Figure 2.1: A Tic-tac-toe b
- Page 20 and 21:
Algorithm 2 Alpha-beta algorithm fu
- Page 22 and 23:
ewards on all the runs through node
- Page 24 and 25:
2.4.1 Monopoly In Monopoly, there i
- Page 26 and 27:
2.4.3 Poker Poker 4 is a zero-sum (
- Page 28 and 29:
2.5.2 State of the art FPS AI consi
- Page 30 and 31:
2.6.2 State of the art Methods used
- Page 32 and 33:
are no generic and efficient approa
- Page 34 and 35:
Strategy Tactics Action Strategic d
- Page 36 and 37:
2.8.4 Time constant(s) For novice t
- Page 38 and 39:
effects. In RTS games, there is a l
- Page 41 and 42:
Chapter 3 Bayesian modeling of mult
- Page 43 and 44:
programmer-specified states, the (m
- Page 45 and 46:
which derives the laws of probabili
- Page 47 and 48:
Indeed, when evaluating two models
- Page 49 and 50:
• energy/mana/stamina regenerator
- Page 51 and 52:
• The probability that the ith un
- Page 53 and 54:
3.3.4 Example This model has been a
- Page 55 and 56:
and P(Ai = false|T = i) = 0.6. To m
- Page 57 and 58:
Chapter 4 RTS AI: StarCraft: Broodw
- Page 59 and 60:
eplay is shown in appendix in Table
- Page 61 and 62:
Supply/Max supply Build Note (popul
- Page 63 and 64:
Figure 4.3: Military moves from a S
- Page 65 and 66:
Technology Strategy Army How? Econo
- Page 67:
• Robotic player (bot): chapter 8
- Page 70 and 71:
• Complexity: pspace-complete [Pa
- Page 72 and 73:
educing the complexity (no communic
- Page 74 and 75:
(grid-based) pathfinding was recent
- Page 76 and 77:
U A E Figure 5.4: In both figures,
- Page 78 and 79:
Fire Reload Figure 5.6: Fight FSM o
- Page 80 and 81:
• Obj i∈�1...n� ∈ {T rue,
- Page 82 and 83:
Identification Parameters and proba
- Page 84 and 85:
⎧⎪ Bayesian program ⎨ ⎧⎪
- Page 86 and 87:
36 units setup vs OAI. For Bayesian
- Page 88 and 89:
we learned, by optimizing the effic
- Page 90 and 91:
Probabilistic modality Finally, we
- Page 92 and 93:
• Type: prediction is problem of
- Page 94 and 95:
can possibly be in the future. In t
- Page 96 and 97:
6.3.2 Evaluating regions Partial ob
- Page 98 and 99:
Pylon Gate Core Range Gate Core Pyl
- Page 100 and 101:
events (detected by a heuristic, se
- Page 102 and 103:
• AD1:n ∈ {no, low, med, high}:
- Page 104 and 105:
The model is highly modular, and so
- Page 106 and 107:
attacked: P(ei�=r, ti�=r, tai
- Page 108 and 109:
Figure 6.7: P(A) for varying values
- Page 110 and 111:
Towards a baseline heuristic The me
- Page 112 and 113:
Figure 6.9 displays the mean P(A, H
- Page 114 and 115:
Finally, our approach is not exclus
- Page 117 and 118:
Chapter 7 Strategy Strategy without
- Page 119 and 120:
etween economy, technology and mili
- Page 121 and 122:
power to the player (it allows for
- Page 123 and 124:
7.4.2 Probabilistic labeling Instea
- Page 125 and 126:
• Variables: - X i∈�1...n�
- Page 127 and 128:
Figure 7.3: Protoss vs Terran distr
- Page 129 and 130: 7.5 Build tree prediction The work
- Page 131 and 132: Questions The question that we will
- Page 133 and 134: Table 7.4 shows the full results, t
- Page 135 and 136: that this average “missed” (unp
- Page 137 and 138: 7.6 Openings 7.6.1 Bayesian model W
- Page 139 and 140: Questions The question that we will
- Page 141 and 142: Figure 7.12: Evolution of P(Opening
- Page 143 and 144: Table 7.5: Prediction probabilities
- Page 145 and 146: 7.6.3 Possible uses We recall that
- Page 147 and 148: • U t+1 ∈ ([0, 1] . . . [0, 1])
- Page 149 and 150: (tt) if it allows for building all
- Page 151 and 152: 7.7.2 Results We did not evaluate d
- Page 153 and 154: forces scores PvP PvT PvZ TvT TvZ Z
- Page 155 and 156: shows a brutal transition from the
- Page 157 and 158: Chapter 8 BroodwarBotQ Dealing with
- Page 159 and 160: 8.1.2 Tactical goals The decision t
- Page 161 and 162: • X t i∈�1...n� ∈ �r1 .
- Page 163 and 164: 3 2 player 1 1 6 4 resources 8 play
- Page 165 and 166: the construction plan as our Produc
- Page 167 and 168: Figure 8.5: Crops of screenshots of
- Page 169: anking). We consider that this rati
- Page 172 and 173: This is an extension of the work on
- Page 174 and 175: • differences in situations: as t
- Page 176 and 177: 9.2.4 Inter-game Adaptation (Meta-g
- Page 178 and 179: fog of war hiding of some of the fe
- Page 182 and 183: Sander C. J. Bakkes, Pieter H. M. S
- Page 184 and 185: Julien Diard, Pierre Bessière, and
- Page 186 and 187: Damian Isla. Handling complexity in
- Page 188 and 189: Kinshuk Mishra, Santiago Ontañón,
- Page 190 and 191: Mark Riedl, Boyang Li, Hua Ai, and
- Page 192 and 193: Adrien Treuille, Seth Cooper, and Z
- Page 194 and 195: • 0 (not at all, or irrelevant)
- Page 197 and 198: Appendix B StarCraft AI B.1 Micro-m
- Page 199 and 200: 1|B, B ′ ) = 1.0 iff B = B ′ ,
- Page 201 and 202: Figure B.2: Top: StarCraft’s Lost
- Page 203 and 204: 0,1 B' [0..5] T [0..2] E' 0,1 λ B
- Page 205 and 206: C C A A C C A A 4 B B 4 3 4 B B 4 3