An Introduction to Evolutionary Game Theory: Lecture 2 - School of ...

An Introduction to Evolutionary Game Theory:Lecture 2Mauro MobiliaLectures delivered at the Graduate School on Nonlinear and Stochastic Systems in Biologyheld in the Department of Applied Mathematics, School of MathematicsUniversity of Leeds, U.K.31/03/2009 - 01/04/2009Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

OutlineThe goal of this lecture is to give some insight into the followingtopics:Some Properties of the Replicator DynamicsReplicator Equations for 2 × 2 GamesMoran Process & Evolutionary DynamicsThe Concept of Fixation ProbabilityEvolutionary Game Theory in Finite PopulationInfluence of Fluctuations on Evolutionary DynamicsMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Replicator DynamicsPopulation of Q different species: e 1 ,...,e Q , with frequencies x 1 ,...,x QState of the system described by x = (x 1 ,...,x Q ) ∈ S Q , whereS Q = {x;x i ≥ 0,∑ Q i=1 x i = 1}To set up the dynamics, we need a functional expression for thefitness f i (x)Between various possibilities, a very popular choice is:ẋ i = x i(fi (x) −¯f (x) ) ,where, one (out of many) possible choices, for the fitness is theexpected payoff: f i (x) = ∑ Q i=1 A ijx jand ¯f (x) is the average fitness: ¯f (x) = ∑ Q ix i f i (x)This choice corresponds to the so-called replicator dynamics onwhich most of evolutionary game theory is centeredMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Some Properties of the Replicator Dynamics (I)Replicator equations (REs):ẋ i = x i [(Ax) i − x.Ax]Set of coupled cubic equations (when x.Ax ≠ 0)Let x ∗ = (x1 ∗,...,x Q ∗ ) be a fixed point (steady state) of the REsx ∗ can be (Lyapunov-) stable, unstable, attractive (i.e. there isbasin of attraction), asymptotically stable=attractor ( =stable +attractive), globally stable (basin of attraction is S Q )Only possible interior fixed point satisfies (there is either 1 or 0):(Ax ∗ ) 1 = (Ax ∗ ) 2 = ... = (Ax ∗ ) Q = x ∗ .Ax ∗x 1 + ... + x Q = 1Same dynamics if one adds a constant[c j to the payoff matrixA = (A ij ): ẋ i = x i [(Ax) i − x.Ax] = x i (Ãx) i −], x.Ãx whereà = (A ij + c j )Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Some Properties of the Replicator Dynamics (II)Dynamic versus evolutionary stability: connection between dynamicstability (of REs) and NE/evolutionary stability?Notions do not perfectly overlap ⇒ Folks Theorem of EGT:Let x ∗ = (x1 ∗,...,x Q ∗ ) be a fixed point (steady state) of the REsNEs are rest points (of the REs)Strict NEs are attractorsA stable rest point (of the REs) is an NEInterior orbit converges to x ∗ ⇒ x ∗ is an NEESSs are attractors (asymptotically stable)Interior ESSs are global attractorsConverse statements generally do not hold!For 2 × 2 matrix games x ∗ is an ESS iff it is an attractorREs with Q strategies can be mapped ( onto Lotka-Volterra )equations for Q − 1 species: ẏ i = y i r i + ∑ Q−1j=1 b ijy jReplicator dynamics is non-innovative: cannot generate newstrategiesMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Replicator Dynamics for 2 × 2 Games (I)2 strategies: say A and BN players: N A are A-players and N B are B-players, N A + N B = NGeneral payoff matrix:vs A BA 1 + p 11 1 + p 12B 1 + p 21 1 + p 22where selection → p ij and the neutral component → 1Frequency of A and B strategists is resp.x = N A /N and y = N B /N = 1 − xFitness (expected payoff) of A and B strategists is resp.f A (x) = p 11 x + p 12 (1 − x) + 1 and f B (x) = p 21 x + p 22 (1 − x) + 1Average fitness: ¯f (x) = xf A (x) + (1 − x)f B (x)Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Replicator Dynamics for 2 × 2 Games (II)Replicator dynamics:dxdt= x[f A (x) −¯f (x)] = x(1 − x)[f A (x) − f B (x)]= x(1 − x)[x(p 11 − p 21 ) + (1 − x)(p 12 − p 22 )]xy = x(1 − x): interpreted as the probability that A and B interactf A (x) − f B (x) = x(p 11 − p 12 ) + (1 − x)(p 12 − p 22 ): says thatreproduction (“success”) depends on the difference of fitnessEquivalent payoff matrix (A i1 → A i1 − p 11 , A i2 → A i2 − p 22 ), withµ A = p 21 − p 11 and µ B = p 12 − p 22 :vs A BA 1 1 + µ AB 1 + µ B 1dxdt = x(1 − x)[−xµ A + (1 − x)µ B ] = x(1 − x)[µ B − (µ A + µ B )x]⇒ For 2 × 2 games, the dynamics is simple: no limit cycles, nooscillations, no chaotic behaviourMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Replicator Dynamics for 2 × 2 Games (III)dxdt = x(1 − x)[µ B − (µ A + µ B )x]1µ A > 0 and µ B > 0: Hawk-Dove gamex ∗ =µ Bµ A +µis stable (attractor, ESS) interior FPB2µ A > 0 and µ B < 0: Prisoner’s DilemmaB always better off, x ∗ = 0 is ESS3µ A < 0 and µ B < 0: Stag-Hunt GameEither A or B can be better off, i.e. x ∗ = 0 and x ∗ = 1 are ESS.x ∗ =µ Bµ A +µis unstable FP (non-ESS)B4µ A < 0 and µ B > 0: Pure Dominance ClassA always better off, x ∗ = 1 is ESSMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Some Remarks on Replicator DynamicsFor x small: ẋ = µ B xdxdt = x(1 − x)[µ B − (µ A + µ B )x]For x ≈ 1: ẏ = (d/dt)(1 − x) = µ A (1 − x)Thus, the stability of x ∗ = 0 and x ∗ = 1 simply depends on the sign ofµ B and µ A , respectivelyAnother popular dynamics is the so-called “adjusted replicatordynamics”, for which the equations read:dxdt= x f [ ]A(x) −¯f (x)fA (x) − f= x(1 − x)B (x)¯f (x) ¯f (x)These equations equations share the same fixed points with the REs.In general, replicator dynamics and adjusted replicator dynamics giverise to different behaviours. However, for 2 × 2 games: samequalitative behaviourMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Stochastic Dynamics & Moran ProcessEvolutionary dynamics involves a finite number of discrete individuals⇒ “Microscopic” stochastic rules given by the Moran processMoran Process is a Markov birth-death process in 4 steps:2 species, i individuals of species A and N − i of species B1An individual A could be chosen for birth and death withprobability (i/N) 2 . The number of A remains the same2An individual B could be chosen for birth and death withprobability ((N − i)/N) 2 . The number of B remains the same3An individual A could be chosen for reproduction and a Bindividual for death with probability i(N − i)/N 2 . For this event:i → i + 1 and N − i → N − 1 − i4An individual B could be chosen for reproduction and a Aindividual for death with probability i(N − i)/N 2 . For this event:i → i − 1 and N − i → N + 1 − iMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Stochastic Dynamics & Moran ProcessEvolutionary dynamics given by the Moran process: Markovbirth-death process in 4 stepsThere are two absorbing states in the Moran process: all-B and all-AWhat is the probability F i of ending in a state with all A (i = N) startingfrom i individuals A? For i = 1, F 1 is the “fixation” probability of ATransition from i → i + 1 given by rate α iTransition i → i − 1 given by rate β iF i = β i F i−1 + (1 − α i − β i )F i + α i F i+1 , for i = 1,...,N − 1F 0 = 0 and F N = 1Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Moran Process & Fixation ProbabilityWhat is the fixation probability F 1 of A individuals?F i = β i F i−1 + (1 − α i − β i )F i + α i F i+1 , for i = 1,...,N − 1F 0 = 0 and F N = 1Introducing g i = F i − F i−1 (i = 1,...,N − 1), one notes that ∑ N i=1 g i = 1and g i+1 = γ i g i , where γ i = β i /α i ⇒ one recovers a classic results onMarkov chains: F i = 1+∑i−1 j=1 ∏j k=1 γ k1+∑ N−1j=1 ∏j k=1 γ k1⇒ Fixation probability of species A is F A = F 1 =1+∑ N−1j=1 ∏j k=1 γ kAs i = 0 and i = N are absorbing states ⇒always absorption (all-A or all-B) ⇒ Fixation probability of species Bis F B = 1 − F N−1 =∏ N−1k=1 γ k1+∑ N−1j=1 ∏j k=1 γ kMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Fixation in the Neutral & Constant Fitness CasesFixation Probabilities:1F A = and F1+∑ N−1B = F A ∏ N−1j=1 ∏j k=1 γ k=1 γ k, with γ i = β i /α ikWhen α i = β i = γ i = 1, this is the neutral case where there is noselection but only random drift:F A =F B =1/NThis means that the chance that an individual will generate alineage which will inheritate the entire population is 1/NCase where A and B have constant but different fitnesses, f A = rfor A and f B = 1 for B, α i =ri(N−i)N(N+(r−1)i) and β i =i(N−i)N(N+(r−1)i)1−r −1Thus, F A = and F1−r −N B = 1−r1−r NIf r > 1, F A > N −1 for N ≫ 1: selection favours the fixation of AIf r < 1, F B > N −1 for N ≫ 1: selection favours the fixation of BMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Evolutionary Games in Finite Populations (I)Finite population of 2 species: i individuals of species A and N − iindividuals of species B interact according to the payoff matrix:vs A BA a bB c dProbability to draw a A and B is i/N and (N − i)/N, respectively ⇒Probability that a given individual A interacts with another A is(i − 1)/(N − 1)Probability that a given individual A interacts with a B is(N − i)/(N − 1)Probability that a given individual B interacts with another B is(N − i − 1)/(N − 1)Probability that a given individual B interacts with a A is i/(N − 1)The states i = 0 (All-A) and i = N (All-B) are absorbingExpected payoff for A and B, respectively:Ei A a(i − 1) + b(N − i)=N − 1andEi B ci + d(N − i − 1)=N − 1Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Evolutionary Games in Finite Populations (II)() −1F A = 1 + ∑ N−1j=1 ∏j k=1 γ k and FB = F A ∏ N−1k=1 γ k, with γ i = β i /α iE Ai= a(i−1)+b(N−i)N−1and EiB = ci+d(N−i−1)N−1Expected payoffs E A,Biare usually interpreted as fitness.Recent idea (Nowak et al.): Introduce a parameter w accounting forbackground random drift contribution to fitness f Aif Ai= 1 − w + wEi A and fiB = 1 − w + wEiBfor A and f Bifor BAverage fitness: ¯f = (i/N)fiA + (1 − (i/N))fiBParameter w measures the intensity of selection: w = 0 ⇒ noselection (only random drift), w = 1 ⇒ only selection, w ≪ 1 ⇒ “weakselection”Consider a Moran process with frequency-dependent hopping rates:α i = f iA ( )( )i N − iand β i = f B ( )( )i i N − i⇒ γ i = f iB¯f N N¯f N NfiA()Thus, F A = 1/ 1 + ∑ N−1j=1 ∏j k=1 (f k B/f k A) and F B = F A ∏ N−1k=1 (f k B/f k A)Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Influence of Fluctuations on Evolutionary Dynamics (I)(F A = 1 + ∑ N−1j=1 ∏j i=1f AiFixation ) Probabilities:−1and F B = F A ∏ N−1i=1f Bif Ai= 1 − w + w a(i−1)+b(N−i)N−1and f Bi(fBi/fiA ), with= 1 − w + w ci+d(N−i−1)N−1Does selection favour fixation of A? Yes, only if F A > 1/NIn the weak selection limit (w → 0):F A ≈N1 [1 −w6({a + 2b − c − 2d}N − {2a + b + c − 4d}) ] −1Thus, F A > 1/N if a(N − 2) + b(2N − 1) > c(N + 1) + 2d(N − 2)N = 2 b > cN = 3 a + 5b > 2(2c + d)N = 4 2a + 7b > 5c + 4d... ...N ≫ 1 a + 2b > c + 2dFor large N, F A > 1/N if a + 2b > c + 2dMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Influence of Fluctuations & Finite-Size Effects (II)In the weak selection limit (w → 0):F A ≈N1 [1 −w6({a + 2b − c − 2d}N − {2a + b + c − 4d}) ] −1For large N, F A > 1/N if a + 2b > c + 2dConsequences of finite-size fluctuations?Reconsider a 2 × 2 game with a > c and b < d (“Stag-Hunt game”):Rational game: all-A and all-B are strict-NE and ESSReplicator Dynamics: all-A & all-B attractors and x ∗ =an unstable interior rest point (NE, but not ESS)d−ba−c+d−b isIn finite (yet large) population (stochastic Moran process, weakselection): The condition a − c > 2(d − b) to favour fixation of Aleads to x ∗ < 1/3- If the unstable rest point x ∗ occurs at frequency < 1/3, in a largeyet finite population and for w ≪ 1, selection favours the fixation of A- Probability that a single A takes over the entire population of N − 1individuals B is greater than 1/NMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Influence of Fluctuations & Finite-Size Effects (II)In the weak selection limit (w → 0):F A ≈N1 [1 −w6({a + 2b − c − 2d}N − {2a + b + c − 4d}) ] −1For large N, F A > 1/N if a + 2b > c + 2dConsequences of finite-size fluctuations?Reconsider a 2 × 2 game with a > c and b < d (“Stag-Hunt game”):Rational game: all-A and all-B are strict-NE and ESSReplicator Dynamics: all-A & all-B attractors and x ∗ =an unstable interior rest point (NE, but not ESS)In finite (yet large) population (stochastic Moran process, weakselection): The condition a − c > 2(d − b) to favour fixation of Aleads to x ∗ < 1/3d−ba−c+d−b isMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Influence of Fluctuations & Finite-Size Effects (III)In the weak selection limit (w → 0, 2 species systems):F A ≈N1 [1 −w6({a + 2b − c − 2d}N − {2a + b + c − 4d}) ] −1Previous result hints that the concept of evolutionary stability shouldbe modified to account for finite-size fluctuations ⇒ leads to theconcept of ESS N : A finite population of B is evolutionary stable isevolutionary stable against a second species A if1The fitness of B is greater than that of A, i.e. fiB > fi A , ∀i. Thismeans: “selection opposes A invading B”2F A < 1/N, implying that selection opposes A replacing BThis leads to the criteria for evolutionary stability of B:Determinsitic (N = ∞)Stochastic (N finite)(1) d > b (d − b)N > 2d − (b + c)(2) if b = d, then c > a c(N + 1) + 2d(N − 2) > a(N − 2) + b(2N − 1)Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

Influence of Fluctuations & Finite-Size Effects (III)In the weak selection limit (w → 0, 2 × 2 games):F A ≈N1 [1 −w6({a + 2b − c − 2d}N − {2a + b + c − 4d}) ] −1Criteria for evolutionary stability of B in a population of size N:Determinsitic (N = ∞)Stochastic (N finite)(1) d > b (d − b)N > 2d − (b + c)(2) if b = d, then c > a c(N + 1) + 2d(N − 2) > a(N − 2) + b(2N − 1)Conditions for evolutionary stability depend on the population size:B is ESS N if N = 2 N ≫ 1 (finite)Condition (1): c > b d > bCondition (2): c > b x ∗ =a−c+d−b d−b For small N, the traditional ESS conditions are neither necessarynor sufficient to guarantee evolutionary stabilityFor large N, the traditional ESS conditions are necessary but notsufficient to guarantee evolutionary stabilityMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

OutlookIn this set of lectures dedicated to an introduction to evolutionarygame theory, we have discussedSome concepts of classic (rational) game theory which wereillustrated by a series of examples (hawk-doves, prisoner’sdilemma and stag-hunt games)Notion of evolutionary dynamics via the concept of fitnessReplicator dynamics and discussed its properties: connectionbetween dynamic stability, NEs and ESSReplicator dynamics for 2 × 2 games: classificationStochastic evolutionary dynamics according to the MoranprocessFixation probability as a Markov chain problemFixation probability for (a) the neutral case, (b) the case withconstant fitness, (c) 2 × 2 games with finite populationsInfluence of fluctuations: fixation and new criteria for evolutionarystability (ESS N for 2 × 2 games)Mauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2

OutlookFurther topics and some open problems (non-exhaustive list):Replicator dynamics for Q × Q games:For Q ≥ 3: Replicator equations ⇒ cycles, oscillations, chaos(Q > 3), ...Spatial degrees of freedom and role of mobility (PDE): patternformationQ × Q games with mutationsStochastic evolutionary dynamics:Stochastic evolutionary game theory on lattices and graphsCombined effects of mobility, fluctuations, and selectionFor Q × Q games, with Q ≥ 3:Diffusion approximation (Fokker-Planck equation)Fixation and extinction times (e.g. as first-passage problems)Generalization of the concept of ESS NMauro Mobilia Evolutionary Game Theory: An Introduction, Lecture 2