The El Farol Bar Problem for next generation systems

**The** **El** **Farol** **Bar** **Problem** **for** **next** **generation**

**systems**

Athanasios Papakonstantinou

Dissertation submitted **for** the MSc in Mathematics

with Modern Applications

Department of Mathematics

August 2006

Supervisor Dr. Maziar Nekovee, BT Research

Contents

1. Chaos, Complexity and Irish Music 1

1.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2. **The** **El** **Farol** **Bar** **Problem** (EFBP) . . . . . . . . . . . . . . . . . 4

1.3. Modelling the original problem . . . . . . . . . . . . . . . . . . . 5

1.4. Game **The**ory Definitions . . . . . . . . . . . . . . . . . . . . . . . 7

2. Previous Approaches to the **El** **Farol** 9

2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2. Approaches to EFBP . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.1. Minority Game . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.2. Evolutionary Learning . . . . . . . . . . . . . . . . . . . . 11

2.2.3. Stochastic Adaptive Learning . . . . . . . . . . . . . . . . 13

3. Analysis and Extension of the Stochastic Algorithm 23

3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2. Taxing/Payoffs Algorithms . . . . . . . . . . . . . . . . . . . . . . 23

3.2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2.2. Fairness and Efficiency . . . . . . . . . . . . . . . . . . . . 29

3.3. Budget Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3.2. Fairness and Efficiency . . . . . . . . . . . . . . . . . . . . 34

4. Case Study: Multiple **Bar**s in Santa Fe 37

5. Conclusions 43

A. C Code **for** the **El** **Farol** **Bar** **Problem** 47

ii

List of Figures

1.1. **Bar** attendance in the first 100 weeks [1]. . . . . . . . . . . . . . . 5

2.1. Behaviour of the average attendance (Top) and of the fluctuations

(bottom) in the **El** **Farol** problem with L = 60 seats, ā = 1/2 and

m = 2, 3, 6 from left to right [2]. . . . . . . . . . . . . . . . . . . . 10

2.2. Mean weekly attendance **for** all 300 trials[3]. . . . . . . . . . . . . 12

2.3. **The** attendance in a typical trial[3]. . . . . . . . . . . . . . . . . . 12

2.4. **The** normalised one-step transition matrix[3]. . . . . . . . . . . . 13

2.5. **The** overall attendance and the probabilities **for** each of the M

agents **for** ‘partial in**for**mation’, ‘full in**for**mation’ and ‘signs’ algorithms.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.6. histograms **for** partial and full info algorithms. . . . . . . . . . . . 18

2.7. fairness and efficiency plots **for** original algorithms. . . . . . . . . 20

2.8. standard deviation off attendance **for** original algorithms. . . . . . 21

3.1. histograms **for** partial and full info taxing algorithms. . . . . . . . 25

3.2. **The** overall attendance and the probabilities **for** each of the M

agents **for** ’partial’ in**for**mation tax, modified tax algorithms and

the full in**for**mation tax algorithm. . . . . . . . . . . . . . . . . . 26

3.3. fairness and efficiency plots **for** taxing algorithms. . . . . . . . . . 30

3.4. standard deviation off attendance **for** taxing algorithms. . . . . . . 31

3.5. histograms **for** budget algorithms (b = 10, w = 0/6). . . . . . . . . 32

3.6. **The** overall attendance and the probabilities **for** each of the M

agents **for** budget (budget= 10) and modified budget (budget= 10,

wait=3, 6) algorithms. . . . . . . . . . . . . . . . . . . . . . . . . 33

3.7. standard deviation off attendance **for** budget algorithms. . . . . . 34

3.8. fairness and efficiency plots **for** budget algorithms. . . . . . . . . . 35

4.1. **The** overall attendance and the probabilities **for** each of the M

agents **for** each bar. . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.2. **The** cumulative attendance **for** all three bars. . . . . . . . . . . . . 39

4.3. Agent attendances **for** each bar . . . . . . . . . . . . . . . . . . . 40

4.4. payoff probabilities **for** each bar and cumulative payoff probability

**for** the 3-bar version . . . . . . . . . . . . . . . . . . . . . . . . . 41

iv

List of Tables

1.1. **The** Prisoner’s Dilemma in matrix **for**m with utility pay-offs. . . . 8

2.1. Behaviour of the ‘partial in**for**mation’ algorithm. . . . . . . . . . . 16

2.2. Behaviour of the ‘full in**for**mation’ algorithm. . . . . . . . . . . . . 16

2.3. Behaviour of the ‘signs’ algorithm. . . . . . . . . . . . . . . . . . 18

2.4. standard deviation and mean of payoff probabilities **for** original

algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1. Behaviour of the ‘tax’ algorithm with ctax = 6. . . . . . . . . . . . 24

3.2. Simulation results **for** various values of ctax, cp. . . . . . . . . . . . 25

3.3. standard deviation and mean of payoff probabilities **for** taxing algorithms.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.4. Behaviour of the ‘modified budget’ algorithm **for** budget=10 and

staying at home=6. . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.5. Simulation results **for** budget algorithms. . . . . . . . . . . . . . . 32

3.6. standard deviation and mean of payoff probabilities **for** budget

algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.1. mean and std **for** all bars. . . . . . . . . . . . . . . . . . . . . . . 39

vi

Acknowledgements

I would like to dedicate this dissertation to my parents Costantinos and Valentini

Papakonstantinou . I want to thank them **for** giving me the opportunity to study

in the UK and **for** always supporting me in all my decisions.

Special thanks to my supervisor in BT Research, Maziar Nekovee, **for** his help

and guidance. He was a very valuable source of in**for**mation and always available

when I needed him. Also Keith Briggs in BT Research was very helpful and this

is very much appreciated.

Futhermore, I would like to thank all the friends in York and in Ipswich **for**

this last year, as well as all friends in Greece.

vi

Chapter 1

Chaos, Complexity and Irish

Music

1.1 Overview

**The**re is a common misconception that complexity and chaos are synonymous.

Besides the nonlinearities that occur in both **systems**, the other properties those

two areas of mathematics share are disambiguity in definitions and having many

interesting and different applications.

Although a definition of chaos that everyone would accept does not exist,

almost everyone agrees to three properties a chaotic system should have: Chaos

is aperiodic long-term behaviour in a deterministic system that exhibits sensitive

dependence on initial conditions[4]. A time evolving property such as the move of

tectonic plates or planets, the temperature or any weather characteristic or even

the price movements in stock markets or dreams and emotions [5] may display

chaotic behaviour.

Chaos is often related to complexity, but does not follow from it in all cases.

Chaos might be occurring when studying phenomena as they progress in time,

but when the same phenomena are examined from a microscopic point of view

then, the interaction of the various parts of which the system is consisted creates

patterns and not ‘erratic’ chaotic behaviour. This is where complexity enters.

It is rather difficult to define complexity in mathematical terms, although

there is a measure of complexity there is no other way to give mathematical

definition. Dictionaries might be useful in this quest **for** defining complexity.

According to an online Dictionary by Ox**for**d University Press, complex is an adjective

used to describe nouns which ‘are consisted of different and connected

parts’. An even more precise definition is ‘consisted of interconnected or interwoven

parts.’[6] That means, that in order to understand the behaviour of a complex

system we should understand the behaviour of each part, as well as how they interact

to **for**m the behaviour of the whole. Our incapacity to describe the whole

1

2 CHAPTER 1. CHAOS, COMPLEXITY AND IRISH MUSIC

without describing each part combined with the necessity to relate each part with

another when describing makes the study of complex **systems** very difficult.

Finally, based on [6] an attempt to **for**malise all the above definitions can be

made: a complex system is a system **for**med out of many components whose behaviour

is emergent,that is, the behaviour of the system cannot be simply inferred

only from the behaviour of its components. **The** amount of in**for**mation needed

to describe the behaviour of such a system is a measure of its complexity. If the

number of the possible states the system could have is Ω and it is needed to

specify in which state it is in, then the number of binary digits needed to specify

this state is related to the number of the possible states:

I = log 2(Ω) (1.1.1)

In order to realise which state the system is in, all the possible states must

be examined. **The** fact that the unique representation of each state requires

a number equal to the number of the states, leads to the conclusion that the

number of states of the representation is equal to the number possible states of

the system. For a string of N bits, there are 2 N possible states, there**for**e:

Ω = 2 N ⇔ N = I (1.1.2)

**The**re are many applications of complex **systems**, in statistical physics, meteorology,

geology, biology, engineering, economics, even social sciences and psychology.

In all sciences we could find **systems** that could be dismantled in their

core components and study each part and the system as a whole simultaneously.

It is very interesting trying to examine and **for**ecast the behaviour of **systems**

that consist of human beings, **systems** like a family, or a business, or even a government.

Humans have the capacity to learn and to constantly evolve, making

models that deal with them unrealistic and of no use if this property is not taken

into consideration. **The**re**for**e the need to model this human behaviour created

the complex adaptive **systems** (CAS). **The** most common definition and universally

approved is the one given by one of its founders, John H. Holland of Santa

Fe Institute: ‘A Complex Adaptive System (CAS) is a dynamic network of many

agents (which may represent cells, species, individuals, firms, nations) acting in

parallel, constantly acting and reacting to what the other agents are doing. **The**

control of a CAS tends to be highly dispersed and decentralised. If there is to

be any coherent behaviour in the system, it has to arise from competition and

cooperation among the agents themselves. **The** overall behaviour of the system is

the result of a huge number of decisions made every moment by many individual

agents’[7]. It is even more intriguing that CAS do not appear only in human

networks but wherever there is a system with interacting elements. It could be

cells in a cellular automaton, ions in a spin glass or even cells in an immune

system[8]. A complex adaptive system besides the property of complexity, has

the properties of emergence and self-organisation.

1.1. OVERVIEW 3

Emergence occurs when agents that operate in the same environment start

to interact which each other. **The** number of the interactions increases when

the number of agents increases, this leads to the appearance of new types of

behaviour. This process can result to an increase of the complexity and since it

is an internal property of the system and not managed by an outside source, it

is a ‘self-organised’ process.

**The** subject of this dissertation is the study and expansion of a famous complex

adaptive system known as **El** **Farol** **Bar** **Problem** which was introduced by

the economist W. B. Arthur in 1994[1]. **El** **Farol** is a bar in Santa Fe in New Mexico

which plays each Thursday Irish music. People enjoy visiting it and hearing

some quality music but eventually it becomes overcrowded, so people stop enjoying

themselves. Each customer decides independently whether to attend or not,

based on a set of predictors. This scenario provides a simplified mathematical

model of a class of congestion and coordination problems that arise in modern

In**for**mation and Communications Technology (ICT) **systems**.

One application of great interest is networks of cognitive radios, where agents

compete with each other **for** the same resource (RF spectrum). Cognitive radios

are autonomous agents that have the ability to sense the external environment,

learn from history and make intelligent decisions in order to optimise their per**for**mance

and adjust better to the environment[9]. Another application is internet

when a large number of people try to visit the same web page or access the same

ftp server simultaneously and independently.

In this first chapter, the original Arthur’s EFBP is defined and analysed. In

the end of the chapter, some basic Game **The**ory concepts are explained and

defined.

In chapter two, various different approaches to the **El** **Farol** **Bar** problem are

reviewed. First it is viewed as a minority game and various techniques from

statistical mechanics are implemented. Also strategies are redefined using a binary

approach as an attempt to reduce complexity. **The** **next** approach tries to

overcome the restrictive strategies using an evolutionary learning algorithm and

viewing the problem as a Markov stochastic process. **The** last approach suggests

a very simple adaptive algorithm which is based on the maximisation of the

probability of attendance **for** each agent. **The**re are no specific strategies to guide

agents during their decision process, only their intention to attend the bar.

In chapter three, the last algorithm is analysed in depth. In this original work,

the stochastic adaptive learning algorithm is extended and several derivatives of

it are examined as an attempt to deal with the unfairness or low efficiency issues

that occurred in some cases with the original algorithm. Considerable ef**for**t was

put in order to define the stationary state of one variation. Also fairness and

efficiency are defined and measured both from the bar management’s and agent’s

point of view.

In chapter four, it is examined whether three bars in the same town would

affect the agents ways of entertainment. **The**y attempt to enter the bars in a

4 CHAPTER 1. CHAOS, COMPLEXITY AND IRISH MUSIC

random order, but their decision is made using the original algorithm.

1.2 **The** **El** **Farol** **Bar** **Problem** (EFBP)

Arthur in his paper, tries to predict the bar’s attendance. He assumed that the

number of the resindents of Santa Fe or prospective clients of the bar is 100 and

60 as the maximum number of the clients the bar should have so it could not be

overcrowded. **The**n he answered the above question mentioning that if a person

expects fewer than or equal to 60 to show up, he attends the bar, otherwise he

stays at home. Each person cannot communicate with others, so no one has any

in**for**mation about the intentions of everybody else. **The**y only have access to the

number of the clients of the previous weeks.

But there is more than one model, based on the numbers of the previous

weeks, which can be used to predict the current week’s attendance. This makes

a rational solution impossible and the problem from the client’s point of view

ill-defined. But even if there was only one model, or due to mysterious reasons

the clients managed to have common **for**ecasts, the model would fail. If most of

the people believe that the bar will be overcrowded, then they will not attend

leaving the bar almost empty. **The** opposite will happen if most of them think

that the bar will have less than 60 customers.

In order to overcome problems such as this, Arthur issued the use of ‘predictors’.

A predictor is a function that maps the in**for**mation of d-recent attendances

into a prediction **for** the **next** attendance. Arthur suggested that although there

are many predictors, each individual has a set of k predictors in his disposal which

will guide him through the decision process. Each client will decide whether to

go to the bar or not, according to the most accurate predictor in his set, which

will be called ‘active predictor’. Inaccurate predictors do not affect the long term

behaviour of the system, since they will rarely achieve the status of ‘active’ predictor,

there**for**e will be rarely used from the clients. **The** predictors that were

actually used in the original problem are described in subsection 1.3.

**The** results of the computational experiment are shown in fig 1.1. **The**se results

indicate a tendency of the mean attendance to converge to 60. It seems that

the predictors self-organise into an equilibrium pattern or ‘ecology’[1]. **The** active

predictors **for**ecast above 60 with propability 0.4 and below 60 with propability

0.6. In terms of game theory, the above mixed strategy is a Nash equilibrium.

(Game theory terms like Nash equilibrium, strategies, repeated games and other

are defined in subsection 1.4.)

**The** EFBP is a congested resource problem, because the final decision of an

agent (this is how clients, customers or individuals who decide whether or not

they should attend **El** **Farol** **Bar** are going to be referred from now on) depends

on the decision of the other agents. In other words they compete **for** a resource.

This congestion appears in many real life **systems** like the internet where the

1.3. MODELLING THE ORIGINAL PROBLEM 5

Figure 1.1: **Bar** attendance in the first 100 weeks [1].

users compete **for** bandwith or roads and highways. **The** source of congestion,

in a deterministic framework like EFBP, is the inability of agents to coordinate

their actions[10], since there is a lack of a centralised mechanism that could guide

them. In order to understand better and analyse such **systems** the traditional

perfect, logical, deductive rationality has given its place to bounded rationality.

**The** agents make their decision based on incomplete knowledge, they know that

they have access to limited in**for**mation and do their best to fight this uncertainty

using a combination of rational rules and empirical evidence.

1.3 Modelling the original problem

Now that Arthur’s original EFBP is explained (section 1.2) the model can be

summarised as following:

Suppose that there are N agents that have to decide whether they will or not

attend the bar, and L is the maximum number of clients the bar can accommodate,

be**for**e becoming overcrowded. Arthur wanted to predict the binary action

of the i th customer denoted by a i ∈ {0, 1}, where 1 stands **for** going to the bar

and 0 is not going. **The** total attendance is A = a i .

As mentioned above, the only in**for**mation available to the agents is the number

of the customers of the bar, the previous d weeks:

It = {A(t − d), . . . , A(t − 2), A(t − 1)} (1.3.1)

A predictor is a function I ∈ [0, N] d → [0, N]. **The**re are (N + 1) (N+1)d

predictors [2] **The** number of the possible predictors that could be used is rather

large, that is why a selection of S predictors is used. Arthur used the following

predictors:

6 CHAPTER 1. CHAOS, COMPLEXITY AND IRISH MUSIC

1. **The** same attendace as k weeks ago:

A(It) = A(t − k),

where k in the original models 1, 2 and 5. (k-period cycle detectors)

2. A mirror image around 50% of last week’s attendance:

3. A fixed predictor:

A(It) = N − A(t − 1)

A(It) = 67

4. A rounded average of the last four weeks:

A(It) = 1/4

4

A(t − r)

5. A rounded and bound by 0 and N, last 8 weeks trend, computed using the

least squares method.:

r=1

A(It) = min([trend{A8}] + , N)[11]

Each predictor has a score associated to it, which evolves according to:

Ui,s(t + 1) = Ui,s(t) + Θ{[Ai,s(It) − L][A(t) − L]}[2]

Where, s ∈ [1, S], Ai,s(It) the s t h predictor **for** i t h customer and Θ is the Heaviside

function (Θ(x) = 0 **for** x < 0 and Θ(x) = 1 **for** x ≥ 0).

Also the predictor s used by i th customer is given by:

and

si(t) = argmax s ′Ui,s ′(t)

ai(t) = Θ[L − Ai,si(t)(It)]

where, argmax xf(x) is the value of x **for** which, f(x) gets its maximum value.

1.4. GAME THEORY DEFINITIONS 7

1.4 Game **The**ory Definitions

Games and solutions

A game is a description of strategic interaction that includes the constraints on the

actions the players can take and the players interests, without actually specifying

the actions that the players do take[12]. A solution is how rational people play

the game. A ‘rational solution’ corresponds to a set of unique strategies (plans

**for** player’s actions) **for** each player that they will have no rational reasons to

regret choosing[13].

Best reply strategy

A strategy **for** player, Ri is best reply(or best response) to C’s strategy Cj if it

gives R the largest pay-off, provided that C has played the game.

Pure and mixed strategies

Pure strategy is the simplest kind of strategy, where someone chooses a specific

course of action. However there might be a case where there is uncertainty

about which best pure strategy to choose, due to lack of in**for**mation or any

other reason. At those cases, the pure strategy is chosen following a random

probability distribution. This type of strategy is called mixed strategy. A more

strict definition follows:

If a player has N available pure strategies (S1, S2, . . . , SN), a mixed strategy M

is defined by the probabilities (p1, p2, . . . , pN) of each strategy to be selected[13].

For M to be well defined the sum of the probabilities should be equal to one.

Nash equilibrium

**The** outcome of strategies Ri **for** player R and Cj **for** player C is a Nash equilibrium

of the game, and thus a potential solution if Ri and Cj are the best solutions

to each other. In a way player R chooses Ri because he is expecting C to chose

Cj (and vice verse). When people select Nash equilibrium strategies there is no

guarantee that they will be happy. **The**re is however a guarantee that they will

have no reason to change it.

Nash equilibrium in pure strategies

Let G be a game, which involves N players. Each player chooses among a finite

set of strategies Si: That is, player i (i = 1, . . . , N) has access to strategy set

Si from which he/she chooses strategy σi ∈ Si. A set of pure strategies S =

(σ1, σ2, . . . , σi, . . . , σN) constitutes a Nash equilibrium if and only if pure strategy

σi is a best reply to the combination of the strategies of all other players in S **for**

all i = 1, . . . , N[13].

Nash equilibrium in mixed strategies

Mixed strategies are in Nash equilibrium, when there is not any strategy available

the player could choose in order to improve his/her expected utility.

8 CHAPTER 1. CHAOS, COMPLEXITY AND IRISH MUSIC

Example of Nash equilibrium:Prisoner’s Dilemma

This is a very known example in game theory about two arrested suspects **for** a

crime. **The**y are put in different cells and are promised that if someone confesses,

he will be freed and used as a witness against the other, who will be sentenced

to four years. If they both confess, they receive a three year sentence and if

nobody confesses they will be convicted **for** a year, due to lack of evidence. **The**

problem is represented in a matrix **for**m in table 1.1, using utility payoffs. For

Cooperate Defect

Cooperate 3,3 0,4

Defect 4,0 1,1

Table 1.1: **The** Prisoner’s Dilemma in matrix **for**m with utility pay-offs.

each player action ‘D’ (Defect) dominates action ‘C’ (Cooperate). Comparing

the first numbers in each column or the second numbers in each row, shows that

no matter what player 1 (column) chooses, player 2 (row) will win by choosing

Defect, since the reward (utility payoff) is higher. This demonstrates that action

(D,D) is a unique Nash equilibrium.

Pareto improvement and efficiency

Pareto improvement is when an agent chooses a strategy that will have no negative

effects on the others. A system is Pareto efficient or Pareto optimal, when no

Pareto improvements can be made. In other words, in a Pareto efficient system

no individual can make an improvement without worsening the others.

Repeated games

Features of ‘one-shot’ games like ‘Prisoner’s Dilemma’ are the total lack of cooperation

and the inability to study how each player’s actions affect the others as

time progresses. **The** model of a repeated game is designed to examine long term

interaction, based on the idea that a player will take into account other players

estimated future behaviour, when planning his current strategy. This theory tries

to isolate types of strategies that support mutually desirable outcomes in every

game and punishes players with undesirable behaviour. ‘Folk’ theorems [12] give

the conditions under which the set of the payments, that are acquired when in

equilibrium, will consist of nearly all reasonable payoff profiles. With the use of

these theorems it is proved acceptable results cannot be sustained if player are

‘short-sighted’ and only look after their own interests.

Chapter 2

Previous Approaches to the **El**

**Farol**

2.1 Overview

In this section, various different approaches to **El** **Farol** **Bar** **Problem** are reviewed.

Furthermore, three ways to extend the original model and analyse it using a

different perspective, are introduced.

2.2 Approaches to EFBP

2.2.1 Minority Game

In this approach, results known from minority games and tools of statistical

mechanics are used **for** EFB model analysis. A minority game is a binary game

where N (N:odd) players must choose one of the two sides independently and

those on the minority side win. Players use a set of strategies, based on the past,

to make their selections [14]. **The** greatest difference of this model with Arthur’s

is the introduction of strategies instead of predictors. A strategy is a function

a(I) from [0, N] m like predictors, but to {0, 1} instead to [0, N]. In other words

strategies estimate if an agent should visit or not the bar, based on the previous

history of attendance but in terms of ones (below the level of attendance) and

zeros (above it). This is of great importance since the number of strategies is

2N+1d which is significantly less from the number of the predictors **for** large N.

This could be denoted as:

which will depend only on the in**for**mation

a µ

s,i = Θ[L − As,i(It)] (2.2.1)

µ(t) = {Θ[L − A(t − 1)], . . . , Θ[L − A(t − d)]} (2.2.2)

9

10 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

**The**re are only 22d [2] strategies of this type, which is even less from 2N+1d and

independent from N. Each agent is assigned S strategies drawn from the pool

with distribution: P (a) ≡ Prob{a µ

s,i } = āδ(a − 1) + (1 − ā)δ(a). For this model

to work, it is considered that on average clients attend the bar with a frequency

L/N. This leads to ā ≈ L/N, where ā is the average of a µ

s,i .

Figure 2.1, illustrates what happens to the model when L, ā and m remain

fixed, while the number of agents N increases. **The** top graph shows that when

Nā ≈ L, the attendance converges to the com**for**t level, while as m is increasing

the area where 〈A〉 ≈ L is shrinking.

Another feature of this approach is that it measures wasted resources which

appear when the bar is under or over utilised. Although A(t) equals L on average,

the amount of unexploited resources (A(t) < L) or over-exploited resources

(A(t) > L), is equal to the distance |A(t) − L|. Thus, the quality of the cooperation

of the agents is measured by:

where 〈. . . 〉 is the average on the stationary state.

σ 2 = 〈(A − L) 2 〉 (2.2.3)

Figure 2.1: Behaviour of the average attendance (Top) and of the fluctuations

(bottom) in the **El** **Farol** problem with L = 60 seats, ā = 1/2 and m = 2, 3, 6

from left to right [2].

On the bottom part of figure 2.1 it is shown that **for** small m, σ 2 /N is at is

peak at the point where āN = L, while as m increases the maximum is getting

shallower until it is disappeared **for** m = 6. This leads to the conclusion that, **for**

larger values of m, the efficiency increases. More in**for**mation about this model

can be found in [2].

2.2. APPROACHES TO EFBP 11

2.2.2 Evolutionary Learning

This approach is based on the fact, that since EFBP is a case where inductive

reasoning and bounded rationality are experienced, models based on a closed set

of strategies are inadequate. It introduces a stochastic element, since new models

are created by randomly varying existing ones, as well as a selective process which

eliminates the ineffectual models.

Suppose that, like in Arthur’s original experiment, each agent is given k = 10

predictive models. For simplicity these models are autoregressive and their output

unsigned and rounded. According to [3] **for** the i th individual their j th predictor’s

output is given by:

ˆx i ⎛

j(n) = round ⎝

aij(0) +

l i j

t=1

a i ⎞

j(t)x(n − t) ⎠

(2.2.4)

where x(n − t) is the attendance on week (n − t), l i j is the number of lag terms in

the j th predictor of individual i, a i j(t) is the coefficient **for** the lag t steps in the

past, and a i j(0) is the constant term of the AR model.

**The** absolute value and the rounded output makes sure that no negative values

are assigned and all predictions above 100 were set to 100 according to the

original model. For each individual the number of lag terms is chosen uni**for**mly

from the integers {1, . . . , 10} [3]. Be**for**e the prediction of the attendance of the

current week, each individual evolves its set of models **for** ten **generation**s. This

procedure, analysed in [3], is synopsised in the following five steps:

1. As it was mentioned above, each agent chooses from 10 models. In this stage

an offspring is created **for** each agents k th model. Lag in the offsprings from

parents j is set to be one or ten. If l i j = 1 then l i j − 1 is not allowed, while if

it is equal to ten, l i j + 1 is not allowed. **The** AR coefficients of the offspring

are generated with the addition of a zero mean Gaussian variable with

standard deviation (std) equal to 0.1. Any newly generated AR coefficients,

are chosen by sampling **for**m N(0, 0.1). In the end of this stage, there are

ten parent and ten offspring AR models assigned to each individual.

2. In this stage, the 20 models assigned to each agent are evaluated based

on the sum of the squared errors made during the prediction of the bar’s

attendance in the last 12 weeks.

3. For each agent, the ten models with the least error are selected and set as

parents **for** the **next** **generation**.

4. If less than ten **generation**s are conducted, then it starts over again from

stage 1. Otherwise the best model **for** each agent is used to predict current

week’s bar attendance.

12 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

5. If the maximum number of weeks is achieved the algorithm ends, if else,

the predicted attendance is recorded and the simulations starts over again

from stage 1.

Figure 2.2: Mean weekly attendance **for** all 300 trials[3].

Figure 2.3: **The** attendance in a typical trial[3].

**The** results of this procedure being repeated 300 times, are shown in figure

2.2. **The** mean weekly attendance **for** the first 12 weeks was 59.5, but **for** the

**next** 50 weeks, large oscillations appeared until week 100. From 100 to 982 (end)

weeks, the behaviour of the model could be described as ‘transient’. **The** mean

attendance was 56.3155 and std was 1.0456, which is statistically significantly

different (p < 0.01)[3] from the results of the Arthur’s original paper. None of

the 300 trials showed convergence to 60 and the results of each trial were similar

with those illustrated in figure 2.3.

**The** dynamics of this system do not provide useful results about the model’s

overall behaviour. That is why stochastic models based on Markov chains were

2.2. APPROACHES TO EFBP 13

Figure 2.4: **The** normalised one-step transition matrix[3].

used. **The** weekly attendance is the system’s ‘state’ in a simple first-order random

process. Each of the attendance transitions from week to week **for** all the 300

trials were tabulated and the transition matrix in fig 2.4 was **for**med. It was also

proved that the system has the Markov property by executing 300 additional

trials and recording each final weekly attendance at week 982. **The** cumulative

distribution of these attendances is similar to the cumulative distribution function

after the summing of the limiting probability masses calculated from raising the

transition matrix to a large number.

2.2.3 Stochastic Adaptive Learning

In Arthur’s original paper each agent tries to predict how many others will attend

**El** **Farol** **Bar**. Each individual decides based on a set of strategies (predictors)

which estimate the attendance of the **Bar**. Approaches 1 and 2 mentioned in

subsections 2.2.1 and 2.2.2 also use similar methods, although they try to refine

the decision process. In this approach it is shown that there is no need **for** the

agents to use different strategies and change them trying to find which is the more

accurate. **The** problem is considered in stochastic terms instead of deterministic

and a simple adaptive learning algorithm is implemented. **The** main advantage

of this method is that the algorithm is more simple and the decision process is

less complex[10] since the agents do not decide based on the decision of all the

others but based only on their recent experiences in the **Bar**.

**The** agents have identical payoffs, b is the payoff **for** attending a noncrowded

bar, g **for** attending a crowded bar and f **for** staying at home. Without loss

of generality h is considered to be zero. **The**re are two strategies: either the

agent attends the bar and receives payment b or g according to the attendance

14 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

of the bar, or he stays at home and receives no payment. In a mixed strategy

equilibrium the expected payment of following one action is equal to the expected

payment of following the other. This could be denoted as following:

g P(N −i ≤ N − 1) + b P(N −i > N − 1) = 0 ⇔

P(N −i ≤ N − 1) = b

b − g

where, b,g are the payoffs, M the total number of players, N the total observed

attendance, N −i the observed attendance without agent i and N the maximum

capacity of an uncrowded bar.

Using a deterministic setting where agents would only use pure strategies

has the disadvantage that each agent must predict the attendance, based on the

predictions of the others. This generates results with a high level of noise and high

deviation. In this bounded rational model, the adaptive learning rule depends

only on the history of the decisions of each single one agent.

To overcome this problem a common sense concept is taken into consideration:

people in general prefer to experience good times, tend to repeat the enjoyable

and minimise the unpleasant[10]. So according to this if an agent initially attends

the bar p percent of time, he will increase this if the bar is uncrowded or will

decrease this the bar is crowded. As time goes by , agents gather in**for**mation

about the attendance of the bar, in the **for**m of the parameter p which differs

**for** each ith agent. If k is the iteration counter (time), pi the probability that

ith agent attends and µ the parameter that defines the degree of change of pi

according to the attendance, then the number of agents attending at time k is

given by:

M

N(k) − xi(k) (2.2.5)

i=1

where xi(k) are independent Bernoulli random variables that are equal to one

with probability pi(k) and equal to zero otherwise[10]. **The** following simple

algorithm describes the evolution of pi(k):

⎧

⎪⎨ 0, pi(k) − µ(N(k) − N )xi(k) < 0

pi(k + 1) = 1, pi(k) − µ(N(k) − N )xi(k) > 1

⎪⎩

pi(k) − µ(N(k) − N )xi(k), otherwise

(2.2.6)

At each timestep k the agent attends the **Bar** with a probability pi(k) after

tossing a biased coin. If the bar is uncrowded N(k)−N is added to pi(k), while if

it is crowded N(k) − N is subtracted. Also pi(k) ∈ [0, 1] since it is a probability.

If the agent does not attend the bar, xi(k) = 0 leads to pi(k + 1) = pi(k).For

now on the algorithm 2.2.6 will be referred as ‘partial in**for**mation’ because the

agents make their decision relying only on their previous experience. But this

2.2. APPROACHES TO EFBP 15

algorithm could be modified in an attempt to generate results close to Arthur’s

original algorithm. In the following ‘full in**for**mation’ algorithm the decisions are

made after having a full record of attendance.

⎧

⎪⎨ 0, pi(k) − µ(N(k) − N ) < 0

pi(k + 1) = 1,

⎪⎩

pi(k) − µ(N(k) − N ),

pi(k) − µ(N(k) − N ) > 1

otherwise

(2.2.7)

Both these algorithms rely solely on attendance and not on payoffs. A way to

implement payoffs is setting the payoff **for** attending an uncrowded bar to µ and

−µ **for** attending a crowded bar. If the payoff **for** staying at home is set to 0, the

following algorithm depends on payoffs.

⎧

⎪⎨ 0, pi(k) − µsgn(N(k) − N )xi(k) < 0

pi(k + 1) = 1, pi(k) − µsgn(N(k) − N )xi(k) > 1

⎪⎩

pi(k) − µsgn(N(k) − N )xi(k), otherwise

(2.2.8)

where sign is the following function:

⎧

⎪⎨ −1, (N(k) − N ) < 0

sgn(N(k) − N ) = 0, (N(k) − N ) = 0

⎪⎩

1, (N(k) − N ) > 0

**The** general behaviour of the ‘partial in**for**mation’ algorithm can be seen in

the first two plots of figure 2.5. **The** simulation is run **for** M = 100, N = 60,

µ = 0.01 and the initial probabilities follow a random uni**for**m distribution. After

many iterations, the agents are separated into two groups with those of the first

group attending every day, while those of the other do not attend at all, or attend

very rarely. **The** observed attendance after many iterations is slightly below the

Nash equilibrium point which is always equal to the maximum capacity of the

uncrowded bar. Partial in**for**mation algorithm converges to a value near that

point N − 1 and never reaches N , which is a Pareto efficient point **for** that

algorithm since the only way **for** an agent to be in a better position is to worsen

somebody else[15].

This algorithm is heavily dependant on the value of µ. Its convergence is

strongly affected by this parameter, **for** µ = 0.1, M = 100 and N = 60 it

converges after 800 iterations, while **for** µ = 0.001, 13000 iterations were not

enough. In order to explore the behaviour and the limitations of the algorithm

a lighter version of the original C program was used. It is different from that in

the Appendix, since it produces no files **for** plots and the only outputs are the

average and standard deviation. A conclusion that could be drawn only from the

simulations (Table: 2.1) is that the algorithm can handle relatively large numbers

such as M = 10000, N = 6000, provided that µ is very small, in expense of

16 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

convergence time . But the conclusion is that only hardware limitations affect its

behaviour.

N /M 60/100 600/1000 6000/10000

iterations 2 · 10 8 2 · 10 8 2 · 10 8

µ 0.01 0.001 0.001

average 59.00020 599.000142 5898.745966

std 0.006524 0.052298 74.159661

Table 2.1: Behaviour of the ‘partial in**for**mation’ algorithm.

**The** comparison of the first two with the **next** two plots of figure 2.5 leads to

the conclusion that agents coordinate successfully when they have access to partial

in**for**mation instead of full in**for**mation. This happens due to the congestion,

which comes as a result of the similar response of the agents who have available

all the in**for**mation.

**The** results of the full in**for**mation algorithm are very similar to Arthur’s

original simulation (Figure:1.1). **The** mean attendance is 60 but the variation

never settles down. Although the probabilities bounce randomly, they increase

or decrease simultaneously. This proves the assumption made above, that when

agents have access to full in**for**mation, they tend to have the same behaviour.

**The** behaviour of the algorithm is summarised in table 2.2

N /M 60/100 600/1000 6000/10000

iterations 2 · 10 8 2 · 10 8 2 · 10 8

µ 0.01 0.001 0.001

average 60.00000 599.99999 5999.999994

std 6.454943 18.371018 57.254445

Table 2.2: Behaviour of the ‘full in**for**mation’ algorithm.

**The** first and third plots in figure 2.5 show that ‘partial in**for**mation’ and

‘signs’ algorithm have similar behaviour. A close look in the probabilities **for** each

agent, reveals that the new algorithm inherits all the properties of the original.

**The** only difference is that standard deviation is slightly greater, but that can be

explained since the new algorithm converges after more iterations. With the same

constants, more than 3000 iterations are needed **for** the algorithm to converge.

**The** behaviour of the algorithm is summarised in table 2.3. It can be seen that

this variation does not have any problems with large numbers, probably because

it is more simple. Instead of using in every iteration the quantity N(k) − N , it

uses sign(N(k) − N ).

**The** histograms of the attendances (figure: 2.6) **for** partial and full in**for**mation

algorithms show that they have a completely different distribution and indicate

that attendances in full in**for**mation algorithm may follow normal distribution.

2.2. APPROACHES TO EFBP 17

attendance

attendance

attendance

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

100

80

60

40

20

0

0 1000 2000 3000 4000 5000

time (iterations)

probabities **for** each agent

probabities **for** each agent

probabities **for** each agent

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0 1000 2000 3000 4000 5000

time (iterations)

Figure 2.5: **The** overall attendance and the probabilities **for** each of the M agents

**for** ‘partial in**for**mation’, ‘full in**for**mation’ and ‘signs’ algorithms.

18 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

frequency

0 500 1000 1500 2000

N /M 60/100 600/1000 6000/10000

iterations 2 · 10 8 2 · 10 8 2 · 10 8

µ 0.01 0.001 0.001

average 59.000049 599.059126 5998.999914

std 0.011685 0.286160 1.078615

Table 2.3: Behaviour of the ‘signs’ algorithm.

Histogram of attendances in partial info

50 55 60

attendances

65

frequency

0 50 100 150 200

Histogram of attendances in full info

40 50 60

attendances

70 80

Figure 2.6: histograms **for** partial and full info algorithms.

Except from the fast convergence and the ability to handle large numbers, it

is very important that some of the quantitive characteristics of the algorithms

are examined too. It is necessary that tools to measure abstract terms like fairness

and efficiency are developed. A fair outcome requires that agents with

similar utilities have similar probabilities to attend the bar. A more strict definition

would demand exact probabilities of attendance **for** agents with identical

payoffs[16]. **The** algorithm, must also be efficient both **for** the agents and the bar

management.

Be**for**e proceeding to the tools used, utility payoff, which as a term was first

introduced in Prisoner’s Dilemma definition in section 1.4, must be defined. Utility

payoffs are the rewards or penalties that each agent has, after following a pure

strategy. In this case, agents who attend a not crowded bar get a reward of 1,

those who attend a crowded bar get a penalty of −1, while those who stay at

home get nothing (reward=0):

⎧

⎪⎨ −1, xi(k) = 1, (N(k) − N ) > 0

U(k) = 1, xi(k) = 1, (N(k) − N ) ≤ 0

⎪⎩

0, xi(k) = 0

So, fairness and efficiency could be measured using the following methods based

on utility functions:

• efficiency is determined by the average payoff. **The** higher the average

payoff probability is, the higher is the average reward **for** each player

2.2. APPROACHES TO EFBP 19

• fairness is determined by the distribution of payoffs. A histogram which

shows the creation of groups is a clear indication that the algorithm is

unfair, similar conclusions could be drawn from the standard deviation.

High values of std indicate that a significant number of agents has payoffs

less than the average, while others have greater.

**The**re are also other ways to measure fairness and efficiency, which have nothing

to do with utility functions:

• system’s efficiency is determined by the attendance’s std. A system is

considered to be efficient, when the attendances are really close to system’s

capacity.

• system’s fairness can be also determined by the distribution of attendances

**for** each agent. At each iteration of the algorithm, the decision of

each agent is recorded. In the end of the algorithm it easy to calculate how

many times each agent has attended the bar. Histogram or std could used

in this case also.

It is called system’s efficiency, because most bar managements have interest in

keeping a stable attendance, with not many fluctuations which would leave the

bar some days underutilised and some days overutilised. Although system’s fairness

gives a good picture of the choices of each agent, the original fairness is more

important since it is calculated from the utility payoffs. After all, it could be said,

that what matters is the consequences of each agent’s action and the only way

to measure it, is in terms of reward or penalty.

Considering system’s fairness, it can be seen in figure 2.7 that both three

algorithms are not fair. Agents are divided in two categories: 41 of them rarely

attend the bar, and the rest of them attend it almost always. Although ‘partial

in**for**mation with signs’ algorithm was designed as an improvement of the original

one it inherits all of its properties. One of those properties is minimal attendance

**for** 41 agents and maximal **for** the rest. It seems that from the agents point of

view, this algorithm is more fair (Table: 2.4, stdsigns < stdpartial ), but in reality

it is not. This is a result of the lower profits **for** the always attending group and

not of a wider distribution of payoff probability.

std mean

partial 0.401591 0.4814233

signs partial 0.3018023 0.36663

full 0.02300045 −0.00500668

Table 2.4: standard deviation and mean of payoff probabilities **for** original algorithms.

**The** most fair algorithm regarding the agents is the full in**for**mation, indeed

standard deviation of payoff probability is nearly zero but so is average payoff

20 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

probability. Nobody makes profit, nobody is happy, but everybody is content

and they seem to have no intention of changing their strategy, all these are properties

of Nash equilibrium points. But in terms of attendance ‘full in**for**mation

algorithm’ is not absolutely fair, since again there is a classification of agents in

two groups, although this time casual bargoers have a much wider distribution.

frequency

0 10 20 30 40

frequency

0 5 10 15 20

frequency

0 10 20 30

partial in**for**mation

0.0 0.2 0.4 0.6 0.8 1.0

probability of attendance

full in**for**mation

0.2 0.3 0.4 0.5 0.6 0.7 0.8

probability of attendance

partial in**for**mation with signs

0.0 0.2 0.4 0.6 0.8 1.0

probability of attendance

frequency

0 10 20 30 40 50

frequency

0 1 2 3 4 5 6 7

frequency

0 5 10 15 20 25 30

partial in**for**mation

0.0 0.2 0.4

payoff probability

0.6 0.8

full in**for**mation

−0.06 −0.04 −0.02 0.00 0.02 0.04 0.06

payoff probability

partial in**for**mation with signs

0.0 0.1 0.2 0.3 0.4 0.5 0.6

payoff probability

Figure 2.7: fairness and efficiency plots **for** original algorithms.

It is obvious that these results are open to different interpretations, and vary

according to the objectives set each time. In general partial in**for**mation is the

most appropriate algorithm if the objective is to maximise the average payoff **for**

each agent. It might be the most unfair of all, but even full in**for**mation is not

fair enough to justify the minimal profits.

Considering system’s efficiency, it can be seen in the cumulative plot of the

running std **for** the original algorithms (Figure: 2.8), that ‘partial in**for**mation’

has the least standard deviation and a negative slope. On the contrary ‘full

2.2. APPROACHES TO EFBP 21

in**for**mation’ has a relatively high std, with no signs of improvement.

standard deviation

0 1 2 3 4 5 6

original algorithms

partial

full

signed

0 500 1000 1500

iterations

2000 2500 3000

Figure 2.8: standard deviation off attendance **for** original algorithms.

22 CHAPTER 2. PREVIOUS APPROACHES TO THE EL FAROL

Chapter 3

Analysis and Extension of the

Stochastic Algorithm

3.1 Overview

None of the previous three variations was proved to be fair and efficient. Fairness

and efficiency seem to be contradicting terms. In this chapter, new variations

of the stochastic adaptive learning algorithms that were presented in subsection2.2.3,

are examined. **The** variations could be divided into two categories: the

taxing/payoff algorithms which have to do with an adaptive change of µ and

the budget algorithms, where the agents are **for**ced not to attend the bar after

having attended it consecutively **for** a number of weeks, as a result of inadequate

resources.

3.2 Taxing/Payoffs Algorithms

3.2.1 Overview

**The** basic idea behind this algorithm is that in the system there are three types

of agents. **The** ‘selfish’ who attend a bar regardless if it is crowded or not, those

who attend an uncrowded bar and those who never attend the bar. ‘Partial in**for**mation’

maximises the probability of attendance incrementing a small quantity

to the original probability when the bar is uncrowded or subtracting it from the

original probability when the bar is crowded. This quantity depends on µ which

is constant. In this version (equations: 3.2.1), the parameter µtax is inserted so

that selfish behaviour can be penalised. When a selfish agent attends a crowded

bar, the quantity that is subtracted from the original probability is ctax times

greater than in the ‘partial in**for**mation’ algorithm.

Simulations indicated that **for** rather large values of ctax (ctax ≥ 8) the bar

is underutilised (mean ≤ 58). **The** behaviour of this algorithm is summarised in

23

24 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

table 3.1 and in the first two plots of figure 3.2.

N /M 60/100 600/1000

iterations 2 · 10 8 2 · 10 8

µ 0.01 0.001

average 59.072334 598.99924

std 0.259638 0.052933

Table 3.1: Behaviour of the ‘tax’ algorithm with ctax = 6.

This algorithm could be more fair **for** the agents if those who do not attend

the bar were encouraged to attend it. By definition of the algorithm this cannot

be done through the parameter µ, since there is no change **for** the probabilities

pi of those who choose or perhaps are **for**ced not to attend.

A rather aggressive way to change this, is to multiply these probabilities with

a number close to 1. This way in every iteration the attendance probability **for**

all those that do not attend will slightly increase and finally they will ‘decide’ to

attend the bar. That leads to the following equations **for** the partial algorithm

with taxing:

⎧

0,

⎪⎨

where µppi(k) − µtax(N(k) − N )xi(k) < 0

pi(k + 1) = 1,

where µppi(k) − µtax(N(k) − N )xi(k) > 1

⎪⎩

pi(k) − µppi(k) − µtax(N(k) − N )xi(k), otherwise

(3.2.1)

µp and µtax are defined as following:

1, xi(k) = 1

µp =

(3.2.2)

cp, xi(k) = 0

where i = 1 . . . M and c1 a constant which affects the degree pi is changing **for**

those agents that do not attend (xi(k) = 0)

µ, N(k) ≤ N

µtax =

(3.2.3)

ctaxµ, N(k) > N

where c2 > 1 is a constant that defines the degree pi is changing **for** selfish agents

who insist to attend even if the bar was crowded the previous time.

**The** behaviour of this algorithm is summarised in the 3rd and 4th plots of

figure 3.2. Simulations showed that in a reasonable taxed system (ctax ≥ 3),

large values of mp result in an underutilised bar (Table:3.2. An interpretation of

this is that, although people are encouraged to attend, taxation prevents them

3.2. TAXING/PAYOFFS ALGORITHMS 25

from doing so. Of course as someone would expect minimal taxation results in

an overcrowded bar. Another interesting feature of this algorithm is, its almost

rapid convergence, even when it is compared with the original.

ctax 6 8 6 1 6

cp 1 1.01 4 4 n/a (full info)

average 58.929000 59.682000 49.002333 70.489000 58.165750

std 1.382448 2.200574 0.437992 4.642286 4.851578

Table 3.2: Simulation results **for** various values of ctax, cp.

For the full in**for**mation tax algorithm there is no need of the parameter mp:

⎧

⎪⎨ 0, pi(k) − µtax(N(k) − N ) < 0

pi(k + 1) = 1,

⎪⎩

pi(k) − µtax(N(k) − N ),

pi(k) − µtax(N(k) − N ) > 1

otherwise

(3.2.4)

**The** histograms of the attendances (figure 3.1) **for** partial and full in**for**mation

taxing algorithms have much in common with those of the original algorithms

and very few differences. Both full in**for**mation algorithms seem to follow the

normal distribution, although in the partial in**for**mation taxing algorithm, value

59 is dominating.

frequency

0 500 1000 1500 2000 2500 3000

Histogram of attendances in modified tax partial info

10 20 30 40

attendances

50 60 70

frequency

0 50 100 150 200 250

Histogram of attendances in tax full info

40 45 50 55

attendances

60 65 70

Figure 3.1: histograms **for** partial and full info taxing algorithms.

In [10] the solutions and convergence properties **for** the original algorithms

are examined. After defining the equilibrium points, they are used to derive a

set of deterministic ODE’s. **The** results concerning the nature of p(k) in steady

state, are used in the analysis of the convergence behaviour and the stability

properties of those ODE’s. Although the outputs of the simulations of the original

algorithms can be reproduced and proved analytically, there are many reasons

**for** making this procedure extremely difficult **for** the other variations.

From the last plot in figure 3.2 there is an indication that probabilities might

converge to a stationary state around 0.6. Although full in**for**mation algorithms

are simpler cases, the adaptive behaviour of µtax(k) is a source of complexity.

26 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

attendance

attendance

attendance

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

probabities **for** each agent

probabities **for** each agent

probabities **for** each agent

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

Figure 3.2: **The** overall attendance and the probabilities **for** each of the M agents

**for** ’partial’ in**for**mation tax, modified tax algorithms and the full in**for**mation tax

algorithm.

3.2. TAXING/PAYOFFS ALGORITHMS 27

In the following a set of equations that describe the stationary state behaviour

of the probabilities will be derived pi(k).

**The** number of agents attending at time k is N(k) = M i=1 xi(k) and xi(k) are

independent Bernoulli trials given by:

1, pi(k)

xi(k) =

(3.2.5)

0, 1 − pi(k)

In the stationary state we have:

pi(k + 1) = pi(k) ∀i ∈ N

Taking the expectation values results to:

E(pi(k + 1)) = E(pi(k)) ⇒ E(pi(k + 1)) − E(pi(k)) = 0 ⇒

E(µtax(N(k) − N )) = 0

N(k) is the number of agents attending at time k is equal to M i=1 xi(k), where

xi(k) are independent Bernoulli trials.

**The** expectation value is calculated using the following **for**mula:

E(XY ) =

fx,y(x, y)

where X,Y random variables and fx,y(x, y) joined mass function. In this case

Y = µtax and X = N(k) − N . By definition, Y = µtax is a function of N(k). So,

Y = g(X) and fx,y(x, y) = fx(x, g(x)) = fx(x). Hence:

E(µtax(N(k)) · (N(k) − N )) =

(N(k) − N ) · µtax(N(k)) · fN(k)(N(k)) ⇒

cµ

N(k)>N

N(k)−N

(N(k) − N ) · fN(k)(N(k)) + µ

cµ

N(k)>N

µ

N(k)≤N

x

N(k)≤N

N(k) · fN(k)(N(k)) − cµN

(N(k) − N ) · fN(k)(N(k)) ⇒

N(k)>N

N(k) · fN(k)(N(k)) − µN

N(k)≤N

fN(k)(N(k))+

fN(k)(N(k)) (3.2.6)

Kolmogorov-Smirnov test indicated that z = N(k) might follow Poisson distribution

f(z) = λz

z! e−λ , with λ = E(z). Hence:

E(f(z)) =

zf(z) =

zf(z) +

zf(z) = λ ⇒

z

z≤N

z>N

28 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

zf(z) = λ −

zf(z) (3.2.7)

z≤N

and:

f(z) = 1 ⇒

f(z) +

f(z) = 1 ⇒

f(z) = 1 −

f(z) (3.2.8)

z

z≤N

zN

z>N

z>N

Since µ = 0, (3.2.9) becomes:

zf(z)(c − 1) − N

f(z)(c − 1) = N − λ (3.2.10)

z>N

z>N

Since f(z) follows the Poisson distribution and after replacing z with N(k) and:

and

we get the final equation:

A(λ, M) =

B(λ, M) =

M

z>N

M

z>N

z>N

z≤N

λ N(k)

N(k)! e−λ

N λN(k)

N(k)! e−λ

z>N

z>N

B(λ, M)(c − 1) − N A(λ, M)(c − 1) = N − λ (3.2.11)

After solving equation (3.2.11), a λ should be found, which **for** M = 100

and c = 8 should be equal or close to the average of attendances. With a

numerical approximation of λ available, the values of pi in equilibrium point

could be determined following a very simple procedure:

M

M

λ = E(N(k) = E( xj(k)) = E( xj(k)) =

j=1

j=1

M

E(xj(k))

Since xi(k) are Bernoulli trials with θ = pi(k) then E(xi(k)) = pi(k). Hence:

λ =

j=1

M

pj(k) (3.2.12)

j=1

It must be noted that c = 1 in equation (3.2.11) leads to M

j=1 pj = N . So

**for** this value of c the original result, mentioned in [10] was recovered.

3.2. TAXING/PAYOFFS ALGORITHMS 29

3.2.2 Fairness and Efficiency

Like ‘partial in**for**mation with taxing’ algorithm also here two groups of agents

are created. Again 41 of them hardly ever attend and 59 almost always and **for**

those who do attend the profits are very close to 1 while **for** those who do not,

profits are close to 0. In the modified version, although not attending agents are

encouraged to attend with a slight increase of the probability pi the results are

not significantly different. This time 40 stay at home almost always and 60 go

to the bar. A larger pi will lead to more agents attending the bar, but there is

also the potential danger of overcrowding. **The** second algorithm has a slightly

better behaviour since most agents have a marginal raise in the profits and the

differences in std are of no significance (Table: 3.3). Although these algorithms

std mean

partial info with taxing 0.4748791 0.57201

modified partial info with taxing 0.4863357 0.5930867

full info with taxing 0.04937645 0.1766167

Table 3.3: standard deviation and mean of payoff probabilities **for** taxing algorithms.

seem to be as unfair as the original, there is an indication that they might be

slightly more efficient. **The** average value of payoff probabilities are larger in the

taxing variation.

But the suprise comes from the ‘full in**for**mation with taxing’ algorithm. As

it can been seen in figure 3.3 one group has 99 agents with probabilities of attendance

varying from 0.46 to 0.76 and only one agent never goes to the bar. Also

compared to the original full in**for**mation algorithm, this one is more efficient

from the agents point of view, since the payoff probability **for** those who attend

varies from 0.08 to 0.27 and has an average of 0.1766167. On the contrary in the

original full in**for**mation, the payoff probability varies from −0.05 to 0.04 with an

average of −0.005 (figure: 2.7, table: 2.4). It can be said that the improvement

is quite significant.

As expected, the first two algorithms are better than the third in terms of

**systems** efficiency, although are worse when compared with original ones. That

happens because of the relatively high distribution. **The** two first have almost

identical behaviour, although the effects of the slight increase of pi in the 2nd algorithm,

are visible in (Figure: 3.4). **The** behaviour of the third taxing algorithm

is almost identical with the behaviour of the original.

30 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

frequency

0 10 20 30 40 50

frequency

0 10 20 30 40 50

frequency

0 2 4 6 8

partial in**for**mation with taxing

0.0 0.2 0.4 0.6 0.8 1.0

probability of attendance

modified partial in**for**mation with taxing

0.0 0.2 0.4 0.6 0.8 1.0

probability of attendance

full in**for**mation with taxing

0.0 0.2 0.4

probability of attendance

0.6 0.8

frequency

0 10 20 30 40 50

frequency

0 10 20 30

frequency

0 1 2 3 4 5 6 7

partial in**for**mation with taxing

0.0 0.2 0.4 0.6 0.8 1.0

payoff probability

modified partial in**for**mation with taxing

0.0 0.2 0.4 0.6 0.8 1.0

payoff probability

full in**for**mation with taxing

0.00 0.05 0.10 0.15 0.20 0.25

payoff probability

Figure 3.3: fairness and efficiency plots **for** taxing algorithms.

3.3. BUDGET ALGORITHMS 31

standard deviation

0 5 10 15

taxing algorithms

taxing partial

modified taxing partial

taxing full

0 500 1000 1500

iterations

2000 2500 3000

Figure 3.4: standard deviation off attendance **for** taxing algorithms.

3.3 Budget Algorithms

3.3.1 Overview

In this algorithm agents have a limited budget which does not allow them to

attend the bar every night. In another version (‘modified budget algorithm’) of

this algorithm they are **for**ced to stay at home after attending the bar several

consecutive nights. This could be denoted as following:

∀ i ∈ [1, M], k ∈ [1, n], if

b

xi(k − j) = b ⇒

j=0

xi(k + 1) = · · · = xi(k + w) = 0 (3.3.1)

where b ∈ N is the constant that determines how many nights i-agent can attend

the bar consecutively and w ∈ N the maximum nights an agent must stay indoors

after having attended the bar **for** b nights

**The** algorithm settles down after almost 2000 iterations, but it is very CPU

intensive. In table 3.4 there are some results as well as the time needed **for** them.

Although the so far algoririthms have given results **for** 2 · 10 8 iterations in almost

3 days, this one **for** 2˙10 6 after 2 weeks has not given any result yet. **The** reason

of this increase in CPU power is that instead of using an array of attendances in

terms of 1 and 0 which is is initialised in every iteration and its size is equal to the

number of the agents, it uses a matrix which contains the attendances **for** every

iteration. That is a matrix with number of agents×iterations elements. Despite

the fact that this algorithm is the most fair and efficient of all (see subsection:

3.3.2), it might not be suitable **for** networks of hundreds of nodes, or in cases of

large numbers, because in every iteration the columns of this matrix are scanned

**for** sequences of 1’s. **The** **next** element or sequence of elements in the ‘modified

budget’ is set to 0 reflecting the inability of the agents to attend the bar.

As the ratio of budget to ‘nights staying at home’ reaches 1, the bar becomes

underutilised. It is as if the town is hit by recession: when people experience

32 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

N /M 60/100 60/100 60/100

iterations 5000 4 · 10 4 2 · 10 6

µ 0.01 0.01 0.001

average 59.133400 58.161900 n/a yet

std 0.999281 1.673139 n/a yet

time 6 sec 5 min 2 weeks...

Table 3.4: Behaviour of the ‘modified budget’ algorithm **for** budget=10 and

staying at home=6.

financial difficulties they do not attend bars. For values of this ratio much less

than 1 the algorithm settles down almost instantly. If the budget is large the

results are similar to the original ‘partial in**for**mation’ algorithm (Table: 3.5 and

Figure: 3.6).

frequency

0 200 400 600

budget 10 100 10 10 10

waiting 0 0 3 10 6

average 59.385200 59.287200 59.292000 48.981200 58.165750

std 2.916301 1.132950 1.861538 7.423848 4.851578

Table 3.5: Simulation results **for** budget algorithms.

Histogram of attendances in budget with b=10

35 40 45 50

attendances

55 60 65 70

frequency

0 200 400 600 800 1000

Histogram of attendances in modified budget with b=10,w=6

40 50

attendances

60 70

Figure 3.5: histograms **for** budget algorithms (b = 10, w = 0/6).

Although budget algorithms are a variation of the partial in**for**mation algorithm,

their histograms (figure: 3.5) are quite different. Especially the first which has a

wider distribution than the original one.

3.3. BUDGET ALGORITHMS 33

probabities **for** each agent

probabities **for** each agent

probabities **for** each agent

1.0

0.8

0.6

0.4

0.2

0.0

0 1000 2000 3000 4000 5000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0

1.0

1000 2000

time (iterations)

3000 4000

0.8

0.6

0.4

0.2

0.0

0 1000 2000 3000 4000 5000

time (iterations)

attendance

attendance

attendance

100

80

60

40

20

0

0 1000 2000 3000 4000 5000

time (iterations)

100

80

60

40

20

0

0

100

1000 2000

time (iterations)

3000 4000

80

60

40

20

0

0 1000 2000 3000 4000 5000

time (iterations)

Figure 3.6: **The** overall attendance and the probabilities **for** each of the M

agents **for** budget (budget= 10) and modified budget (budget= 10, wait=3, 6)

algorithms.

34 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

3.3.2 Fairness and Efficiency

In the first variation of the ‘budget algorithms’ the **for**mation of the two groups

persists, although it is clear that this is the fairest partial in**for**mation algorithm

considered so far, since only 34 agents decided to stay at home. **The** modified

version of this algorithm improves it by resolving most of fairness and efficiency

issues. Indeed, the more days agents have to stay at home in order to save

resources, more people have thew opportunity to attend. **The** number of the

always no attending agents goes from 23 in the 2nd example, to 6 in the 3rd one.

**The** improved version is also more efficient **for** the agents (Figure: 3.8). For

the first time, an algorithm can become fairer without becoming less efficient and

vice versa. As days at home increase, standard deviation diminishes and payoff

probability is raising (Table: 3.6). Like it was expected, **for**cing agents to stay

at home after systematic visits, gives a chance **for** those that would have never

thought to attend it and increases everybody’s profits. **The** only setback seems

to be that, this increase of profits is made in expense of the bar management,

since the average of attendance is around 58.

days waiting std mean

0 0.0953156 0.132056

3 0.232645 0.42336

6 0.1286092 0.471414

Table 3.6: standard deviation and mean of payoff probabilities **for** budget algorithms.

standard deviation

0 2 4 6 8 10

budget algorithms

0 days at home

3 days at home

6 days at home

0 1000 2000 3000 4000 5000

iterations

Figure 3.7: standard deviation off attendance **for** budget algorithms.

**The** last two algorithms are more efficient when the interests of the bar are

considered. **The**y have almost identical behaviour, although in figure: 3.7 it can

be seen that the algorithm behaves slightly better **for** smaller values of the ‘staying

at home’ constant. **The** std of the attendance **for** the original budget algorithm,

seems to be stable, with no indication of improvement. **The** only problem is that

3.3. BUDGET ALGORITHMS 35

frequency

0 10 20 30 40 50

frequency

0 10 20 30 40

frequency

0 10 20 30 40

budget algorithm

0.0 0.2 0.4 0.6 0.8

probability of attendance

budget algorithm with 3 days at home

0.0 0.2 0.4

probability of attendance

0.6 0.8

budget algorithm with 6 days at home

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

probability of attendance

frequency

0 5 10 15

frequency

0 5 10 15 20 25 30 35

frequency

0 10 20 30

budget algorithm

0.00 0.05 0.10

payoff probability

0.15 0.20

budget algorithm with 3 days at home

0.0 0.1 0.2 0.3

payoff probability

0.4 0.5 0.6

budget algorithm with 6 days at home

0.0 0.1 0.2 0.3 0.4 0.5

payoff probability

Figure 3.8: fairness and efficiency plots **for** budget algorithms.

although the fluctuations in standard deviations are lower when agents stay many

days at home, the bar is underutilised. Considering system’s fairness and based

on the histograms of figure:3.5 the first case, where w = 0 is the fairest, since the

distribution of the attendances is wider.

36 CHAPTER 3. ANALYSIS AND EXTENSION OF THE STOCHASTIC ALGORITHM

Chapter 4

Case Study: Multiple **Bar**s in

Santa Fe

What if the citizens of **El** **Farol** had a chance to choose from a pool of more than

one bar? Would the same algorithms behave differently? In this chapter, the case

of three bars is studied. **The** main intention is to see, if the extremely efficient

‘partial in**for**mation’ algorithm can become more fair when three bars instead of

one are available. Also this problem tends to be more realistic, since in most cases

more than one choices exist. **The** rule used in the decision process by each agent,

is based on a rather simplistic assumption that everybody prefers going out to

staying at home. **The** decision is similar to the process described in subsection

2.2.3. Only in this case, instead of using a biased coin to decide whether to attend

the bar or not, the agent uses three distinct and slightly biased coins. Each coin

corresponds to a bar, **for** example, if the first coin shows he should not attend,

he tosses the second, if it shows not to attend again, he tosses the third and if it

shows not to attend he stays at home.

Instead of using only one fixed sequence of the three bars, 3! were used. At

each iteration of the algorithm, after the probabilities of attendance have been

calculated, a sequence of bars is chosen randomly.

Simulations showed similar results to the one bar version. In order to achieve

consistency the 6/10 ration is preserved and again µ is set to be 0.01. Santa Fe is

now consisted of M = 300 citizens and has three bars, each one having a capacity

of N = 60 **for** each one of them. Overall Nall = 180 which is the 60% of 300. As

it is seen in table 4 and in figures 4.1 and 4.2 the behaviour of the algorithm is

almost identical.

37

38 CHAPTER 4. CASE STUDY: MULTIPLE BARS IN SANTA FE

attendance **for** **Bar** 1

attendance **for** **Bar** 2

attendance **for** **Bar** 3

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

100

80

60

40

20

0

0 500 1000 1500 2000 2500 3000

time (iterations)

probabities **for** each agent

probabities **for** each agent

probabities **for** each agent

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

1.0

0.8

0.6

0.4

0.2

0.0

0 500 1000 1500 2000 2500 3000

time (iterations)

Figure 4.1: **The** overall attendance and the probabilities **for** each of the M agents

**for** each bar.

overall attendance

300

250

200

150

100

50

0

0 500 1000 1500 2000 2500 3000

time (iterations)

Figure 4.2: **The** cumulative attendance **for** all three bars.

µ = 0.01 mean std

All **Bar**s 177.689333 3.052926

**Bar** 1 59.190000 2.360598

**Bar** 2 59.201667 2.207228

**Bar** 3 59.297667 2.524133

Table 4.1: mean and std **for** all bars.

**The** shared characteristics with the original algorithms continue when proceeding

to the fairness and efficiency analysis. Indeed, figure 4.3 indicates that

there are no agents that would want to attend more than one bar regularly. Each

bar has its set of very devoted customers, despite the randomised decision process.

This plots (barplots) are used in [17] as a token of unfairness. This unfairness is

even more clear in payoff probability figures (fig:4.4), where it is seen that 120

agents have payoff probability close to 0, while 179 are very close to 0.8. It seems

that even in a town with three bars, agents insist on behaving selfishly, when

using a partial in**for**mation based algorithm.

39

40 CHAPTER 4. CASE STUDY: MULTIPLE BARS IN SANTA FE

attendance probability

attendance probability

attendance probability

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

**Bar** 1

agents

**Bar** 2

agents

**Bar** 3

agents

Figure 4.3: Agent attendances **for** each bar

frequency

0 50 100 150 200

frequency

0 50 100 150 200

Agent payoffs **for** **Bar** 1

0.0 0.2 0.4

payoff probability

0.6 0.8

Agent payoffs **for** **Bar** 3

0.0 0.2 0.4

payoff probability

0.6 0.8

frequency

0 50 100 150 200

frequency

0 20 40 60 80

Agent payoffs **for** **Bar** 2

0.0 0.2 0.4

payoff probability

0.6 0.8

Agent payoffs **for** all bars

0.0 0.2 0.4

payoff probability

0.6 0.8

Figure 4.4: payoff probabilities **for** each bar and cumulative payoff probability

**for** the 3-bar version

41

42 CHAPTER 4. CASE STUDY: MULTIPLE BARS IN SANTA FE

Chapter 5

Conclusions

As it was mentioned in the first chapter the objective of this dissertation was

to analyse the **El** **Farol** **Bar** **Problem**, view the differences between various approaches

and try to expand some of them. It is a typical complex adaptive system,

in which agents interact with each other competing **for** the same resource.

**The** first three solutions insist on Arthur’s original concept of predictors with

minor variation. A closer look reveals, that the only difference they have is

the definition of predictors. Arthur uses a close set of predictors, while Challet

et all [2] introduce the use of binary strategies in an attempt to simplify the

problem and Fogel et all [3] use evolutionary learning algorithms **for** a more

precise, although slightly more complicated, definition of the predictors.

Arthur’s belief that any solution should require agents that pursue different

strategies[18] is abandoned in the stochastic adaptive solution, proposed by Bell

et all in [10]. In partial in**for**mation algorithms and their variations, there is

no need **for** the agents to make prediction about the attendance of the **Bar**.

**The**y make their decisions based only on their own previous experience. In the

full in**for**mation algorithm, agents do know the full record of attendances. **The**

behaviour of this algorithm is similar to the first three, although it does not

require the use of predictors

Partial in**for**mation algorithms are more efficient and always converge with

a very low standard deviation. Un**for**tunately this happens due to their unfair

nature, which resembles the ‘greedy’ concept behind this algorithm. Agents who

take decisions based only on their previous experiences act extremely selfishly

when competing **for** the same resource. Fairness and efficiency could be measured

with the use of utility payoffs. **The** results showed that in partial in**for**mation algorithms

two groups of agents were **for**med. Those who almost always attended,

having an average payoff very close to 1.0 and those who almost never attended,

who had a payoff near 0. It is clear that these behaviour is very unfair, especially

when compared to full in**for**mation results, but it is also efficient since the average

utility payoff **for** all agents was higher than the one of the full in**for**mation

algorithm.

43

44 CHAPTER 5. CONCLUSIONS

Another negative aspect of these algorithms, is that they converge to N − 1,

leaving always **El** **Farol** **Bar** slightly underutilised. But even in that case the losses

**for** the bar management are less than the full in**for**mation algorithm because of

its almost zero standrad deviation. Partial in**for**mation algorithms after their

convergence have almost zero std and mean 59 while full in**for**mation has a mean

of 60 but also std of 6.5.

This algorithm could be divided in three parts. First comes the probability of

attendance, which is affected by the µ parameter and determines the attendances.

In order to introduce fairness and efficiency, each single one of those parts was

altered.

Directly changing the probabilities of attendance which exceeded a threshold,

did not give better results. In this variation of the stochastic adaptive algorithms

after several iterations another value of probability was assigned, if they were

larger than an upper bound, or lower than a low bound. Despite the fact that this

algorithm had countless variations, none of the simulations showed significantly

good results. This aggressive way of changing the probabilities created more

unfairness and less efficiency and that is why the results are not included in the

report, although the code is included in the Appendix.

**The** results were much better when taxation was implemented. Although

the partial in**for**mation algorithms did not become more fair with the use of

an adaptive µ, they become more effective. Full in**for**mation algorithms were

more fair but also significantly more effective. But probably the best results

came when agents were **for**ced to stay indoors after a succession of attendances.

Partial in**for**mation with budget algorithm gave almost perfectly fair results and

slightly less efficient than the original one. Although the results of the partial

in**for**mation with budget were more than acceptable, this algorithm is very CPU

intensive and demanding. Also the use of a budget meant that the system was

no more a decentralised one, where agents made their own decisions. Although

agents still do not need access to full record, a central mechanism which will

ensure they do not spent their budget and that they will stay indoors as many

days as needed, is necessary.

Finally, the problem was re**for**mulated, so it could include three bars. Each

agent was randomly assigned one sequence of bars, which changed in every iteration

of the algorithm. Instead of deciding whether to go to **El** **Farol** or stay at

home, he had to decide whether to go to the first bar of the sequence, second,

third or stay at home. Despite the randomised sequence, the results were almost

similar to the previous results.

In all cases, it was shown that the original algorithms could be extended in

many ways. **The**re is much work left to be done in this field. Papers [10], [17], [18]

and [15] have mentioned only the first three variations of the stochastic adaptive

learning algorithms (partial, full, partial with signs) and examined their convergence

properties. **The** most important is that they could be used accordingly in

order to fulfil a very versatile set of tasks. **The**y could be combined or used to

calculate attendances in a network of many nodes (bars) which accepts dynamically

evolving population. **The** C code used to generate all the results, is modular

enough so even more variations can be included. Also the mathematical **for**mulation

of the algorithms make it possible to study their convergence properties,

following the steps defined in the original paper[10] and further extended in this

dissertation **for** the tax algorithms.

45

46 CHAPTER 5. CONCLUSIONS

Appendix A

C Code **for** the **El** **Farol** **Bar**

**Problem**

/***************************************************************************/

/* Program that solves **El** **Farol** **Bar** problem based in paper: */

/* Coordination Failure as a Source of Congestion in In**for**mation Networks */

/* Ann M. Bell, William A. Sethares, James A. Bucklew */

/* IEEE Trans. Signal Processing, 2003 */

/***************************************************************************/

/***************************************************************************/

/* This source code is a part of MSc dissertation: */

/* **The** **El** **Farol** **Bar** **Problem** **for** **next** **generation** **systems** */

/* by Athanasios Papakonstantinou */

/***************************************************************************/

/* In order to compile and run this program, type the following from

the directory the source is saved:

gcc -Wall -lgsl -lgslcblas -lm demo3.c support.c modules.c debug.c */

/* Bugs and known issues:

i. Un**for**tunately the GSL C library is needed **for** the random generators.

You can install it from yours distribution package management system. This

remains to be fixed in the future, since this code intended to show quick

results of many algorithms. It is No 1 priority, because I want the program

to be portable.

ii. To generate plots you must run the ef.sh script provided, and install the

plotutils package.

iii. Some algorithms can be very demanding. Especially budget algorithms. You

should not use more than 20000 iterations. This is more a feature than a bug

iv. This is not optimised code, the results **for** 2*10^8 iterations were calculated

from a variation of this code, which does not produce any files **for** plotting

v. **The**re is a total lack of substantial user’s interface. **The**re was no time **for**

47

48 APPENDIX A. C CODE FOR THE EL FAROL BAR PROBLEM

that, see Bugs and known issues bullet "i". This will be fixed in the future but

only if the code is uploaded in the internet

vi. No other bugs are known. This code was compiled with gcc 4.1.1 in a

Gentoo GNU/Linux box with 2.6.17-gentoo-r4 kernel and with gcc 3.4.6 in

a MandrakeLinux 10.1 box with 2.6.8.1-12mdksmp kernel. */

#include

#include

#include

#include

#include

#include "elfarol.h"

/*** *** prototypes section *** ***/

/* function to write integers in files, used **for** plotting the attendaces,

mean and running average of them */

void writei_mean (char[], int, int[]);

/* function to write floats in files, used **for** plotting probabilities */

void writed(char[], int,int, int, float[]);

/* function that is used in fairness plots! */

void write_fair(char[],int[], int, int);

/* module used in partial and full algorithms */

float 1eq5 (float, float, int, int);

/* module used in signed partial algorithm */

float eq7 (float, float, float, int, int);

/* module used in taxing algorithms */

float tax (float, float, float, int, int);

/* utility function - returns {-1 0 1} */

int utility (int, int,int);

/*** ***files section *** ***/

#define attendance "att"

#define propability "prop"

#define fairness "fair"

#define payoff "pay"

/*** *** constant variables *** ***/

//*** General Section ***

const int M=100; //available agents in the system

const int cp=60; //capacity of the bar (happy number)

const int nt=3000; //no of iterations

const float m=0.01; //the \mu parameter

//** Taxing Algorithms **

/* m1 is used **for** taxing selfish agents and m3 to encourage

those staying at home to attend the bar.*/

const float m1=8;

const float m3=1.01;

//** Critical Values **

/* up and down are the boundaries and d is used in the shuffle

variation */

const float up=0.9;

const float down=0.1;

const int d=2;

//** Budget Algorithms **

const int wait=10; //maximum waiting time

/*** ***Main program *** ***/

int main()

{

int i,k; //**for**

int *a_agent; //array of attendances

float *p_agent //array of M propabilities **for** nt iterations

int *god; //a_agent **for** all iterations

int *a_total; //attendance after each iteration

int *sgn; //array in partial sign algorithm

float *tax_agent; //array in tax algorithm

float *mu; //array full of \mu’s

int *u_total; //array of total payoffs after each iteration

int ans1,ans2; //Answers in interface

int j,sum,l; //used in budget algorithms

int max,max_out; //maximum attendances in budget

const gsl_rng_type *T; //random generator

gsl_rng *r; //random generator

/* create a generator chosen by the environment variable

GSL_RNG_TYPE */

gsl_rng_env_setup();

T = gsl_rng_default;

r = gsl_rng_alloc (T);

printf("How many agents should be viewed in probabilities plot?\n");

scanf("%d",&ans2);

/* Initializing variables i=0 and Initial Conditions IC */

p_agent = (float *)malloc((M*nt)*sizeof(float));

god = (int *)malloc((M*nt)*sizeof(int));

a_total = (int *)malloc((nt)*sizeof(int));

a_agent = (int *)malloc((M)*sizeof(int));

sgn = (int *)malloc((nt)*sizeof(int));

tax_agent = (float *)malloc((M)*sizeof(float));

mu = (float *)malloc((M)*sizeof(float));

u_total = (int *)malloc((M*nt)*sizeof(int));

**for**(i=0; i

50 APPENDIX A. C CODE FOR THE EL FAROL BAR PROBLEM

god[i]=9;

}

**for**(i=0; i

eak;

//Algorithm 2 full in**for**mation algorithm

case(2):

**for**(i=1; i

52 APPENDIX A. C CODE FOR THE EL FAROL BAR PROBLEM

if(p_agent[i*M+k]>gsl_ran_flat(r,0,1)) {a_agent[k]=1;god[i*M+k]=1;}

else {a_agent[k]=0;god[i*M+k]=0;}

}

a_total[i]=sumar(M,a_agent);

**for**(k=0; k

* In this bit, budget is implemented, god matrices columns are scanned **for**

sequences of 1’s which indicate continuous attendances. All unassigned

values are equal to 9, so that not any sum of 1’s is equal to budget but

only those that have the **next** after element unassigned (=9) This element

is set equal to 99 */

**for** (k=0;k

54 APPENDIX A. C CODE FOR THE EL FAROL BAR PROBLEM

/* Uncomment **for** up/down shuffle */

// if (p_agent[i*M+k]>up) {p_agent[i*M+k]=down;}

// if (p_agent[i*M+k]up)

{p_agent[i*M+k]=(1.0/d)*p_agent[i*M+k];}

if (p_agent[i*M+k]

}

/* This bit calculates the attendances. If the new god element is not 99

then the attendance is as usual calculated by the probability. If it is to 99,

it is so because it is after a sequence of attendances and this and the

following w-1 must be set to 0 */

**for**(k=0; kgsl_ran_flat(r,0,1)) {a_agent[k]=1;god[i*M+k]=1;}

else {a_agent[k]=0;god[i*M+k]=0;}

}

else if(god[i*M+k]==99) {god[i*M+k]=0; a_agent[k]=0;}

}

a_total[i]=sumar(M,a_agent);

**for**(k=0; k

56 APPENDIX A. C CODE FOR THE EL FAROL BAR PROBLEM

}

printf("5. Full info with taxing\n");

printf("6. Partial info with budget\n");

printf("7. Partial with critical values\n");

printf("8. Partial with budget, exotic!\n");

void writei_mean (char filename[], int array_len, int x[])

{

FILE *fptr;

int i;

float sum1; //sum1 is the trend, mean changes with considering every attendance

float sum2; //sum2 is the mean of all iterations attendances

sum1=0.0;

sum2=0.0;

fptr= fopen (filename, "w");

if (fptr==NULL) {

printf("Unable to open file,check directory permissions and try again\n");

exit(-1);

}

**for** (i=0; i

void write_fair(char filename[],int a[], int array_len1, int array_len2)

/* array_len1:M, array_len2:nt */

{

FILE *fptr;

int i,j;

float sum;

fptr= fopen (filename, "w");

if (fptr==NULL) {

printf("Unable to open file,check directory permissions and try again\n");

exit(-1);

}

**for** (i=0; i

58 APPENDIX A. C CODE FOR THE EL FAROL BAR PROBLEM

Bibliography

[1] Arthur W. B. Inductive reasoning and bounded rationality: **The** el farol

problem. Amer. Econ. Rev., 84:406–411, May 1994.

[2] Marsili M. Challet D. and Ottino G. Shedding light on el farol. Game

**The**ory and In**for**mation 0406002, f. EconWPA, June 2004. available at

http://ideas.repec.org/p/wpa/wuwpga/0406002.html.

[3] Chellapilla K Fogel D. B. and Angeline P. J. Inductive reasoning and

bounded rationality reconsidered. IEEE Transactions on Evolutionary Computation,

3(2), 1999.

[4] Strogatz S. H. Nonlinear Dynamics and Chaos. Westview Press, 2000.

[5] Goertzel B. From Complexity to Creativity, Computational Models of Evolutionary,

Autopoietic and Cognitive Dynamics. Plenum Press, 1997.

[6] **Bar**-Yam Y. Dynamics of complex **systems**. Westview Press, July 1993.

[7] Waldrop M. M. Complexity: **The** Emerging Science at the Edge of Order

and Chaos. Penguin Books Ltd, January 1994.

[8] Arthur W. B. Complexity and the economy. Science, 284:107–109, April

1999.

[9] Nie N. and Comaniciu C. Adaptive channel allocation spectrum etiquette

**for** cognitive radio networks, 2006.

[10] Sethares W. A. Bell A. M. and Bucklew J. A. Coordination failure as a

source of congestion in in**for**mation networks. IEEE Trans. Signal Processing,

51(3):875–885, March 2003.

[11] Garogalo M. **The** ’el farol bar problem’ in netlogo. Unpublished Draft.

[12] Osborne M. J. and Rubinstein A. A Course in Game **The**ory. MIT Press,

1994.

[13] Hargreaves-Heap S. P. and Varoufakis Y. Game **The**ory, A critical text.

Routledge, 2004.

59

60 BIBLIOGRAPHY

[14] Challet D. and Zhang Y.-C. Emergence of cooperation and organization in

an evolutionary game. Physica A, 246:407–418, 1997.

[15] Bell A. M. and Sethares W. A. **The** el farol problem and the internet:

Congestion and coordination failure. Computing in Economics and Finance

1999 812, Society **for** Computational Economics, March 1999. available at

http://ideas.repec.org/p/sce/scecf9/812.html.

[16] Greenwald A. Farago J. and Hall K. Fair and efficient solutions to the santa fe

bar problem. http://www.clsp.jhu.edu/~khall/pubs/santafe_02.pdf.

[17] Sivanadyan T. and Sethares W. A. A decentralized optimal solution to the

‘minority game’. preprint submitted to Physica A, February 2006. http:

//homepages.cae.wisc.edu/~thiagars/pdfs/MinorityGame.pdf.

[18] Bell A. M. and Sethares W. A. Avoiding global congestion using decentralized

adaptive agents. IEEE Transactions in Signal Processing, 49(11):2873–

2879, November 2001.