Basic Probability Theory Lecture 4 - SICS

Basic Probability Theory 

Lecture 4 

Lecturer: Ali Ghodsi 

Notes: Ian Marsh 

October 12, 2007 

1 Repetition from previous week 

Problem 17, page 56: 

How an inferior player with a superior strategy can gain an advantage. Boris 

is about to play a two-game chess match with an opponent, and wants to find 

the strategy that maximises his winning chances. Each game ends with either 

a win by one of the players, or a draw. If the score is tied at the end of the 

two games, the match goes into sudden-death mode, and the players continue 

to play until the first time one of them wins a game (and the match). Boris has 

two playing styles, timid and bold, and he can choose one of the two at will in 

each game, no matter what style he chose in previous games. With timid play, 

he draws with probability Pd > 0, and he loses with probability 1 − Pd. With 

bold play, he wins with probability Pw, and he loses with probability 1 − Pw. 

Boris will always play bold during sudden death, but may switch style between 

games 1 and 2. 

• Find the probability that Boris wins the match for each of the following 

strategies: 

1. Play bold in both games 1 and 2. 

2. Play timid in both games 1 and 2. 

3. Play timid whenever he is ahead in the score, and play bold otherwise. 

• Assume that Pw < 1/2, so Boris is the worse player, regardless of the 

playing style he adopts. Show that with the strategy in (3) above, and 

depending on the values of Pw and Pd, Boris may have a better than a 

50-50 chance to win the match. How do you explain this advantage? 

Solution, define Boris’ strategies: 

Bold he wins with probability Pw and loses with probability 1 − Pw 

Timid draws with probability Pd and loses with probability 1 − Pd 

1

Figure 1: Boris’s strategies 

P(Boris wins) = p 2 w + 2pw(1 − pw)pw 

The two separate terms correspond to the two outcomes we are interested 

in (win-win) and win-lose-win and lose-win-win (verify for yourselves which is 

which). With Pw = 7/16 and Pd = 7/8 the probability of P(Win) = 0.51. See also this weeks 

problems 

Problem 18, page 56 

Two players take turns removing a ball from a jar that initially contains m 

white and n black balls. The first player to remove a white ball wins. Develop 

a recursive formula that allows the convenient computation of the probability 

that the starting player wins. 

There are m = white balls and n = black balls, define the events: 

W = starting player wins 

I = starting player wins immediately 

We want, P(W) so using the total probability theorem: 

P(W) = P(I)P(W |I) + P(I c )P(W |I c ) 

= m n 

· 1 + 

m + n m + n · P(W |Ic ) 

2

W(m,n) is the probability that start wins given m white and n black balls. 

M(m,n) = m n 

+ W(m,n − 1) 

m + n m + n 

In recursive solutions we also need a stopping condition, W(m,0) = 1 

Problem 21, page 57 

The prisoner’s dilemma. Two out of three prisoners are to be released. One of 

the prisoners asks a guard to tell him the identity of a prisoner other than himself 

that will be released. The guard refuses with the following rationale: at your 

present state of knowledge, your probability of being released is 2/3, but after 

you know my answer, your probability of being released will become 1/2, since 

there will be two prisoners (including yourself) whose fate is unknown and exactly 

one of the two will be released. What is wrong with the guards reasoning? 

Ri = prisoner to be released 

3 prisoners of which you are one: 1 (you), 2, 3 An example of the wrong 

way to calculate the probability of release, i.e. not taking into account what 

the guard says. 

P(R1|R2) = P(R3|R2)P(R1|R2 ∩ R3) + P(R c 3 ∩ R2)P(R1|R2 ∩ R c 3) 

= 1/2(0) + 1/2(1) = 1/2 

The probability of saying that one is going to be released and actually being 

released is not the same. Using the table below helps solve the problem correctly. 

Alternatives Probability 

1, 2 (guard says 2) 1/3 

1, 3 (guard says 3) 1/3 

2, 3 (guard says 2) 1/6 

2, 3 (guard says 3) 1/6 

P(1 is to be released | guard says 2) = 

1/3 

= 2/3 

1/3 + 1/6 

3 

P(1 is to be released and guard says 2) 

P(guard says 2)

So, the probability of 1 being released is 2/3 despite what the guard says. 

2 Independence 

Events A and B are independent, iff (if and only if ) 

P(A|B) = P(A) 

One other way to see it is that the event B does not bring any extra information. 

This is equivalent to: 

iff A is independent of B then: 

P(A|B) = 

P(A ∩ B) 

P(B) 

P(A)P(B) = P(A ∩ B) 

which works even when P(B) = 0. Also it shows that the order is not important. 

Ali’s quote “the point is that is is distinct and non-interacting physical 

process.” 

Note: independence does not mean disjunctivity. The reverse is true, disjoint 

implies dependence. 

One other quote “Proportion of A to Ω is equal to the proportion of (A ∩ B) 

to B”. 

Example: P(A) > 0,P(B) > 0 P(A ∩ B) = 0 i.e. disjoint 

Example 1.19, page 34 

a) Are the events, Ai and Bj independent? 

Ai = {first roll is i} 

Bj = {second roll is j} 

P(Ai) = 1/4,P(Bi) = 1/4 

P(Ai ∩ Bj) = 1/16 

which is equal to P(Ai)P(Bj) 

So, the events are independent. 

b) 2 rolls of a 4 sided die, are the following events independent? 

4

(add epsfig) 

Ai = {first roll is a 1} 

Bi = {sum of the two rolls is 5} 

P(Ai) = 4/16 

P(Bi) = 4/16 

P(Ai ∩ Bi) = 1/16 

P(Ai)P(Bi) = 4/16 · 4/16 = 1/16 

P(A ∩ B) = P(A)P(B), therefore events A and B are independent. 

c) Are these independent? 

A = {maximum of the two rolls is 2} 

B = {minimum of the two rolls is 2} 

P(A ∩ B) = P(< 2,2 > −roll) = 1/16 

P(A) = 3/16 (1,1),(1,2)(2,1) 

P(B) = 5/16 

P(A)P(B) = 3 · 5/2 8 = 1/16 

Problem 28. The king’s sibling. The king has only one sibling. What is 

the probability that the sibling is male? Assume that every birth results in a 

boy with probability 1/2, independent of other births. Be careful to state any 

additional assumptions you have to make in order to arrive at an answer. 

Probability A king who has one sibling, parents getting a boy was/is 0.5 and 

independent. 

Prob({king having brotherhood} = 1/2 — king is a boy) = 1/2 

BB = parents had a boy, then another boy (BB, BG, GB, BB) 

OB = parents had at least one boy 

P(BB|OB) = (BB∩OB) 

P(OB) 

1/4 

= 3/4 = 1/3 

2.1 Conditional independence 

Commonly used in statistics. 

A and B are conditionally independent given C 

P(A|B ∩ C) = P(A|C) 

5

P(A|B ∩ C) = P(A∩B∩C) 

= P(C)P(B|C)P(A) 

P(B∩C) 

P(A ∩ B|C) = P(A∩B∩C) 

P(C) 

P(B|C)P(A|B ∩ C) 

using P(A|B ∩ C) = P(A|C) 

= P(C)P(B|C)P(A|B∩C) 

P(C) 

P(B|C)P(A|C) no divide by zero problems. 

Now, compare conditional independent with independence 


Example 1.20. Consider two independent fair coin tosses, in which all four possible 

outcomes are equally likely. Two coin tosses, H1 = 1st head, H2 = 2nd 

head, D = tosses have different values P(H1 ∩ H2) are they independent? 

= 1/4 (H, H) 

P(H1) = 1/2 and P(H2) = 1/2 

P(H1|D) = 1/2 

P(H2|D) = 1/2 

P(H1 ∩ H2|D) = 0 = P(H1|D)P(H2|D) 

H1 and H2 conditionally independent 

A and B are not conditionally independent given D. 


Example 1.21. There are two coins, a blue and a red one. We choose one of the 

two at random, each being chosen with probability 1/2, and proceed with two 

independent tosses. The coins are biased: with the blue coin, the probability 

of heads in any given toss is 0.99, whereas for the red coin it is 0.01. 

Red coin, blue coin. Pick one coin randomly 

P(R) = 1/2 

P(B) = 1/2 

Toss that coin twice 

H1 = first toss head H2 = second toss head 

P(Hi|B) = 0.99 P(Hi|R) = 0.01 

Find the independence and conditional independence of H1 and H2. 

6

P(H1 ∩ H2|R) = P(H1|R)P(H2|R) 

Lhs = (0.01) 2 

H1 and H2 are conditional independent given R and B. 

P(H1) = P(R)P(H1|R) + P(R c )P(H1|R c ) 

using the total probability theorem, divide the state space into red and blue 

1/2 · 0.01 + 0.99 ∗ 1/2 = 1/2 

H(H2) = 1/2 

P(H1) · P(H2) = 1/4 

P(H1 ∩ H2) = P(R)(P(H1 ∩ H2|R) + P(R c )P(H1 ∩ H2|R c ) 

= 1/2 · 0.01 2 + 1/2 · 0.99 2 

≈ 1/2 

conditional independence and independence do not imply each other. 

3 Homework problems 

In problem 17 we looked at different strategies for winning a chess game. The 

question is, how bad can a player be and yet still win the game. Motivate your 

choice of values and explain your answer in simple terms to a non-technical 

person (i.e. probability student). In other words reason your answer. 

Do 23, 24, 25, 26 from the problem section (pages 57-58) 

7

Basic Probability Theory Lecture 4 - SICS

Create successful ePaper yourself

Delete template?

Save as template?