handout - Vrije Universiteit Amsterdam

Stochastic Operations Research

Lecture 1: Introduction & Poisson Process

(Chapter 1)

A.A.N. Ridder

Department EOR

**Vrije** **Universiteit** **Amsterdam**

Huispagina: http://personal.vu.nl/a.a.n.ridder/sor/default.htm

31 October 2012

c○ Ad Ridder (VU) SOR– Fall 2012 1 / 26

Many Books on Stochastic Modeling

To mention a few.

Edward P.C. Kao. An Introduction to in Stochastic Processes, Duxbury

Press 1997.

Samuel Karlin and Howard M. Taylor. A First Course in Stochastic

Processes, Academic Press 1975.

Vidyadhar G. Kulkarni. Modeling and Analysis of Stochastic Systems,

Chapman & Hall 2009 (2nd ed).

Sheldon M. Ross. Introduction to Probability Models, Academic Press

2007 (9th ed).

Ronald W. Wolff. Stochastic Modeling and the Theory of Queues,

Prentice Hall 1989.

c○ Ad Ridder (VU) SOR– Fall 2012 3 / 26

Organization

Topics

Text Book

Henk C. Tijms

A First Course in Stochastic Models

Wiley, 2003

Website

http://personal.vu.nl/a.a.n.ridder/sor/

Grading: written exam (18 December)

c○ Ad Ridder (VU) SOR– Fall 2012 2 / 26

1. Poisson Process (Chapter 1)

2. Renewal Theory (Chapter 2)

3. Discrete-Time Markov Chains (Chapter 3)

4. Continuous-Time Markov Chains (Chapter 4)

Emphasis on Markov chains.

c○ Ad Ridder (VU) SOR– Fall 2012 4 / 26

Prerequisite Courses

◮ Introduction to Probability (1st year)

◮ Linear Algebra (1st year)

◮ Simulation (2nd year)

c○ Ad Ridder (VU) SOR– Fall 2012 5 / 26

Stochastic Models Everywhere: Bus Arrivals

◮ Based on Poisson process;

◮ Waiting time paradox: arriving at a bus stop, on average you wait as long

as if the bus has just departed;

c○ Ad Ridder (VU) SOR– Fall 2012 7 / 26

Follow-up Courses

◮ Essay (period 3):

◮ Queues (chapter 5)

◮ Discrete-event simulation of queues (Java)

◮ SOR 2 (period 4):

◮ Discrete-time Markov decision processes (chapter 6)

◮ Semi-Markov decision processes (chapter 7)

c○ Ad Ridder (VU) SOR– Fall 2012 6 / 26

Stochastic Models Everywhere: Queues

c○ Ad Ridder (VU) SOR– Fall 2012 8 / 26

Stochastic Models Everywhere: Virus Infection

◮ Spread of malware through computer network;

◮ Spread of epidemic disease through people network.

c○ Ad Ridder (VU) SOR– Fall 2012 9 / 26

Stochastic Models Everywhere: WiFi

◮ Based on Markov chains;

◮ Randomly chosen time to send your “packet” to hotspot .

c○ Ad Ridder (VU) SOR– Fall 2012 11 / 26

Stochastic Models Everywhere: Google’s PageRank

◮ Based on Markov chains;

◮ Pages are ranked according their equilibrium (or stationary or limit)

probabilities;

c○ Ad Ridder (VU) SOR– Fall 2012 10 / 26

Stochastic Models Everywhere: Kruskal’s Count

◮ A card trick

◮ Based on coupling times of renewal processes;

c○ Ad Ridder (VU) SOR– Fall 2012 12 / 26

§1.1 Poisson Process

◮ Known from your probability courses;

◮ It models arrivals or events occurring at various (random) times;

◮ It is a specific counting process;

◮ Let N(t), t ≥ 0 be the random variable that counts the number of events

up to time t.

◮ The number of events N(t) − N(s) in an interval (s, t] is called an

increment.

c○ Ad Ridder (VU) SOR– Fall 2012 13 / 26

Properties of Counting Processes II

(E) For all t > 0, the number of events in (0, t] is Poisson distributed with

mean λt; i.e.,

(N(t) = n) = e −λt (λt) n

n!

(F) The interarrival (or interoccurrence) times X1, X2, . . . are IID random

variables with an exponential (λ) distribution.

c○ Ad Ridder (VU) SOR– Fall 2012 15 / 26

Properties of Counting Processes I

(A) {N(t)} has independent increments; i.e., let (s1, t1] and (s2, t2] be disjoint

time intervals, then

(N(t1) − N(s1) = n1; N(t2) − N(s2) = n2)

= (N(t1) − N(s1) = n1) (N(t2) − N(s2) = n2)

(B) {N(t)} has stationary increments; i.e.,

(N(t) − N(s) = n) = (N(t − s) = n)

(C) The probability of exactly one event in a small time interval is

(approximately) proportional to the length of the interval:

(D) Events occur one by one:

(N(t + h) − N(t) = 1) = λh + o(h) for h → 0.

(N(t + h) − N(t) ≥ 2) = o(h) for h → 0.

c○ Ad Ridder (VU) SOR– Fall 2012 14 / 26

Construction PP

Definition

A counting process {N(t)} with N(0) = 0 is a Poisson process with intensity λ

if one the following holds.

(i). Property (F) (this is done in the book; definition 1.1.1);

(ii). Properties (A),(B),(C),(D);

(iii). Properties (A),(E).

Theorem

(i), (ii), and (iii) are equivalent.

See Theorem 1.1.1 for a proof of (F) ⇒ (E).

c○ Ad Ridder (VU) SOR– Fall 2012 16 / 26

Event Epochs

◮ Sk = X1 + · · · + Xk is k-th event time;

◮ Sk has a Gamma(k, λ) distribution;

◮ N(t) = max{k : Sk ≤ t}, t ≥ 0;

◮ N(t) < k ⇔ Sk > t;

c○ Ad Ridder (VU) SOR– Fall 2012 17 / 26

§1.1.2 Merging and Splitting

Theorem 1.1.3(a)

The superposition of two independent Poisson processes with rates λ1 and

λ2 is a Poisson process with rate λ1 + λ2.

Theorem 1.1.3(b)

Thinning a Poisson process by assigning events a type 1 with probability p or

a type 2 with probability 1 − p, results in two independent Poisson processes

with rates λp and λ(1 − p), respectively.

c○ Ad Ridder (VU) SOR– Fall 2012 19 / 26

Memoryless Property PP

◮ Define SN(t) as the last event before or at time t; formally

SN(t) = max{Sk : Sk ≤ t}

◮ Let γt = SN(t)+1 − t the waiting time until the next event.

Theorem 1.1.2

γt has exponential distribution with rate λ.

c○ Ad Ridder (VU) SOR– Fall 2012 18 / 26

§1.1.4 PP and Uniform Distribution

Given n events in (0, t], they are distributed as if drawn from the uniform

distribution.

Theorem 1.1.5

Specifically:

(S1 ≤ x1, . . . , Sn ≤ xn | N(t) = n) = (U(1) ≤ x1, . . . , U(n) ≤ xn).

(S1 ≤ s|N(t) = 1) = s

, 0 ≤ s ≤ t.

t

c○ Ad Ridder (VU) SOR– Fall 2012 20 / 26

Generalizations of the PP

◮ §1.2 Compound Poisson Process

◮ §1.3 Non-Stattionary Poisson Process

c○ Ad Ridder (VU) SOR– Fall 2012 21 / 26

§1.3 Non-Stationary PP

Definition 1.3.1

The counting process {N(t)} is a non-stationary Poisson process with

intensity function λ(t), t ≥ 0, if

(a) N(0) = 0;

(b) {N(t)} has independent increments;

(c) (N(t + h) − N(t) ≥ 2) = o(h), (h → 0);

(d) (N(t + h) − N(t) = 1) = λ(t)h + o(h), (h → 0).

c○ Ad Ridder (VU) SOR– Fall 2012 23 / 26

§1.2 Compound PP

Definition 1.2.1

Let

◮ {N(t)} be a Poisson process;

◮ D1, D2, . . . be IID, independent of {N(t)};

Then

N(t)

X(t) = Di, t ≥ 0,

i=1

is called a compound Poisson process.

Note:

[X(t)] = λt[D]; ar[X(t)] = λt[D 2 ].

c○ Ad Ridder (VU) SOR– Fall 2012 22 / 26

Number of Events in Non-Stationary PP

Theorem 1.3.1

(N(t + s) − N(t) = k) = e −(M(t+s)−M(t))

k M(t + s) − M(t)

,

k!

where

M(x) =

x

0

λ(y) dy

is the expected number of events occurring in interval (0, x] by the

nonstationary Poison process.

c○ Ad Ridder (VU) SOR– Fall 2012 24 / 26

Thinning a PP

Lemma

◮ Let {N(t)} be a (stationary) PP with rate λ;

◮ Suppose that an event occurring at time s is assigned type 1 with

probability p(s) and type 2 with probability 1 − p(s);

◮ Assignments are made independently of the past;

◮ Denote N1(t) for the number of type 1 events upto time t;

◮ Similarly N2(t) for type 2 events.

Then {N1(t)} and {N2(t)} are independent non-stationary Poisson processes

with intensity functions λ1(t) = λp(t) and λ2(t) = λ(1 − p(t)), respectively.

c○ Ad Ridder (VU) SOR– Fall 2012 25 / 26

Exercises

Chapter 1 (pp 28 - 32):

1.4, 1.5, 1.6, 1.8, 1.9, 1.16, 1.20, 1.22, 1.23, 1.24, 1.25

c○ Ad Ridder (VU) SOR– Fall 2012 26 / 26