Nonparametric Bayesian Discrete Latent Variable Models for ...

increment process with the corresponding Lévy measure given by
3.1 The Dirichlet Process
dN(x) = x −1 e −x/β αdx, (3.10)
where α and β are positive scalars, t ∈ [0, 1] and Q(t) is a distribution function on [0, 1].
That is, defining the distribution of the random jump heights J1 ≥ J2, . . . to be
P (J1 ≤ x1) = exp N(x1)
P (Jj ≤ xj | Jj−1 = xj−1, . . . , J1 = x1) = exp (3.11)
N(xj) − N(xj−1)
and the random variables related to the jump times U1, U2, . . . to be i.i.d. Uniform(0, 1),
independent from J1, J2, . . . , the gamma process Zt is defined as
Zt =
∞
JjI Uj ∈ 0, Q(t) . (3.12)
j=1
See Ferguson and Klass (1972) and Ferguson (1973) for details.
Let Γ (1) ≥ Γ (2) ≥ . . . denote the jump heights of a gamma process with the scale
parameter β = 1. And let θk ∼ G0 i.i.d., independent also of the Γ (k). The random
measure defined as
has a DP(α, G0) distribution where
G =
Pk =
∞
k=1
Pkδθk (·) (3.13)
Γ (k)
∞ j=1 Γ . (3.14)
(j)
This definition explicitly shows the discreteness of the DP as it expresses G as an
infinite sum of atomic measures. The practical limitation of this construction is that we
need to know the value of the infinite sum to know the value of any of the weights. The
next section summarizes another infinite sum representation of the DP which does not
require evaluating the infinite sum, and allows approximating the DP by truncation.
3.1.4 Stick Breaking Construction
Sethuraman and Tiwari (1982) proposed a constructive definition of the DP based on a
sequence of i.i.d. random variables. The proof is provided by Sethuraman (1994). Let
vk be i.i.d. with a common distribution vk ∼ Beta(1, α). Define
πk = vk
j=1
k−1
(1 − vj), (3.15)
and let θk be independent of the vk and i.i.d. among themselves with common distribution
G0. The random probability measure G that puts weights πk at the degenerate
17