Nonparametric Bayesian Discrete Latent Variable Models for ...

α
G o
G
θi
x
i
N
3.1 The Dirichlet Process
Figure 3.1: Graphical representation of a Dirichlet process mixture model. The data is assumed
to be generated from a distribution parameterized by θ. The distribution of the parameter
θ has a Dirichlet process prior with base distribution G0 and concentration
parameter α.
i. Gn converges almost surely as n → ∞ to a random discrete distribution G.
ii. G has DP (α, G0) distribution.
iii. The sequence {θi} n 1
is a sample from G.
Note that eq. (3.7) gives an expression for generating samples from G, which is infinite
dimensional, without having to represent it explicitly. The graphical representation of
the DPM model corresponding to this representation is depicted in Figure 3.2 .
The evolution of Gn with increasing number of samples is shown in Figure 3.3 for
a Pólya urn sequence with a Gaussian base distribution and α = 5. Note that the
contribution of G0 to the distribution of Gn gets smaller as the number of samples
increases, and it vanishes for large sample sizes.
The generalized Pólya urn scheme shows that the draws from a DP exhibit a clustering
property by the fact that a new sample has positive probability of being equal to one
of the previous samples, and that the more often a color is sampled, the more likely it
will be drawn again. Note that α determines the probability of choosing a new color.
For small values of α, Gn has only a few atoms whereas for large values, the atoms are
numerous, concentrating on the G0 distribution. This is illustrated in Figure 3.3 using
α = 1 and α = 100.
13