Nonparametric Bayesian Discrete Latent Variable Models for ...

model can be written in the form of eq. (3.19):
3.3 Empirical Study on the Choice of the Base Distribution
xi | ci, Θ ∼ N (µ ci , S−1 ci )
ci | π ∼ Discrete(π1, . . . , πK)
(µ j, Sj) ∼ G0
π | α ∼ D(α/K, . . . , α/K).
(3.54)
We obtain the Dirichlet Process mixture of Gaussians (DPMoG) model by integrating
out the mixing proportions and taking the limit K → ∞, as discussed in Section 3.1.5.
Equivalently, we can define the model by starting with Gaussian distributed data
xi|θ ∼ N (xi|µ i, S −1
i ), (3.55)
with a random distribution of the model parameters (µ i, Si) ∼ G drawn from a DP,
G ∼ DP (α, G0).
We put an inverse gamma prior on the DP concentration parameter α,
α −1 ∼ G(1/2, 1/2), (3.56)
the choice of which is discussed in detail in Section 3.1.6. We need to specify the
base distribution G0 to complete the model. For DPMoG, G0 specifies the mean of
the joint distribution of µ and S. For this model, a conjugate base distribution exists
however it has the unappealing property of prior dependency between the mean and the
covariance. We proceed by giving the detailed model formulation for both conjugate
and conditionally conjugate cases.
Conjugate DPMoG
The natural choice of priors for the mean of the Gaussian is a Gaussian, and a Wishart
distribution for the precision (inverse-Wishart for the covariance). To accomplish conjugacy
of the joint prior distribution of the mean and the precision to the likelihood,
the distribution of the mean has to depend on the precision. The prior distribution of
the mean µ j is Gaussian conditioned on Sj:
and the prior distribution of Sj is Wishart:
(µ j|Sj, ξ, ρ) ∼ N ξ, (ρSj) −1 , (3.57)
(Sj|β, W ) ∼ W β, (βW ) −1 . (3.58)
The joint distribution of µ j and Sj is the Normal/Wishart distribution denoted as:
(µ j, Sj) ∼ N W(ξ, ρ, β, βW ), (3.59)
with ξ, ρ, β and W being hyperparameters common to all mixture components, expressing
the belief that the component parameters should be similar, centered around some
41