Nonparametric Bayesian Discrete Latent Variable Models for ...

µ y Σy
ξ R 0 ν
w
normal
µ
−1
Σy k
normal
D
Wishart
x i
Λ
qk
normal N
1 1
3.4 Dirichlet Process Mixtures of Factor Analyzers
K
Q
σ 1
−1
y
σ 2
Ψ
D −1
inv gamma gamma inv gamma
β
1
inv gamma
hyperpriors
hyperparameters
mixture parameters
observations
Figure 3.16: Graphical representation of the layered structure of the hierarchical priors in the
mixtures of factor analyzers model. Variables are labelled below by the name of
their distribution, and the parameters of these distributions are given above. The
number of observations (N), number of mixture components (K) and the latent
factor dimension (Q) are denoted by the numerals in the lower left hand corner of
the large rectangles.
condition on the latent factor z which results in the likelihood:
p(x|z, cj, µ j, Λj, Ψ) = N (µ j + Λjz, Ψ) (3.76)
This function factorizes over the data dimensions. To make use of this factorization,
we can express the prior distribution on the rows of Λ as p(Λ j
d· ) ∼ N 0, Υj −1 , where
Λ j
d. denotes the dth row of the jth factor loading matrix and Υj is the diagonal matrix
which has ν1, . . . , νQ as its entries.
The likelihood function in eq. (3.76) is used to compute the posteriors of Λj, Ψ and
z. Once Λj and Ψ are updated conditioned on z, we can compute the covariance matrix
Σj = ΛjΛj T + Ψ and condition on the full covariance to update the means and the
indicators using eq. (3.68) with the hidden factors integrated out. The conditional
posteriors for these variables are of standard form and therefore can be sampled from
easily.
The conditional posterior of the indicator ci is obtained by combining the prior of
the components with the likelihood. The likelihood of the components that have data
(other than xi) associated with them is Gaussian with mean µ j and covariance ΛΛj T +Ψ.
The likelihood pertaining to the remaining infinitely many components is obtained by
integrating over the prior distribution of the parameters,
p(ci = ci ′ for all i = i′ |c−i, ξ, R, ν,β, w, α)
α
∝
n − 1 + α ×
p(xi|µ, Λ, Ψ)p(µ, Λ|ξ, R, ν)dµdΛ.
(3.77)
This integral is not analytically tractable, therefore we need to use an inference technique
55