Nonparametric Bayesian Discrete Latent Variable Models for ...

B Mathematical Appendix
B.1 Dirichlet Distribution
In the following, we describe the Dirichlet distribution and give some of its properties
important for understanding the Dirichlet processes.
The gamma distribution with shape parameter α ≥ 0 and scale parameter β > 0 is
given as
G(θ | α, β) =
Let θi, i = 1, . . . , k be independent random variables with
1
Γ(α)β α θα−1 exp{−θ/β}. (B.1)
θi ∼ G(αi, 1). (B.2)
The Dirichlet distribution D(α1, . . . , αk) is defined as the distribution of (π1, . . . , πk),
where
k
πi = θi/ θj for i = 1, . . . , k. (B.3)
The density function is given as,
j=1
D(π1, . . . , πk | α1, . . . , αk) = Γ( k
j=1 αj)
k
j=1 Γ(αj)
k
j=1
π αj−1
j . (B.4)
Dirichlet distribution is conjugate to the multinomial distribution
A random variable taking two different values with probability π1 and π2 = 1 − π1 is
Bernoulli distributed:
π1, x = x1
p(x) =
(B.5)
π2, x = x2
The count of the occurrence of one of the events (e.g. x = x1) in a sequence of n
Bernoulli trials is generally represented with the Binomial distribution:
n
p(y | π1) = Bin(y | π1, n) = π
y
y n−y
1 (1 − π1)
The conjugate prior for the Binomial distribution is the Beta distribution:
Γ(α + β)
p(π1) = Beta(π1 | α, β) =
Γ(α)Γ(β) πα−1 1 (1 − π1) β−1 .
119