Two-component mixtures of generalized linear mixed effects models ...

24 DB Hall and L Wang
The model form given by Equation (2.1) is quite flexible, allowing a wide variety of
random effects specifications. For example, a model with random cluster specific
intercepts might assume Z 1ij ¼ x T ij a þ y 1b 1i and Z 2ij ¼ z T ij b þ y 2b 2i . This implies independent
cluster effects in the two components. Correlated components can be induced
by assuming Z 1ij ¼ x T ij a þ y 1b 1i and Z 2ij ¼ z T ij b þ y 2b 2i þ y 3 b 1i , which leads to
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
corr(Z 1ij ; Z 2iij ) ¼ y 3 = y 2 2 þ y 2 3. The form given by Equation (2.1) simply generalizes
these cases to higher dimension, allowing random slope and intercept models and other
more general random effects structures. An alternative approach would have been to
allow correlated random effects b ~ 1i ; b ~ 2i , say, where b ~ ki appears only in the kth linear
predictor and cov( b ~ 1i ; b ~ 2i ) 6¼ 0. However, this more straight forward approach, which is
essentially a reparametrization of the model we focus on, is not as conducive to
estimation via the EM algorithm because it leads to a complete data likelihood, which
does not factor cleanly into terms for each component in the mixture. [In particular, the
second and third terms of formula (3.3) defined subsequently, which is the expected
complete data loglikelihood used in the EM algorithm, would share parameters
pertaining to corr( b ~ 1i ; b ~ 2i ).]
Note that in Equation (2.1) we have assumed canonical links, but this is not
necessary. In general, we allow known links g 1 and g 2 so that m 1ij ¼ g1 1(Z
1ij ) and
m 2ij ¼ g2 1(Z
2ij ). Furthermore, we assume that the mixing mechanisms for each observation
are independent, with probabilities p i ¼ (p i1 ; ...; p ini ) T , i ¼ 1; ...; C, each following
a regression model of the form g p (p i ) ¼ W i c, involving a known link function g p ,
unknown regression parameter c and n i s design matrix W i . Typically, g p will be
taken to be the logit link, but the probit, complementary log–log, or other link function
can be chosen here.
Let ~a ¼ (a T ; h T 1) T and b ~ ¼ (b T ; h T 2; h T 3) T , and denote the combined vector of model
parameters as d ¼ (~a T ; b ~ T ; c T ; s 1 ; s 2 ) T . The loglikelihood for d based on y is given by
‘(d; y) ¼ XC
i¼1
( ð )
Y n i
log f (y ij jb i ; d)f q (b i )db i
j¼1
where f (y ij jb i ; d) ¼ {p ij (c)}f 1 (y ij jb i ; ~a; s 1 ) {1 p ij (c)}f 2 (y ij jb i ; ~ b; s 2 ), f q ( ) denotes the
q-dimensional standard normal density function, and the integral is q-dimensional.
3 Fitting the two-component mixture model via the EM algorithm
The complications of parameter estimation in mixture models are simplified considerably
by applying the EM algorithm. Let u ij , i ¼ 1; ...; C, j ¼ 1; ...; n i denote the
component membership; u ij equals one if Y ij is drawn from distribution F 1 and equals
zero if Y ij is drawn from F 2 . Then the ‘complete’ data for the EM algorithm are (y, u, b).
Here, (u, b) play the role of missing data, where u ¼ (u 11 ; ; u CnC ) T . On the basis of