Dimension Reduction for Model-based Clustering via Mixtures of ...

More documents

Recommendations

Info

simple example. McLachlan and Peel (1998) introduced the idea of fitting mixtures ofmultivariate t-distributions to the data. This provided a more robust approach thanfitting normal mixture models, as outliers were given reduced weight in the calculationof parameters, i.e. the Mahalanobis distance term in the density (2.10).Recent work on model-based clustering using the multivariate t-distribution has beencontributed by McLachlan et al. (2007), Greselin and Ingrassia (2010a,b), Andrews andMcNicholas (2011a,b) and Andrews et al. (2011).Andrews and McNicholas (2012a) used the eigen-decomposition of the multivariatet-distribution covariance matrix to build a family of 20 mixtures which are called thetEIGEN family (as shown in Table 2.2). These take into account the same constraints asthe MCLUST family of models but also include constraints on the degrees of freedom.With the tEIGEN family, clustering is done via the expectation-conditional maximization(ECM) algorithm of Meng and Rubin (1993). In the ECM algorithm the M-stepis replaced by a number of conditional maximization steps which are more efficient froma computational standpoint. The missing data are represented by the same indicatorvariables z ig (2.5) as in the MCLUST case but with the addition of characteristic weightsu ig .Under the model-based clustering framework, the observations do not have knowngroup memberships. Thus the z ig must be initialized in order to run the ECM algorithmand there are several options for this. For MCLUST, Fraley and Raftery (2002) use amodel-based agglomerative hierarchical clustering procedure to obtain starting values.For parsimonious Gaussian mixture models (PGMM), McNicholas and Murphy (2008)use a number of random starting values on their most constrained modelΣ g = ΛΛ ⊤ + Ψ ,where Λ is a (p × q) matrix of parameters (factor loadings), typically with q ≪ p, andΨ is a diagonal noise matrix. The results are then picked from the iteration with thelargest BIC value to initialize for each cluster G. Andrews and McNicholas (2012a)suggest using the agglomerative hierarchical initialization method of MCLUST in orderto avoid algorithm failure when clustering with mixtures of multivariate t-distributions.10
Table 2.2: Nomenclature for models in the tEIGEN family: ‘C’ indicates that a constraintis imposed, ‘U’ indicates that a constraint is not imposed, ‘I’ indicates the identitymatrix of suitable dimension (Andrews and McNicholas, 2012a)Model λ g = λ D g = D A g = A ν g = ν Free covariance parametersCIIC C I I C 1 + 1CIIU C I I U 1 + GUIIC U I I C (G − 1) + 1UIIU U I I U (G − 1) + GCICC C I C C p + 1CICU C I C U p + GUICC U I C C p + (G − 1) + 1UICU U I C U p + (G − 1) + GCIUC C I U C Gp − (G − 1) + 1CIUU C I U U Gp − (G − 1) + GUIUC U I U C Gp + 1UIUU U I U U Gp + GCCCC C C C C [p(p + 1)/2] + 1CCCU C C C U [p(p + 1)/2] + GUCCC U C C C [p(p + 1)/2] + (G − 1) + 1UCCU U C C U [p(p + 1)/2] + (G − 1) + GCUCC C U C C G[p(p + 1)/2] − (G − 1)p + 1CUCU C U C U G[p(p + 1)/2] − (G − 1)p + GUUCC U U C C G[p(p + 1)/2] − (G − 1)(p − 1) + 1UUCU U U C U G[p(p + 1)/2] − (G − 1)(p − 1) + GCUUC C U U C G[p(p + 1)/2] − (G − 1) + 1CUUU C U U U G[p(p + 1)/2] − (G − 1) + GUUUC U U U C G[p(p + 1)/2] + 1UUUU U U U U G[p(p + 1)/2] + G11
Page 1: Dimension Reduction for Model-based
Page 5 and 6: List of Tables2.1 Nomenclature for
Page 7: Chapter 1IntroductionClustering alg
Page 10 and 11: 2.1 Gaussian Mixture ModelsFrom McL
Page 12 and 13: • λ g is a scalar which specifie
Page 14 and 15: In the M-step of the algorithm, the
Page 18: For the tEIGEN models, parameter es
Page 21 and 22: onto the reduced subspace and their
Page 23 and 24: to the others. Specifically, we nee
Page 25 and 26: Definition 3.2 The tMMDR variables,
Page 27 and 28: • the forward step evaluates the
Page 29 and 30: Chapter 4ApplicationsWe now focus o
Page 31 and 32: Figure 4.2: Pairs plot for the thre
Page 33 and 34: Table 4.1: Average ARI (with standa
Page 35 and 36: The known classification of the cof
Page 37 and 38: Table 4.5: Variable description for
Page 39 and 40: 4.2.3 Australian Crabs DataCampbell
Page 41 and 42: Figure 4.6: Plots of tMMDR directio
Page 43 and 44: those notes. The results may indica
Page 45 and 46: Table 4.16: A classification table
Page 47 and 48: Chapter 5ConclusionThis thesis intr
Page 49 and 50: Forina, M., C. Armanino, M. Castino
Page 51: Meng, X. L. and D. B. Rubin (1993).

Dimension Reduction for Model-based Clustering via Mixtures of ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?