Dimension Reduction for Model-based Clustering via Mixtures of ...

Chapter 3MethodologyThere are several approaches to using mixture models in a clustering context which werediscussed in Chapter 2. McNicholas (2011) presents a review of work in model-basedclustering with a particular focus on two families of Gaussian mixture models, namelyMCLUST (Fraley and Raftery, 2002) and PGMM (McNicholas and Murphy, 2008).3.1 Dimension-reduction for Mixtures of MultivariateGaussian Distributions (GMMDR)Scrucca (2010) proposed a novel approach to model-based clustering, namely the dimensionreductionfor mixtures of multivariate Gaussian distributions (GMMDR). The main ideais to find a reduced subspace which captures most of the clustering structure in the data.By following the work of Li (1991, 2000) on sliced inverse regression (SIR) one can obtaininformation on the dimension reduction subspace from two sources:• the variation on group means;• the variation on group covariances (depending on the estimated mixture model).Classical procedures for reducing the dimensions in the data are principal componentsanalysis and factor analysis. These techniques lower dimensionality by forming linearcombinations of the variables. In terms of visualizing any potential clustering structure,neither method is particularly useful.The proposed method reduces dimensionality by identifying a set of linear combinationsof the original variables, ordered by importance via their associated eigenvalues,which capture most of the cluster structure in the data. Observations are then projected14