Dimension Reduction for Model-based Clustering via Mixtures of ...

tMMDR variables which provide no clustering information but require parameter estimation.Thus, the next step in the process of model-based clustering is to detect andremove these unnecessary variables.Scrucca (2010) used the subset selection method of Raftery and Dean (2006) toprune the subset of GMMDR variables. We will also use this approach to select themost appropriate tMMDR variables.Let s be a subset of q features from the original tMMDR variables Z, with dim(s) = qand q ≤ d. Let s ′ = {s \ i} ⊂ s be the set of dim = q − 1 which is obtained by excludingthe i-th feature from s. The comparison of the two subsets can be viewed as a modelcomparison problem and addressed by using the BIC difference which is given in Rafteryand Dean (2006) asBIC diff (Z i∈s ) = BIC clust (Z s ) − BIC not clust (Z s ) , (3.9)where BIC clust (Z s ) is the BIC value for the best model fitted using features in s andBIC not clust (Z s ) is the BIC value for no clustering. We can write the latter asBIC not clust (Z s ) = BIC clust (Z s ′) + BIC reg (Z i |Z s ′) ,where BIC clust (Z s ′) is the BIC value for the best model fitted using features in s ′ andBIC reg (Z i |Z s ′) is the BIC value for the regression of the i-th feature on the remaining(q − 1) features in s ′ . Since the tMMDR variables are orthogonal, the formula for BIC reg(Raftery and Dean, 2006) reduces to( ) RSSBIC reg (Z i |Z s ′) = −n log(2π) − n log − n − (q + 1) log(n) ,nwhere RSS is the residual sum of squares in the regression of Z i on Z s ′.Now, the space of all possible subsets has size dim(s) = q, where q = 1, . . . , d has 2 d −1elements and an exhaustive search would be unfeasible. To bypass this issue, Rafteryand Dean (2006) proposed a greedy search algorithm which finds a local optimum in themodel space. The algorithm comprises the following steps:20