Dimension Reduction for Model-based Clustering via Mixtures of ...

kernel matrixM = M I Σ −1 M I + M II , (3.6)whereM I ≡ Σ B as before,G∑M II = π g (Σ g − ¯Σ)Σ −1 (Σ g − ¯Σ) ⊤ ,g=1G∑¯Σ = π g Σ gg=1is the pooled within-cluster covariance matrix.Thus the kernel matrix in (3.6) contains information on how both cluster means andcluster covariances vary. Now the optimization problem in (3.4) is given byargβmax β ⊤ M β subject to β ⊤ Σ β = I d , (3.7)and it is solved using the generalized eigen-decompositionMv i = l i Σv i , (3.8)where⎧⎨1 if i = j, andvi ⊤ Σv j =⎩0 otherwise,and l 1 ≥ l 2 ≥ . . . ≥ l d > 0.Definition 3.1 The tMMDR directions are the eigenvectors [v 1 , . . . , v d ] ≡ β whichform the basis of the dimension reduction subspace S(β) and constitute the solution to(3.7).Suppose S(β) is the subspace spanned by the tMMDR directions from (3.8) and µ g ,Σ g are the mean and covariance for cluster g. Then the projections of the parametersonto S(β) are given by β ⊤ T µ g and β ⊤ Σ g β.18