Modeling and Multivariate Methods - SAS

494 Performing Discriminant Analysis Chapter 20
Discriminating Groups
Discriminant Method
JMP offers three kinds of Discriminant Analysis. All three calculate distances as the Mahalanobis distance
from each point to each group’s multivariate mean. The difference in the methods is only in the covariance
matrix used in the computations.
Linear Discriminant Analysis
uses a common (within-) covariance matrix for all groups.
Quadratic Discriminant Analysis uses a separate covariance matrix for each group.
Quadratic discriminant suffers in small data sets because it does not have enough data to make nicely
invertible and stable covariance matrices. Regularized discriminant ameliorates these problems and still
allows for differences among groups.
Regularized Discriminant Analysis is a compromise between the linear and quadratic methods,
governed by two arguments. When you choose Regularized Discriminant Analysis, a dialog appears
allowing specification of these two parameters.
The first parameter (Lambda, Shrinkage to Common Covariance) specifies how to mix the individual
and group covariance matrices. For this parameter, 1 corresponds to Linear Discriminant Analysis and 0
corresponds to Quadratic Discriminant Analysis.
The second parameter (Gamma, Shrinkage to Diagonal) specifies whether to deflate the non-diagonal
elements, that is, the covariances across variables. If you choose 1, then the covariance matrix is forced to
be diagonal.
Therefore, assigning 0,0 to these parameters is identical to requesting quadratic discriminant analysis.
Similarly, a 1,0 assignment requests linear discriminant analysis. These cases, along with a Regularized
Discriminant Analysis example with l=0.4 and g=0.4 are shown in Figure 20.2.
Use Table 20.1 to help decide on the regularization.
Table 20.1 Regularized Discriminant Analysis
Use lower Lambda (Gamma) when:
Covariances are different
(Variables are correlated)
lots of data
small number of variables
Use higher Lambda (Gamma) when:
Covariances are the same
(Variables are uncorrelated)
little data
many variables