Data Mining Methods and Models

16 CHAPTER 1 DIMENSION REDUCTION METHODS
among the variables and contributes less to the PCA solution. Communalities that
are very low for a particular variable should be an indication to the analyst that
the particular variable might not participate in the PCA solution (i.e., might not be
a member of any of the principal components). Overall, large communality values
indicate that the principal components have successfully extracted a large proportion
of the variability in the original variables; small communality values show that there
is still much variation in the data set that has not been accounted for by the principal
components.
Communality values are calculated as the sum of squared component weights
for a given variable. We are trying to determine whether to retain component 4, the
“housing age” component. Thus, we calculate the commonality value for the variable
housing median age, using the component weights for this variable (hage-z) from
Table 1.2. Two communality values for housing median age are calculated, one for
retaining three components and the other for retaining four components.
� Communality (housing median age, three components):
(−0.429) 2 + (0.025) 2 + (−0.407) 2 = 0.350315
� Communality (housing median age, four components):
(−0.429) 2 + (0.025) 2 + (−0.407) 2 + (0.806) 2 = 0.999951
Communalities less than 0.5 can be considered to be too low, since this would
mean that the variable shares less than half of its variability in common with the
other variables. Now, suppose that for some reason we wanted or needed to keep
the variable housing median age as an active part of the analysis. Then, extracting
only three components would not be adequate, since housing median age shares only
35% of its variance with the other variables. If we wanted to keep this variable in the
analysis, we would need to extract the fourth component, which lifts the communality
for housing median age over the 50% threshold. This leads us to the statement of the
minimum communality criterion for component selection, which we alluded to earlier.
Minimum Communality Criterion
Suppose that it is required to keep a certain set of variables in the analysis. Then
enough components should be extracted so that the communalities for each of these
variables exceeds a certain threshold (e.g., 50%).
Hence, we are finally ready to decide how many components to retain. We have
decided to retain four components, for the following reasons:
� The eigenvalue criterion recommended three components but did not absolutely
reject the fourth component. Also, for small numbers of variables, this criterion
can underestimate the best number of components to extract.
� The proportion of variance explained criterion stated that we needed to use four
components if we wanted to account for that superb 96% of the variability.
Since our ultimate goal is to substitute these components for the original data