Data Mining Methods and Models

PRINCIPAL COMPONENTS ANALYSIS 5
component, which equals the sum of the eigenvalues, which equals the number
of variables. That is,
m�
m�
m�
Var(Yi) = Var(Zi) = λi = m
i=1
i=1
� Result 2. The partial correlation between a given component and a given variable
is a
√
function of an eigenvector and an eigenvalue. Specifically, Corr(Yi, Z j) =
eij λi, i, j = 1, 2,...,m, where (λ1, e1), (λ2, e2),...,(λm, em) are the
eigenvalue–eigenvector pairs for the correlation matrix ρ, and we note that
λ1 ≥ λ2 ≥···≥λm. Apartial correlation coefficient is a correlation coefficient
that takes into account the effect of all the other variables.
� Result 3. The proportion of the total variability in Z that is explained by the ith
principal component is the ratio of the ith eigenvalue to the number of variables,
that is, the ratio λi/m.
Next, to illustrate how to apply principal components analysis on real data, we
turn to an example.
Applying Principal Components Analysis
to the Houses Data Set
We turn to the houses data set [3], which provides census information from all the
block groups from the 1990 California census. For this data set, a block group has
an average of 1425.5 people living in an area that is geographically compact. Block
groups that contained zero entries for any of the variables were excluded. Median
house value is the response variable; the predictor variables are:
� Median income
� Housing median age
� Total rooms
� Total bedrooms
� Population
� Households
� Latitude
� Longitude
The original data set had 20,640 records, of which 18,540 were selected randomly
for a training data set, and 2100 held out for a test data set. A quick look at
the variables is provided in Figure 1.1. (“Range” is Clementine’s type label for continuous
variables.) Median house value appears to be in dollars, but median income
has been scaled to a continuous scale from 0 to 15. Note that longitude is expressed
in negative terms, meaning west of Greenwich. Larger absolute values for longitude
indicate geographic locations farther west.
Relating this data set to our earlier notation, we have X1 = median income,
X2 = housing median age,..., X8 = longitude, so that m = 8 and n = 18,540. A
glimpse of the first 20 records in the data set looks like Figure 1.2. So, for example, for
the first block group, the median house value is $452,600, the median income is 8.325
(on the census scale), the housing median age is 41, the total rooms is 880, the total
bedrooms is 129, the population is 322, the number of households is 126, the latitude
is 37.88 North and the longitude is 122.23 West. Clearly, this is a smallish block
group with very high median house value. A map search reveals that this block group
i=1