Modeling and Multivariate Methods - SAS

Chapter 21 Fitting Partial Least Squares Models 521
Statistical Details
Partial Least Squares
NIPALS
SIMPLS
Partial least squares fits linear models based on linear combinations, called factors, of the explanatory
variables (Xs). These factors are obtained in a way that attempts to maximize the covariance between the Xs
and the response or responses (Ys). In this way, PLS exploits the correlations between the Xs and the Ys to
reveal underlying latent structures. The factors address the combined goals of explaining response variation
and predictor variation. Partial least squares is particularly useful when you have more X variables than
observations or when the X variables are highly correlated.
The NIPALS method works by extracting one factor at a time. Let X = X 0 be the centered and scaled matrix
of predictors and Y = Y 0 the centered and scaled matrix of response values. The PLS method starts with a
linear combination t =X 0 w of the predictors, where t is called a score vector and w is its associated weight
vector. The PLS method predicts both X 0 and Y 0 by regression on t:
ˆ
X0
= tp´, where p´ =(t´t) -1 t´X 0
ˆ
Y0
= tc´, where c´ =(t´t) -1 t´Y 0
The vectors p and c are called the X- and Y-loadings, respectively.
The specific linear combination t =X 0 w is the one that has maximum covariance t´u with some response
linear combination u = Y 0 q. Another characterization is that the X- and Y-weights, w and q, are
proportional to the first left and right singular vectors of the covariance matrix X 0´Y 0 . Or, equivalently, the
first eigenvectors of X 0´Y 0 Y 0´X 0 and Y 0´X 0 X 0´Y 0 respectively.
This accounts for how the first PLS factor is extracted. The second factor is extracted in the same way by
replacing X 0 and Y 0 with the X- and Y-residuals from the first factor:
ˆ
X 1 = X 0 – X0
ˆ
Y 1 = Y 0 – Y0
These residuals are also called the deflated X and Y blocks. The process of extracting a score vector and
deflating the data matrices is repeated for as many extracted factors as are desired.
The SIMPLS algorithm was developed to optimize a statistical criterion: it finds score vectors that maximize
the covariance between linear combinations of Xs and Ys, subject to the requirement that the X-scores are
orthogonal. Unlike NIPALS, where the matrices X 0 and Y 0 are deflated, SIMPLS deflates the cross-product
matrix, X 0´Y 0 .
In the case of a single Y variable, these two algorithms are equivalent. However, for multivariate Y, the
models differ. SIMPLS was suggested by De Jong (1993).