Applied Statistics Using SPSS, STATISTICA, MATLAB and R

7.2 Multiple Regression
7.2.1 General Linear Regression Model
7.2 Multiple Regression 289
Assuming the existence of p − 1 predictor variables, the general linear regression
model is the direct generalisation of 7.1:
p
i = β + β xi
+ β xi
+ + β p xi
p + ε i = ∑ β k xik
+
k
i
−1
0 1 1 2 2 K −1
, −1
= 0
, 7.34
Y ε
with x 1.
In the following we always consider normal regression models with
i0
=
i.i.d. errors εi ~ N0,σ.
Note that:
– The general linear regression model implies that the observations are
independent normal variables.
– When the xi represent values of different predictor variables the model is
called a first-order model, in which there are no interaction effects between
the predictor variables.
– The general linear regression model encompasses also qualitative predictors.
For example:
Y ε
i = β 0 + β1x
i1
+ β 2 xi2
+ i . 7.35
xi1
=
x i2
patient ’
s
⎧1
= ⎨
⎩0
weight
if patient female
if patient male
Patient is male: Yi = β 0 + β1x
i1
+ ε i .
Patient is female: Yi = ( β 0 + β 2 ) + β1
xi1
+ ε i .
Multiple linear regression can be performed with SPSS, STATISTICA,
MATLAB and R with the same commands and functions listed in Commands 7.1.
7.2.2 General Linear Regression in Matrix Terms
In order to understand the computations performed to fit the general linear
regression model to the data, it is convenient to study the normal equations 7.3 in
matrix form.
We start by expressing the general linear model (generalisation of 7.1) in matrix
terms as:
y = Xβ + ε, 7.36