Applied Statistics Using SPSS, STATISTICA, MATLAB and R

7.1 Simple Linear Regression 275
Figure 7.3. Table obtained with STATISTICA containing the results of the simple
linear regression for the Example 7.1.
The value of Beta, mentioned in Figure 7.3, is related to the so-called
standardised regression model:
*
Y = β x + ε . 7.6
*
i
1
*
i
i
In equation 7.6 only one parameter is used, since Y and
variables (mean = 0, standard deviation = 1) of the observed and predictor
variables, respectively. (By equation 7.5, β 0 = E[ Y ] − β1x
implies ( Yi − E[
Y ] ) / σ Y
*
*
= β 1 ( xi − x)
/ s X + ε i .)
It can be shown that:
*
i
*
x i are standardised
σ *
β1 ⎟β1 ⎟
⎛ Y ⎞
=
⎜ . 7.7
⎝ s X ⎠
The standardised *
β 1 is the so-called beta coefficient, which has the point
*
estimate value b 1 = 0.98 in the table shown in Figure 7.3.
Figure 7.3 also mentions the values of R, R 2
and Adjusted R 2
. These are
measures of association useful to assess the goodness of fit of the model. In order
to understand their meanings we start with the estimation of the error variance, by
computing the error sum of squares or residual sum of squares (SSE) 1
, i.e. the
quantity E in equation 7.2, as follows:
∑
2
∑
2
SSE = ( y i − yˆ
i ) = ei
. 7.8
Note that the deviations are referred to each predicted value; therefore, SSE has
n − 2 degrees of freedom since two degrees of freedom are lost: b0 and b1. The
following quantities can also be computed:
1
–
SSE
Mean square error: MSE = .
n − 2
– Root mean square error, or standard error: RMS = MSE .
Note the analogy of SSE and SST with the corresponding ANOVA sums of squares,
formulas 4.25b and 4.22, respectively.