Preface to First Edition - lib

MULTIPLE LINEAR REGRESSION 101The error terms ε i , i = 1, ...,n, are assumed to be independent randomvariables having a normal distribution with mean zero and constant varianceσ 2 . Consequently, the distribution of the random response variable, y, is alsonormal with expected value given by the linear combination of the explanatoryvariablesE(y|x 1 , ...,x q ) = β 0 + β 1 x 1 + · · · + β q x qand with variance σ 2 .The parameters of the model β k , k = 1, ...,q, are known as regressioncoefficients with β 0 corresponding to the overall mean. The regression coefficientsrepresent the expected change in the response variable associated witha unit change in the corresponding explanatory variable, when the remainingexplanatory variables are held constant. The linear in multiple linear regressionapplies to the regression parameters, not to the response or explanatoryvariables. Consequently, models in which, for example, the logarithm of a responsevariable is modelled in terms of quadratic functions of some of theexplanatory variables would be included in this class of models.The multiple linear regression model can be written most conveniently forall n individuals by using matrices and vectors as y = Xβ + ε where y ⊤ =(y 1 , ...,y n ) is the vector of response variables, β ⊤ = (β 0 , β 1 , ...,β q ) is thevector of regression coefficients, and ε ⊤ = (ε 1 , ...,ε n ) are the error terms. Thedesign or model matrix X consists of the q continuously measured explanatoryvariables and a column of ones corresponding to the intercept term⎛⎞1 x 11 x 12 . .. x 1q1 x 21 x 22 . .. x 2qX = ⎜⎝.... ... ..⎟⎠ .1 x n1 x n2 . .. x nqIn case one or more of the explanatory variables are nominal or ordinal variables,they are represented by a zero-one dummy coding. Assume that x 1 is afactor at m levels, the submatrix of X corresponding to x 1 is a n × m matrixof zeros and ones, where the jth element in the ith row is one when x i1 is atthe jth level.Assuming that the cross-product X ⊤ X is non-singular, i.e., can be inverted,then the least squares estimator of the parameter vector β is unique and canbe calculated by ˆβ = (X ⊤ X) −1 X ⊤ y. The expectation and covariance of thisestimator ˆβ are given by E( ˆβ) = β and Var( ˆβ) = σ 2 (X ⊤ X) −1 . The diagonalelements of the covariance matrix Var( ˆβ) give the variances of ˆβ j , j = 0, ...,q,whereas the off diagonal elements give the covariances between pairs of ˆβ jand ˆβ k . The square roots of the diagonal elements of the covariance matrixare thus the standard errors of the estimates ˆβ j .If the cross-product X ⊤ X is singular we need to reformulate the model toy = XCβ ⋆ + ε such that X ⋆ = XC has full rank. The matrix C is called thecontrast matrix in S and R and the result of the model fit is an estimate ˆβ ⋆ .© 2010 by Taylor and Francis Group, LLC