Springer - Read

6.6 Regression with ARMA Errors 211
many applications of regression analysis, however, this assumption is clearly violated,
as can be seen by examination of the residuals from the fitted regression and
their sample autocorrelations. It is often more appropriate to assume that the errors
are observations of a zero-mean second-order stationary process. Since many autocorrelation
functions can be well approximated by the autocorrelation function of a
suitably chosen ARMA(p, q) process, it is of particular interest to consider the model
Yt x ′
t β + Wt, t 1,...,n, (6.6.1)
or in matrix notation,
Y Xβ + W, (6.6.2)
where Y (Y1,...,Yn) ′ is the vector of observations at times t 1,...,n, X
is the design matrix whose tth row, x ′ t (xt1,...,xtk), consists of the values of
the explanatory variables at time t, β (β1,...,βk) ′ is the vector of regression
coefficients, and the components of W (W1,...,Wn) ′ are values of a causal zeromean
ARMA(p, q) process satisfying
φ(B)Wt θ(B)Zt, {Zt} ∼WN 0,σ 2 . (6.6.3)
The model (6.6.1) arises naturally in trend estimation for time series data. For
example, the explanatory variables xt1 1,xt2 t, and xt3 t 2 can be used to
estimate a quadratic trend, and the variables xt1 1,xt2 cos(ωt), and xt3 sin(ωt)
can be used to estimate a sinusoidal trend with frequency ω. The columns of X are
not necessarily simple functions of t as in these two examples. Any specified column
of relevant variables, e.g., temperatures at times t 1,...,n, can be included in the
design matrix X, in which case the regression is conditional on the observed values
of the variables included in the matrix.
The ordinary least squares (OLS) estimator of β is the value, ˆβOLS, which
minimizes the sum of squares
(Y − Xβ) ′ (Y − Xβ)
n
t1
Yt − x ′
t β 2 .
Equating to zero the partial derivatives with respect to each component of β and
assuming (as we shall) that X ′ X is nonsingular, we find that
ˆβOLS (X ′ X) −1 X ′ Y. (6.6.4)
(If X ′ X is singular, ˆβOLS is not uniquely determined but still satisfies (6.6.4) with
(X ′ X) −1 any generalized inverse of X ′ X.) The OLS estimate also maximizes the
likelihood of the observations when the errors W1,...,Wn are iid and Gaussian. If
the design matrix X is nonrandom, then even when the errors are non-Gaussian and
dependent, the OLS estimator is unbiased (i.e., E
ˆβOLS β) and its covariance
matrix is
Cov( ˆβOLS) X ′ X −1 X ′ ƔnX X ′ X −1 , (6.6.5)