Stochastic Modelling Solutions to Exercises on Time Series∗

<strong>S<strong>to</strong>chastic</strong> <strong>Modelling</strong>
<strong>Solutions</strong> <strong>to</strong> <strong>Exercises</strong> on Time Series ∗
Dr. Iqbal Owadally †
March 3, 2003
<strong>Solutions</strong> <strong>to</strong> Elementary Problems
Q1. (i) (1 − 0.5B)X t = Z t . The characteristic equation 1 − 0.5z = 0 does not have a unit
root and the time series is ARIMA(1, 0, 0). The root of the characteristic equation
is 2. Since |2| > 1, the time series is stationary. It is also (trivially) invertible.
(ii) (1−0.5B)X t = (1−1.3B+0.4B 2 )Z t = (1−0.8B)(1−0.5B)Z t . Equation 1−0.5z = 0
does not have a unit root and the time series is ARIMA(1, 0, 2). The root of the
characteristic equation is 2. Since |2| > 1, the time series is stationary. The roots
of equation (1 − 0.8z)(1 − 0.5z) = 0 are 1.25 and 2. Since |1.25| > 2 and |2| > 1,
the time series is invertible.
(iii) (1 − 1.5B + 0.6B 2 )X t = Z t . The characteristic equation does not have a unit root
and the time series is ARIMA(2, 0, 0). The roots of the c.e. are (15 ± i √ 15)/12.
Since |(15 ± i √ 15)/12| = (15 2 + 15)/12 2 = 5/3 > 1, the time series is stationary.
It is also (trivially) invertible.
(iv) “0.2” causes only a translation of the mean level of the process and may be ignored
here: let Y t = X t − 0.2 and represent the time series as (1 − B)(1 − 0.2B)Y t =
(1−0.5B)Z t . The characteristic equation does have a unit root and the time series
is ARIMA(1, 1, 1). The time series is non-stationary (being an I(1) process) but
is invertible (the root of 1 − 0.5z = 0 being greater than 1 in magnitude).
Q2. These processes are stationary: EX t and VarX t are constant and Cov [X t , X t−k ] depends
on k and not on t.
(i) EX t + 0.5EX t−1 − 0.1EX t−2 = EZ t = 0 and hence EX t = 0 ∀ t and ρ k = ρ −k =
E [X t X t−k ] /VarX t . Furthermore, Z t is independent of X t−k for k ≥ 1, so that
E [Z t X t−k ] = EZ t EX t−k = 0.
Multiplying across X t + 0.5X t−1 − 0.1X t−2 = Z t by X t−k for k ≥ 1 gives
X t X t−k + 0.5X t−1 X t−k − 0.1X t−2 X t−k = Z t X t−k .
∗ <strong>Solutions</strong> <strong>to</strong> past exam questions are adapted from examiners’ reports <strong>to</strong> the Exam Board of the Institute
and Faculty of Actuaries and <strong>to</strong> the Exam Board, Faculty of Actuarial Science and Statistics, Cass Business
School, City University.
† Contact details: Cass Building Room 5071, extension 8478, iqbal@city.ac.uk.
1

Taking expectation on both sides yields
E [X t X t−k ] + 0.5E [X t−1 X t−k ] − 0.1E [X t−2 X t−k ] = 0
and dividing by VarX t on both sides gives ρ k + 0.5ρ k−1 − 0.1ρ k−2 = 0.
When k = 1, ρ 1 + 0.5ρ 0 − 0.1ρ 1 = 0 which solves <strong>to</strong> ρ 1 = −0.5/0.9 = −5/9
(noting that ρ k = ρ −k and ρ 0 = 1 by definition).
When k = 2, ρ 2 + 0.5ρ 1 − 0.1ρ 0 = 0 yielding ρ 2 = 0.1 − 0.5(−5/9) = 17/45.
When k = 3, ρ 3 +0.5ρ 2 −0.1ρ 1 = 0 yielding ρ 3 = 0.1(−5/9)−0.5(17/45) = −11/45.
(ii) {X t } in X t + 0.6X t−2 = Z t will clearly have zero au<strong>to</strong>correlation at odd lags.
ρ k + 0.6ρ k−2 = 0 for k ≥ 1.
When k = 1, ρ 1 + 0.6ρ 1 = 0 and ρ 1 = 0.
When k = 2, ρ 2 + 0.6ρ 0 = 0 and ρ 2 = −0.6.
When k = 3, ρ 3 + 0.6ρ 1 = 0 and ρ 3 = 0.
(iii) ρ k − 1.1ρ k−1 + 0.18ρ k−2 = 0 for k ≥ 1.
When k = 1, ρ 1 − 1.1ρ 0 + 0.18ρ 1 = 0 and ρ 1 = 1.1/1.18 = 55/59.
When k = 2, ρ 2 − 1.1ρ 1 + 0.18ρ 0 = 0 and ρ 2 = 1.1(55/59) − 0.18 = 1247/1475.
When k = 3, ρ 3 − 1.1ρ 2 + 0.18ρ 1 = 0 and ρ 3 = 1.1(1247/1475) − 0.18(55/59) =
5621/7375.
(iv) ρ k + αρ k−1 + α 2 ρ k−2 + α 3 ρ k−3 = 0 for k ≥ 1.
When k = 1, ρ 1 (1 + α 2 ) + α + α 3 ρ 2 = 0.
When k = 2, ρ 1 (α + α 3 ) + α 2 + ρ 2 = 0.
(Note in the above that ρ 0 = 1 and ρ k = ρ −k ).
Hence ρ 1 = −α/(1 + α 2 ) and ρ 2 = 0.
When k = 3, ρ 3 + αρ 2 + α 2 ρ 1 + α 3 ρ 0 = 0.
Hence, ρ 3 = −α 5 /(1 + α 2 ).
Q3. (i) Y t = Z t − βZ t−1 .
VarY t = VarZ t + β 2 VarZ t−1 = σ 2 Z (1 + β2 ).
Cov [Y t , Y t−1 ] = Cov [Z t − βZ t−1 , Z t−1 − βZ t−2 ] = −βσ 2 Z .
Cov [Y t , Y t−k ] = 0 for k ≥ 2.
Since ρ k = γ k /γ 0 , ρ 1 = −β/(1 + β 2 ) and ρ k = 0 for k ≥ 2.
(ii) Y t = Z t + 2.4Z t−1 + 0.8Z t−2 .
VarY t = σ 2 Z (1 + 2.42 + 0.8 2 ) = 7.4σ 2 Z .
Cov [Y t , Y t−1 ] = Cov [Z t + 2.4Z t−1 + 0.8Z t−2 , Z t + 2.4Z t−1 + 0.8Z t−2 ] = (2.4+0.8×
2.4)σ 2 Z = 7.4σ2 Z .
Cov [Y t , Y t−2 ] = 0.8σ 2 Z .
Cov [Y t , Y t−k ] = 0 for k ≥ 3.
Since ρ k = γ k /γ 0 , ρ 1 = 4.32/7.4 = 0.5838, ρ 2 = 0.8/7.4 = 0.1081 and ρ k = 0 for
k ≥ 3.
Q4. (i) The correlogram is a plot of au<strong>to</strong>correlation function against lag, i.e. ρ k vs. |k|. An
MA(q) process has a correlogram with a cu<strong>to</strong>ff at lag q, whereas a stationary AR
process has a correlogram that tapers off gradually (possibly with some damped
oscillations) <strong>to</strong> zero as the lag increases.
Comments. Plotting the correlogram is therefore a way of distinguishing MA from
AR time series, although in practice estimation errors may mean that it is not so
easy <strong>to</strong> discern a difference.
2

(ii) The process is assumed <strong>to</strong> be stationary and we can proceed <strong>to</strong> find the au<strong>to</strong>correlation
function by using the usual method for stationary time series. The time
series is shifted by µ and EX t = µ ∀ t. Let Y t = X t − µ. Then, EY t = 0 ∀ t.
The au<strong>to</strong>covariance function is γ k = Cov [X t , X t−k ] = Cov [Y t , Y t−k ] = E[Y t Y t−k ].
Recall that γ k = γ −k . Also, EZ t = 0 and VarZ t = σ 2 Z ∀ t.
Multiply Y t − αY t−1 = Z t − βZ t−1 by Y t−k on both sides and take expectations
<strong>to</strong> obtain γ k − αγ k−1 = E[Y t−k Z t ] − βE[Y t−k Z t−1 ]. Observe that Z t is statistically
independent of Y t−k , k ≥ 1. Therefore,
when k = 0, γ 0 − αγ 1 = E[Y t Z t ] − βE[Y t Z t−1 ], (1)
when k = 1, γ 1 − αγ 0 = −βE[Y t Z t−1 ], (2)
when k > 1, γ k − αγ k−1 = 0. (3)
The non-zero cross-correlations involving Y t and Z t are easily found. Multiply
Y t − αY t−1 = Z t − βZ t−1 by Z t on both sides and take expectations <strong>to</strong> obtain
E[Y t Z t ] = σZ 2 ∀ t, noting that {Z t} is a sequence of independent and identically
distributed random variables. Also, multiply Y t − αY t−1 = Z t − βZ t−1 by Z t−1
on both sides and take expectations <strong>to</strong> obtain E[Y t Z t−1 ] − αE[Y t−1 Z t−1 ] = −βσZ 2 .
Hence, E[Y t Z t−1 ] = σZ 2 (α − β).
Equation (1) may be simplified <strong>to</strong> γ 0 − αγ 1 = σZ 2 [1 − αβ + β2 ] and equation (2)
simplifies <strong>to</strong> γ 1 − αγ 0 = −βσZ 2 . Solving these two equations simultaneously yields:
γ 0 = σ2 Z (1 − 2αβ + β2 )
1 − α 2 , γ 1 = σ2 Z
(α − β)(1 − αβ)
1 − α 2 .
Finally, equation (3) solves <strong>to</strong> γ k = α k−1 γ 1 , for k > 1.
Since ρ k = γ k /γ 0 by definition,
⎧
⎪⎨ 1 k = 0
ρ k = (α − β)(1 − αβ)/(1 − 2αβ + β
⎪⎩
2 ) k = 1
α k−1 ρ 1 k > 1.
(iii) The correlogram of the ARMA(1, 1) time series has a ‘kink’ at lag 1. At lag 1,
both the MA and AR components affect the correlogram. From lag 2 onwards, it
looks rather like the correlogram of an AR series and it decays exponentially.
Q5. Comments. If the process were known (or assumed) <strong>to</strong> be stationary over time, then
EX t = 0 and ρ k = 0.4 |k| , using the properties of a stationary AR(1) process. But
the process has a finite his<strong>to</strong>ry (with a given initial condition) and so it will be nonstationary
for t < ∞.
Solution.
(i) In terms of the backward shift opera<strong>to</strong>r,
X t = Z t /(1 − 0.4B) = Z t + 0.4Z t−1 + 0.4 2 Z t−2 + · · ·
3

The process is arbitrarily started at t = 0 with X 0 = x 0 and X j = 0 for j ≤ −1.
Hence, Z 0 = x 0 and Z j = 0 for j ≤ −1. Therefore,
∑t−1
X t = 0.4 j Z t−j + x 0 0.4 t .
j=0
{Z t } is a sequence of zero-mean random variables. Hence, EX t = x 0 0.4 t for t ≥ 0.
(ii) Note that
and that
and also that
X t+k − EX t+k =
∑t−1
X t − EX t = 0.4 j Z t−j
t+k−1
∑
j=0
j=0
0.4 j Z t+k−j =
∑t−1
j=−k
0.4 j+k Z t−j
Cov [X t , X t+k ] = E {(X t − EX t )(X t+k − EX t+k )} .
For k ≥ 0,
⎧
⎨∑t−1
Cov [X t , X t+k ] = E 0.4 j Z
⎩
t−j ×
j=0
∑t−1
j=−k
⎫
⎬
0.4 j+k Z t−j
⎭ = ∑t−1
σ2 Z 0.4 j 0.4 j+k
(where σZ
2 = VarZ t) because {Z t } is a sequence of zero-mean independent and
identically distributed random variables. Hence,
for k ≥ 0 and t ≥ 0.
Cov [X t , X t+k ] = σ 2 Z0.4 k (1 − 0.4 2t )/(1 − 0.4 2 ) (4)
(iii) The process is not stationary for t < ∞ as both EX t and Cov [X t , X t+k ] vary
with t.
(iv) The process is stationary in the limit since the magnitude of the root of the characteristic
equation 1 − 0.4z = 0 is 2.5 and |2.5| > 1.
As t → ∞, EX t = x 0 0.4 t → 0 and Cov [X t , X t+k ] → σ 2 Z 0.4k /(1 − 0.4 2 ) and
VarX t → σ 2 Z /(1 − 0.42 ).
In the limit, the au<strong>to</strong>correlation coefficient ρ k = Cov [X t , X t−k ] /VarX t = 0.4 k .
(v) Suppose that the initial value x 0 at time 0 is itself a random variate that is normally
distributed with zero mean and variance σ 2 Z /(1−0.42 ). This distribution is the same
as that of ˜X t ∀ t in the stationary AR(1) process { ˜X t } given by ˜X t = 0.4 ˜X t−1 +Z t .
One may therefore choose <strong>to</strong> put x 0 = ˜X 0 = ∑ ∞
j=0 Z −j0.4 j .
Since X t = ∑ t−1
j=0 0.4j Z t−j + x 0 0.4 t , it follows that
X t =
∑t−1
∑
∞
0.4 j Z t−j + 0.4 t Z −j 0.4 j =
j=0
j=0
j=0
∞∑
0.4 j Z t−j .
In other words, an initial condition that is random and taken from N(0, σ 2 Z /(1 −
0.4 2 )) yields an AR(1) process that is stationary ab initio.
j=0
4

Q6. (i) EW t = EX t +EY t is constant over time. The au<strong>to</strong>covariance of the process {W t } is
Cov [W t , W t−k ] = Cov [X t + Y t , X t−k + Y t−k ] = Cov [X t , X t−k ] + Cov [Y t , Y t−k ] and
depends on lag k and not on time t.
(ii) X t = ln CPI t − ln CPI t−1 and hence ln CPI t = ∑ t
j=−∞ X j. Since X t is an AR(1)
process, it is clear that ln CPI t is ‘integrated’ once and is ARIMA(1, 1, 0). Alternatively,
note that (1 − B)(1 − αB) ln CPI t = Z t + µ(1 − α) and a unit root
occurs.
<strong>Solutions</strong> <strong>to</strong> Past Exam Questions
Q1. (i) (a) γ 0 = Var(e t +β 1 e t−1 ) = (1+β 2 1 )σ2 e and γ 1 = Cov [e t + β 1 e t−1 , e t−1 + β 1 e t−2 ] =
β 1 σ 2 e, with γ k = 0 for k > 1.
This gives ρ 0 = 1, ρ 1 = β 1 /(1 + β 2 1 ), ρ k = 0 otherwise.
(b) Invertibility requires that |β 1 | < 1, so that the sum X t −β 1 X t−1 +β 2 1 X t−2 +· · ·
converges. µ and σ e are irrelevant.
(ii) We need <strong>to</strong> solve (1 + β 2 1 )σ2 e = 14.5, β 1 σ 2 e = 5.0. Eliminating σ 2 e, we have 1 + β 2 1 =
2.9β 1 , or β 1 = 1 2 (2.9 ± √ (2.9 2 − 4)) = 2.5 or 0.4.
β 1 = 2.5 corresponds <strong>to</strong> σ 2 e = 2, whereas β 1 = 0.4 corresponds <strong>to</strong> σ 2 e = 12.5.
For invertibility, solve 1 + β 1 z = 0. In the first case, z = −0.4 (no good); in the
second, z = −2.5 (OK).
(Exam Board of the Institute and Faculty of Actuaries)
Q2. (i) (a) γ 1 = Cov [X t , X t−1 ] = Cov [α 1 X t−1 + α 2 X t−2 + e t , X t ] = α 1 γ 0 +α 2 γ 1 +0, since
e t is independent of X t−1 .
(b) Similarly γ 2 = α 1 γ 1 + α 2 γ 0 and γ 0 = α 1 γ 1 + α 2 γ 2 + Cov [X t , e t ]. A further
application of the same technique gives Cov [X t , e t ] = σe. 2 Thus,
γ 1 = α (
)
1
γ 0 , γ 2 = α 2 + α2 1
γ 0 .
1 − α 2 1 − α 2
(c) ρ k is found by the relation ρ k = γ k /γ 0 .
(ii) We have ˆα 1 = r 1 (1 − ˆα 2 ) and ˆα 2 + ˆα 2 1 /(1 − ˆα 2) = r 2 , which are solved by
ˆα 1 = r 1(1 − r 2 )
1 − r 2 1
, ˆα 2 = r 2 − r1
2
1 − r1
2 .
(Exam Board of the Institute and Faculty of Actuaries)
Q3. (i) γ 1 = 0.8γ 0 − 0.4γ 1 + 0 and γ 2 = 0.8γ 1 − 0.4γ 0 + 0.
Hence, γ 1 = 4 7 γ 0 and γ 2 = 2 35 γ 0, implying that ρ 1 = 4 7 and ρ 2 = 2
φ 1 = ρ 1 = 4 7 and φ 2 = ρ 2−ρ 2 1
= −0.4.
1−ρ 2 1
(ii) The acf will reduce <strong>to</strong> zero as k increases; the pacf, however, will be equal <strong>to</strong> zero
for all k > 2.
35 .
(Exam Board of the Institute and Faculty of Actuaries)
5

Q4. (i) Identification: Determination of the parameters p, d and q in the ARIMA(p, d, q)
model.
Estimation: Determination of the p AR parameters α 1 , α 2 , α 3 , . . . , α p and the q
MA parameters β 1 , β 2 , β 3 , . . . , β q in the stationary ARMA(p, q) model for the
dth differences of the observed series.
Diagnosis: Testing the goodness of fit of the proposed model.
(ii) The optimal value of d is the smallest value that produces a stationary series.
Three criteria used are: (a) sample acf should tend rapidly <strong>to</strong> zero
(b) sample variance should be mimised (c) series should exhibit a constant mean
(iii) Parismony is using the lowest values for the parameters p, d and q which adequately
model the observed series, i.e. additional parameters are only added if
they significantly improve the fit of the model.
(iv) Q 10 = 78 × ∑ 10
k=1 r2 k
= 78 × 0.1163 = 9.07
Under null hypothesis of independence of residuals, Q 10 has a χ 2 distribution with
10 − 1 = 9 degrees of freedom.
From tables, 95th percentile of χ 2 9 is 16.92. Hence, accept H 0.
(v) Let M = number of tps = 48. Let N = number of residuals = 78.
Under null hypothesis of independence of residuals,
( )
M app 2 16N − 29
∼ N (N − 2),
3 90
Then, M ∼ N ( 50 2 3 , 1219 )
90 .
(
)
P (M ≤ 48) = P Z ≤ 48.5−50 2 √ 3
(1219/90)
= P (Z ≤ 0.59) = 1 − Φ(0.59) = 0.2776
⇒ accept H 0 at 5% level.
(Exam Board, Faculty of Actuarial Science and Statistics, Cass Business School, City University)
Q5. (i) Consumer prices do tend <strong>to</strong> exhibit regular seasonal variation, though not a great
deal these days. And, since prices tend <strong>to</strong> go up rather more than they come down,
it is probably worth including a trend term in any model. It is certainly possible
<strong>to</strong> test whether the trend term is equal <strong>to</strong> zero.
(ii) (a) X n+1 − x n = α(x n − x n−1 ) + e n+1 + βe n .
(b) The parameters are α, β and σe.
2 The trend removal process would have
accounted for any µ parameter.
(iii) ˆx n (1) = E [X n+1 | x n , . . . , x 1 ] = x n + α(x n − x n−1 ) + E [e n+1 + βe n | x n , . . . , x 1 ].
Now e n+1 has mean 0 and is conventionally supposed independent of everything
that happens before n.
On the other hand, e n can be deduced from past data,
e.g. e n = x n − x n−1 − α(x n−1 − x n−2 ) − βe n−1 , which may be iterated back <strong>to</strong> get
e n in terms of the known x and the known e 0 .
Thus,
ˆx n (1) = x n + α(x n − x n−1 ) + βe n .
Similarly,
ˆx n (2) = E [X n+2 | F n ] = E [X n+1 + α(X n+1 − x n ) + e n+2 + βe n+1 | F n ]
= (1 + α)ˆx n (1) − αx n .
6

We see that X n+1 − ˆx n (1) = e n+1 , so that the prediction variance is just
Var(e n+1 ) = σ 2 e.
(iv) Since e n = x n − ˆx n−1 (1), we have
ˆx n (1) = x n + α(x n − x n−1 ) + β(x n − ˆx n−1 (1)).
If we set α = 0 and β = −ξ, the equation is identical <strong>to</strong> the updating equation for
exponential smoothing.
(v) An ARIMA(p, d, q) model is I(d); in this case, x is I(1).
A stationary (I(0)) model has an equilibrium distribution: the distribution of the
forecast of X n+k would converge <strong>to</strong> equilibrium for large k. An I(1) process is the
partial sum of an I(0) process, so would have increasing variance, even if the mean
happened <strong>to</strong> be stable.
(vi) Two series {x} and {y} are cointegrated if both are I(1) but there are some constants
a and b such that {ax + by} is stationary.
Two processes are likely <strong>to</strong> be cointegrated if one drives the other, or if both are
driven by the same underlying process. In the given instance the suggestion is
certainly worth investigating.
(Exam Board of the Institute and Faculty of Actuaries)
7