Time Series - STAT - EPFL

Time Series
Anthony Davison
c○2008
http://stat.epfl.ch
Introduction 2
Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
CO 2 at Mauna Loa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Northern hemisphere temperatures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Arosa ozone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Eskdalemuir rainfall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Japanese earthquakes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Beaver body temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Seal position. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Seal position. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Infectious diseases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
FTSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
European stock markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Time-course microarray . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Course details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Mathematical basics 20
Stochastic process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Continuous/discrete time models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Measures of dependence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
White noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Moving average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Autoregression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Periodic series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1

Introduction slide 2
Motivation
□ In basic data analysis we make often assume that observations are independent or even
independent identically distributed,
X 1 ,...,X n
ind
∼ F 1 ,...,F n ,
X 1 ,... ,X n
iid ∼ N(µ,σ 2 ).
□ Time series is the study of observations that arise in some order (almost always time) and which
as a result are dependent.
□ There are many more ways to be dependent than to be independent, and almost all data are
collected in time order, so time series arise in a vast range of disciplines: economics; finance;
marketing; epidemiology; biomedicine; genomics; environmental science; computer science;
electrical engineering; physics; ...
□ Many of these disciplines have developed special techniques for their data types, and we will only
scratch the surface of them in this course, by surveying some main ideas.
Time Series Autumn 2008 – slide 3
CO 2 at Mauna Loa
Monthly levels of carbon dioxide (ppm) at Mauna Loa (Hawaii) from March 1958 to July 2007.
co2
320 340 360 380
1960 1970 1980 1990 2000
Time
Time Series Autumn 2008 – slide 4
2

Northern hemisphere temperatures
Temperature anomaly ( ◦ C) for 0–1979 relative to 1961–1990 instrumental average.
Temperature anomaly (C)
−1.5 −1.0 −0.5 0.0 0.5
0 500 1000 1500 2000
Year (AD)
Time Series Autumn 2008 – slide 5
Arosa ozone
Annual average and daily total atmospheric ozone measurements (in Dobson units) at Arosa.
Annual average ozone (DU)
300 330 360
Daily ozone (DU)
200 350 500
1940 1950 1960 1970 1980 1990 2000
Time
1940 1950 1960 1970 1980 1990 2000
Time
Time Series Autumn 2008 – slide 6
3

Eskdalemuir rainfall
Hourly rainfall totals at Eskdalemuir, in the south of Scotland
Hourly rainfall (0.1 mm)
0 50 100 150
1975 1976 1977 1978 1979 1980
Year
Time Series Autumn 2008 – slide 7
Japanese earthquakes
Times and magnitudes of earthquakes with epicentre less than 100km in an offshore region west of
the main Japanese island of Honshū and south of the northern island of Hokkaidō. The figure shows
all 483 earthquakes of magnitude 6 or more on the Richter scale in the period 1885–1980, about 5
tremors per year, in one of the most seismically active areas of Japan.
Magnitude (Richter units)
6.0 6.5 7.0 7.5 8.0 8.5
0 5000 10000 15000 20000 25000 30000 35000
Days since 1 January 1885
Time Series Autumn 2008 – slide 8
4

Beaver body temperature
100 consecutive telemetric measurements on the body temperature of a female Canadian beaver,
Castor canadensis, taken at 10-minute intervals. The animal remained in its lodge for the first 38
recordings and then moved outside, at which point there was a sustained temperature rise.
Body temperature (C)
36.5 37.0 37.5 38.0 38.5
0 20 40 60 80 100
Time
Time Series Autumn 2008 – slide 9
Seal position
Hawaiian monk seals, Monachus schauinslandi, number around 1300, are endemic to the Hawaiian
Islands and are the most endangered species of marine mammal that lives entirely within the
jurisdiction of the United States. The species has been declining partly owing to poor juvenile survival
which is evidently related to poor foraging success. Data have been collected recently on the foraging
habitats, movements, and behaviors of monk seals throughout the Northwestern and main Hawaiian
Islands.
The central Hawaiian islands
22.0
Kauai
Oahu
21.5
latitude (degrees)
21.0
Molokai
Maui
Pacific Ocean
20.5
−159 −158 −157 −156
longitude (degrees)
Time Series Autumn 2008 – slide 10
5

Seal position
Journey of a juvenile female (4-5 years old) Hawaiian monk seal while she foraged and occasionally
hauled out ashore. She was tagged and released at the southwest corner of Molokai, and tracked from
13 April 2004 through 27 July 2004, using a satellite-linked radio transmitter glued to her dorsal
pelage to document geographic and vertical movements as proxies of foraging behavior.
Seal’s motion between pairs of the well−determined points
100
northing (km)
80
60
40
240 260 280 300 320 340 360
easting (km)
Time Series Autumn 2008 – slide 11
Infectious diseases
Weekly counts of new influenza and meningococcal infections in Germany 2001–2006.
Meningococcus Influenza
0 20 0 1000
2001 2002 2003 2004 2005 2006 2007
Time Series Autumn 2008 – slide 12
6

FTSE
The Financial Times Stock Exchange Index, 1991–1998.
FTSE Index
3000 4000 5000 6000
1992 1993 1994 1995 1996 1997 1998
Time
Time Series Autumn 2008 – slide 13
European stock markets
The rise of European stock markets, 1991–1998.
EuStockMarkets
FTSE CAC SMI DAX
3000 6000 1500 3500
2000 6000 2000 5000
1992 1993 1994 1995 1996 1997 1998
Time Series Autumn 2008 – slide 14
Time
7

Time-course microarray
Expression levels for 2771 genes/sequence tages spotted on acDNA microarray, relating to gene
transcription in the immune response of Anopheline mosquitoes.
Time Series Autumn 2008 – slide 15
Time-course microarray
A clustering of the mosquito genes based on the time courses.
Time Series Autumn 2008 – slide 16
8

Comments
□ The measurements can be continuous (temperatures) or discrete (infectious disease counts) or a
mixture (rainfall), scalar or vector (seal position)
□ Mostly they are at regular intervals, but some are intermittent (quakes)
□ Some are instantaneous values, others are integrals (rain)
□ Some series exhibit strong trend and/or seasonality (CO 2 , infectious diseases, stock markets)
□ There can be missing observations and/or possible outliers (ozone, rainfall)
□ Series can be long (rainfall, temperatures) or very short (microarray)
□ May be one or a few or many series
□ Focus may be
– a possible change in the underlying series, so the dependence is a nuisance (e.g. ozone,
temperatures, beaver, microarray)
– dependence/interaction within or between series (quakes, diseases, stock markets)
– rare events (stock markets, rainfall, ozone)
– comparison of parallel series (microarray)
Time Series Autumn 2008 – slide 17
Objectives
Typically we have in mind one or more of the following general objectives:
□ Description
– Want a ‘simple’ summary of the series
□ Analysis
– Construct stochastic model(s), and try and answer questions with them
– Model will reflect knowledge about phenomenon under study and complexity of data available
– May just need to accommodate dependence as part of larger analysis
□ Monitoring/Control
– Blood pressure/chemical reactor temperature must be kept between x 0 and x 1
– Aim to detect changes as they occur and to influence process in real time
□ Forecasting
– ‘What will the market do tomorrow’ ‘What will the economy do next year’
– Sometimes the model is not so important (though economic models may be complex)
– Often combine different models to get best forecasts (‘model averaging’)
Time Series Autumn 2008 – slide 18
9

Course details
□ Place: MA30 (= MA A3 30)
□ Lectures 8.15–10.00, Monday 29 September 2008 onwards
□ Exercises 10.15–12.00, Monday 29 September 2008 onwards
□ Main reference: Shumway and Stoffer (2006) Time Series Analysis and its Applications. Second
edition. Springer-Verlag
□ Form of exam not yet determined (probably written, may be some project component)
□ Notes and exercises can be downloaded from course web page
http://stat.epfl.ch/page32112.html
Time Series Autumn 2008 – slide 19
Mathematical basics slide 20
Stochastic process
Definition 1 (a) A stochastic process {Y t } t∈T with index set T is a family of random variables
defined on a probability space (Ω, F,P).
(b) A realisation of {Y t } is the outcome {y t } t∈T = {Y t (ω)} t∈T for some ω ∈ Ω.
The index set T :
□ most models have index set T = R, R + , or Z; item owing to digitisation, T cannot in practice
contain a sub-interval of R, but the time step ∆ can be very small in some applications;
□ (almost-)continuous time series can be thinned by subsampling at the points of a grid, or, in some
cases, by integration over intervals of width ∆ (e.g. rainfall data);
□ for general discussion of time series we take T = Z, so that Y t is recorded at times 0, ±1, ±2,...,
and write a realisation of the process observed for a finite period as y 1 ,...,y n .
□ Intermittent time series involve index sets T that are not regular grids.
Time Series Autumn 2008 – slide 21
10

Continuous/discrete time models
Two main situations:
□ available data are part of a random sequence {Y t }, for which time t takes only integer values,
i.e. t ∈ Z. Thus Y t does not exist at (say) t = 0.5;
□ available data are values of a random function {Y (t)} that exists for all t ∈ R (or R + ) but is
only observed at a limited number of times.
In some cases (e.g. rainfall, with the time unit being hours) the observed data are
∫ t
t−1
Y (t)dt.
In this case, and particularly if we will want to use different time scales, there is a good case for
building a continuous-time model but estimating its parameters etc. using the cumulated/discrete
time data. Otherwise conclusions made for different time scales may be incoherent.
Time Series Autumn 2008 – slide 22
Measures of dependence
Definition 2 Let {Y t } t∈T be a stochastic process. Then
(a) if E(|Y t |) < ∞, then we define the mean (or expectation) of the process to be µ t = E(Y t ). If
non-constant µ t is sometimes called the trend;
(b) if var(Y t ) < ∞ for all t ∈ T , then we define the (auto)covariance function of the process to be
γ(s,t) = cov(Y s ,Y t ) = E {(Y s − µ s )(Y t − µ t )} , s,t ∈ T ,
and the (auto)correlation function of the process to be
ρ(s,t) =
γ(s,t)
{γ(s,s)γ(t,t)}
1/2,
s,t ∈ T .
□ Note that var(Y t ) = cov(Y t ,Y t ) = γ(t,t).
□ The Cauchy–Schwarz inequality gives |ρ(s,t)| ≤ 1 for all s,t ∈ T , with ρ(t,t) = 1 for all t.
□ The function γ(s,t) is semi-definite positive: ∑ a i a j γ(t t ,t j ) ≥ 0 for any a 1 ,... ,a k and any
{t 1 ,... ,t k } ⊂ T .
Time Series Autumn 2008 – slide 23
11

Stationarity
If S is a set, then we use u + S to denote the set {u + s : s ∈ S}, and Y S to denote the set of
random variables {Y s : s ∈ S}.
Definition 3 A stochastic process {Y t } t∈T is said to be
(a) strictly stationary if for any finite subset S ⊂ T and any u such that u + S ⊂ T , the joint
distributions of Y S+u and Y S are the same;
(b) second-order stationary (or weakly stationary) if the mean µ t is constant and the covariance
function γ(s,t) depends only on t − s.
□ When T = Z = {0, ±1, ±2,...} and the process is stationary,
say, where h is called the lag.
γ(t,t + h) = γ(0,h) = γ(0, −h) ≡ γ |h| = γ h , h ∈ Z,
□ Similarly, we can write ρ(t,t + h) ≡ ρ |h| = ρ h , say, for h ∈ Z.
□ Thus in the stationary case the covariance and correlation functions are symmetric around h = 0.
Time Series Autumn 2008 – slide 24
Stationarity
□ In practice strict stationarity is impossible to verify, and many computations require only
second-order stationarity.
□ Hereafter ‘stationary’ will mean second-order stationary, when used without comment.
□ We can also define third- and higher-order stationarity by extending (b) to higher moments.
□ In practice we often preprocess the data, by removing trend/seasonality, and model the processed
series using a stationary stochastic process.
□ However treating variation as random or as trend depends on the purpose of analysis. Consider
the figure below, or the temperature data ...
Y(t)
0.0 0.5 1.0 1.5 2.0 2.5
Y(t)
0.0 0.5 1.0 1.5 2.0 2.5
0 20 40 60 80 100
t
0 2 4 6 8 10
t
Time Series Autumn 2008 – slide 25
12

White noise
Definition 4 A stochastic process {Y t } is called white noise if its elements are all uncorrelated, with
mean E(Y t ) = 0 and variance var(Y t ) = σ 2 .
If in addition the Y t are normally (Gaussianly) distributed, then we have Gaussian white noise,
iid
Y t ∼ N(0,σ 2 ).
The term ‘white’ comes from an analogy with white light, and indicates that all frequencies are
equally present ...
−3 −1 1 3
0 100 200 300 400 500
Time
−3 −1 1 3
0 100 200 300 400 500
Time
Time Series Autumn 2008 – slide 26
Moving average
□ The panels on the previous page showed Gaussian white noise {ε t } above, and a smoothed version
Y t = 1 3 (ε t + ε t−1 + ε t−2 ).
□ Averaging reduces the variance, and introduces correlation in {Y t }.
Example 5 Compute the autocorrelation function of the above moving average and show that it is
stationary. Discuss the figure below.
Y_{t+1}
−1.5 −0.5 0.5 1.5
Y_{t+2}
−1.5 −0.5 0.5 1.5
Y_{t+3}
−1.5 −0.5 0.5 1.5
−1.5 −0.5 0.5 1.5 −1.5 −0.5 0.5 1.5 −1.5 −0.5 0.5 1.5
Y_t
Y_t
Y_t
Time Series Autumn 2008 – slide 27
13

Random walk
Example 6 Let T = {0,1,2,...}, let {ε t } be white noise, let Y 0 = 0, and define
Y t = Y t−1 + ε t , t = 1,2,... .
Show that this is not a stationary time series.
Y
0 10 20 30 40
0 100 200 300 400 500
Time
Time Series Autumn 2008 – slide 28
Autoregression
Example 7 Let T = Z, let {ε t } be white noise, and define
Y t = αY t−1 + ε t .
This is an autoregressive process of order 1, AR(1), model. Find a necessary condition for it to be
stationary. The graph below shows examples with α = ±0.9.
Y(t)
−4 −2 0 2 4
Y(t)
−6 −4 −2 0 2 4 6
0 50 100 150 200 0 50 100 150 200
Time
Time
Time Series Autumn 2008 – slide 29
14

Periodic series
Example 8 Let {ε t } be white noise with unit variance, and define
( ) 2πt
Y t = cos + 5ε t , t = 1,... ,500.
50
This is a periodic signal obscured by noise. Compute its mean and autocorrelation function.
−2 0 1 2
0 100 200 300 400 500
Time
−20 0 10
0 100 200 300 400 500
Time
Time Series Autumn 2008 – slide 30
Summary
Today we
□ saw some examples of time series
□ introduced some basic ideas:
– use of stochastic process as model for time series
– mean, covariance and correlation functions
– stationary and strictly stationary series
– white noise
– simple examples: moving average, random walk, autoregression, periodic series
Next week:
□ simple approaches to removing systematic variation (trend and seasonality)
Time Series Autumn 2008 – slide 31
15