Time Series - STAT - EPFL

Time Series
c○ 2013, A. C. Davison
Ecole Polytechnique Fédérale de Lausanne
EPFL-FSB-MATHAA-STAT
Station 8, 1015 Lausanne, Switzerland
http://stat.epfl.ch
Please signal any errors or omissions by email to Anthony.Davison@epfl.ch.
Course web page
Further material, links to data sets, etc., can be found at the course web page
http://stat.epfl.ch/page-90340-en.html
Planned lecture contents and exercises
1. Introduction (18/2/13)
• Motivation and examples
• Basic notions: stationarity; white noise; autoregression; moving average; periodic
series
• Exercises: 1, 2, 3, 4, 5
2. Simple techniques (25/2/13)
• Plotting; outliers; transformations; differencing
• Correlogram and partial correlogram. Examples. Box–Ljung test
• Smoothing: moving averages; polynomials; local polynomials; splines; loess; STL
decomposition
• Exercises: 7, 8, 9, 10
3. Periodogram and smoothing (4/3/13)
• Periodogram
• Smoothing
• Practical 1
4. AR(1) model (11/3/13)
• Theory
• Likelihood, including analysis of beaver data
• Exercises: 12, 14, 15, 16
1

5. General theory of stationary linear processes (18/3/13)
• General theory of stationary processes/spectral density and linear filters
• AR, MA, ARMA models
• Exercises: 18, 19, 20, 21, 23
6. ARMA models (25/3/13)
• Difference equations. First example of ACF for AR2 model
• Computation of ACF for ARMA models, ARMA(1,1) example
• Practical 2
7. ARIMA modelling (8/4/13)
• Computation of PACF; ARIMA modelling and example
• SARIMA modelling and CO2 example
• Exercises: 25, 26, 27, 28,
8. Regression and forecasting (15/4/13)
• Regression structure, with beaver example. Cochrane–Orcutt algorithm.
• Forecasting. Bootstrap for prediction uncertainty.
• Exercises: 29, 30, 31
9. Financial time series (22/4/13)
• Stylised facts, GARCH models and some extensions.
• Basic stochastic volatility models
• Practical 3
10. Frequency domain analysis (29/4/13)
• Reminders: spectrum, periodogram. Harmonics and leakage. Tapering. FFT.
• Distributional properties and their implications. Smoothing. Whittle estimation.
• Exercises: 32, 33, 34, 36, 39
11. State space models (6/5/13)
• Introduction. Local trend model. Filtering, prediction, smoothing. Kalman filter.
• State error recursion. Missing values. Initialisation. General state space model.
Comparison with ARIMA models.
• Exercises: 40, 41; Question 6 of 2010 exam.
12. Multiple series (13/5/13)
• Multiple time series: cross-correlogram etc.; bivariate spectral theory.
• Transfer function modelling
• Models for multiple time series: VAR, VMA, VARMA, factor models.
• Practical 4
2

13. Long-range dependence/Threshold models (27/5/13)
• Long-range dependence
• Continuation; threshold models.
• Work on mini-project
14. Discrete time series/Wrap-up (Reading)
Final mark
• Basic ideas; GLMs; Special models; Markov chain analysis
• Hidden Markov models and Gibbs sampler
• Work on mini-project
The final mark will be determined as follows:
• there will be a data analysis project, to be undertaken in pairs, involving analysis of a
time series data set (to be found by the students concerned, but validated by me), using
R. A report on the data is to be handed in by 7 June 2013. More about this may be
found below;
• a final exam (probably with 5 questions, of which full correct answers to 4 will give full
marks);
• let E be the mark for the final exam (≤ 3) and P be that for the project (≤ 2). Then the
mark M for the course is 1+E+P rounded to the nearest half-mark in {1,1.5,...,5.5,6}.
Data analysis project
Proposal
As soon as possible, I would like to see a brief proposal about your project, consisting of:
• a description of thedata that you plan to analyse, includingsomesuitable plots andmaybe
some other data exploration;
• an indication of the source of the data set;
• the objectives of your investigation;
• a brief overview of the analyses you anticipate completing.
Bear in mind that this will count for 40% of the course mark, and the number of credits for this
course is 4, so this is much less work than a semester project.
The proposal should be made by 8 April 2013 at the very latest, and the earlier the
better—then I can give feedback, and you can start work.
Data
You could look through books in the library, check out links on the course web page, or look
at the many times series data sets available in R. It is best if the data comprise a few hundred
equally-spaced observations, but you should try to find something that interests you.
3

Report
The report should be typed in either English or French. Do not include hand-written material
unless you want to write out equations and don’t know LATEX. Some notes on report-writing
can be found at http://stat.epfl.ch/page-34953.html, though your report should be much
shorter than for a semester project.
The structure should be
Introduction Briefly state the purpose of the analysis, what type of data were analysed, what
methods were used, and what the results were.
Description of the methods used, using your own words. Give the key elements only: you can
refer to the lecture notes and to books, but should give careful references (to pages and
equations etc.) so I can easily look them up. It is not enough simply to give a list of
sources at the end of the work: references should be mentioned in the text, and only those
mentioned in the text should be listed at the end.
Discussion of the results in more detail. Include crucial plots and tables only, make sure they
are legible, and that their axis labels and captions are clear and informative; each graph
should tell the reader a clear and coherent story. The text can give more details, if they
are needed, and show where the graph/table fits into the overall picture. The discussion
shouldshowhowthestatistical methodsshedlightonthedataandtheunderlyingproblem.
Tables should have appropriate numbers of digits.
Conclusions carrying the take-away message from your analysis, in your own words. Again,
this should be related to the data. Convince the reader that you know what you did and
are aware of its strengths and limitations. Sketch what more you might do, if you had
more time.
Marking scheme
Correctness Accurate, appropriate use of Incorrect,
statistical tools many errors
10 9 8 7 6 5 4 3 2 1 0 Score
Interpretation Correct, appropriate Little or none
5 4 3 2 1 0 Score
Discussion Thoughtful, imaginative Banal
5 4 3 2 1 0 Score
Quality of writing
Good grammar and punctuation, Poor grammar and
including mathematics punctuation
5 4 3 2 1 0 Score
Graphics and tables Clearly labelled, well-chosen, Poorly labelled, no
good captions and discussion, discussion, unmotivated,
appropriate numbers of digits unedited output
10 9 8 7 6 5 4 3 2 1 0 Score
Referencing Full, accurate, and detailed Inadequate citation
references given of sources
5 4 3 2 1 0 Score
Originality and scope Original choice of topic, Standard choice of dataset,
wide range of tools/ideas limited range of tools/ideas
10 9 8 7 6 5 4 3 2 1 0 Score
Grand total (max 50)
4

Time Series: Bibliography
There are many books on statistical time series analysis, in addition to the many more in other
fields. There are also two main statistical journals devoted to the area: Journal of Time Series
Analysis, and Journal of Forecasting, plus journals of finance and econometrics/economics plus
general statistical journals in which time series articles appear. So plenty of reading is available!
References
Anderson, T. W. (1994) The Statistical Analysis of Time Series (Classics Edition). Wiley-
Interscience.
Beran, J. (1994) Statistics for Long-Memory Processes. London: Chapman & Hall.
Bloomfield, P. (2000) Fourier Analysis of Time Series: An Introduction. Second edition. New
York: Wiley.
Box, G. E. P., Jenkins, G. M. and Reinsel, G. C. (1994) Time Series Analysis: Forecasting and
Control. Prentice-Hall Inc. ISBN 0-13-060774-6.
Brillinger, D. R. (1981) Time Series: Data Analysis and Theory. Expanded edition. San Francisco:
Holden-Day.
Brockwell, P. J. and Davis, R. A. (1991) Time Series: Theory and Methods. Second edition.
New York: Springer.
Brockwell, P. J. and Davis, R. A. (2002) Introduction to Time Series and Forecasting. Second
edition. New York: Springer.
Bühlmann, P. (2002) Bootstraps for time series. Statistical Science 17, 52–72.
Chan, K.-S. and Tong, H. (2001) Chaos: a Statistical Perspective. Springer-Verlag Inc. ISBN
0-387-95280-2.
Chan, N. H. (2002) Time Series: Applications to Finance. John Wiley & Sons. ISBN 0-471-
41117-5.
Chandler, R. E. and Scott, E. M. (2011) Statistical Models for Trend Detection and Analysis in
the Environmental Sciences. Chichester: Wiley.
Chatfield, C. (1996) The Analysis of Time Series. Fifth edition. London: Chapman & Hall.
Cowpertwaite, P. and Metcalfe, A. (2009) Introductory Time Series with R. New York: Springer.
Cox, D. R.(1981) Statistical analysis of timeseries: Somerecent developments (withdiscussion).
Scandinavian Journal of Statistics 8, 93–115.
Cox, D. R., Hinkley, D. V. and Barndorff-Nielsen, O. E. (eds) (1996) Time Series Models In
Econometrics, Finance and Other Fields. London: Chapman & Hall.
Diggle, P. J. (1990) Time Series: A Biostatistical Introduction. Oxford: Clarendon Press.
Durbin, J. and Koopman, S. J. (2001) Time Series Analysis by State Space Methods. Oxford
University Press. ISBN 0-19-852354-8.
Gouriéroux, C. (1997) ARCH Models and Financial Applications. New York: Springer.
5

Harvey, A. C. (1987) Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge
University Press.
Harvey, A. C. (1993) Time Series Models. Second edition. New York: Philip Allan.
Kedem, B. (1980) Binary Time Series. New York: Dekker.
Kedem, B. and Fokianos, K. (2002) Regression Models for Time Series Analysis. John Wiley &
Sons. ISBN 0-471-36355-3.
MacDonald, I. L. and Zucchini, W. (1997) Hidden Markov and Other Models for Discrete-valued
Time Series. London: Chapman & Hall.
Percival, D. B. and Walden, A. T. (1993) Spectral Analysis for Physical Applications: Multitaper
and Conventional Univariate Techniques. Cambridge: Cambridge University Press.
Percival, D.B. andWalden, A.T.(2000) Wavelet Methods for Time Series Analysis. Cambridge:
Cambridge University Press.
Petris, D., Petrone, S. and Campagnoli, P. (2009) Dynamic Linear Models with R. New York:
Springer.
Prado, R. and West, M. (2010) Time Series: Modeling, Computation, and Inference. New York:
Springer.
Priestley, M. B. (1981a) Spectral Analysis and Time Series. (Vol. 1): Univariate Series. Academic
Press.
Priestley, M. B. (1981b) Spectral Analysis and Time Series. (Vol. 2): Multivariate Series, Prediction
and Control. Academic Press.
Priestley, M. B. (1988) Non-linear and Non-stationary Time Series Analysis. Academic Press.
ISBN 0-12-564910-x.
Robinson, G. K. (2010) Continuous time Brownian motion models for analysis of sequential
data. Applied Statistics 59, 477–494.
Shumway, R. H. and Stoffer, D. S. (2011) Time Series Analysis and its Applications, with R
Examples. Third edition. Springer.
Tong, H. (1983) Threshold Models in Non-linear Time Series Analysis. Springer-Verlag Inc.
ISBN 0-387-90918-4.
Tong, H. (1990) Non-linear Time Series: A Dynamical System Approach. Oxford: Clarendon
Press.
Tong, H. (ed.) (1995) Chaos and Forecasting. World Scientific Publications.
Tsay, R. S. (2010) Analysis of Financial Time Series. Third edition. New York: Wiley.
Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth edition. New
York: Springer.
West, M. and Harrison, J. (1997) Bayesian Forecasting and Dynamic Models. Second edition.
New York: Springer.
Zucchini, W. and MacDonald, I. L. (2009) Hidden Markov Models for Time Series: An Introduction
Using R. London: Chapman & Hall/CRC.
6