Statistics 300A Syllabus Theoretical Statistics Fall 2005 Lectures ...

Statistics 300A Syllabus
Theoretical Statistics
Fall 2005
Lectures: The class meets MWF from 11:00 to 11:50 in Sequoia Hall, Room
200. Instructor: Joe Romano. Office: 142 Sequoia Hall. Email: romano@stanford.edu
Office phone number: 723-6326. Office hours are tenatively scheduled for Monday
and Wednesday from 12:00 to 1:00, and on Friday from 10:00-10:50.
Prerequisites: Probability Theory (at the least Statistics 116), some mathematical
analysis, and an introductory course in statistics.
Texts: Theory of Point Estimation (TPE), second edition, by Lehmann and
Casella, published by Springer. Testing Statistical Hypotheses (TSH), third
edition, by Lehmann and Romano, published by Springer. We will start with
TPE.
Teaching Assistants:
Wei Zhen, zhenwei@stanford.edu, Office 238, Phone 725-5952; Monday 2:30-
4:30.
Hua Zhou, hwachou@stanford.edu, Office 206, Phone 725-6148. Tuesday 12-2.
Guoqiang Hu, hgq@stanford.edu, Office 141, Phone 725-2223. Wednesday 2-4.
Grading: Your grade will be determined by weekly problem sets (approximately
one quarter weight), a midterm (roughly one quarter weight), and a
final exam (roughly half weight). The final exam is on Wednesday, December
14 from 8:30 to 11:30 in the morning.
Course Contents: This course will focus on the finite sample theory of statistical
inference, emphasizing estimation, hypothesis testing, and confidence intervals.
We will systematically develop optimal statistical procedures for many
statistical models. Such procedures include: uniformly minimum variance unbiased
estimators, minimum risk equivariant estimators, Bayes estimators, and
minimax estimators. The Neyman-Pearson theory of hypothesis testing will
be presented, with special emphasis on the role of conditioning to eliminate
nuisance parameters, and the construction of optimal invariant tests.
We will begin with the theory of point estimation, and cover much of Chapters
1-5 in TPE. The hypothesis testing material will be drawn from TSH, Chapter
3 and as much of Chapters 4-6 as we can cover.
More generally, Stat300ABC will eventually cover most of the following:
I. Elementary Finite Sample Theory of Point Estimation: Statistical Models;
Sufficiency; Applications to Exponential Families, Group Families, and
Nonparametric Families; Minimum Risk Unbiased Estimation; Minimum Risk
Equivariant Estimation; Cramer-Rao Inequality.
II. Elementary Decision Theory: Loss and Risk Functions; Bayes Estimation;
Minimax Estimation; Admissibility; Shrinkage Estimators.
1

III. Finite Sample Theory of Hypothesis Testing and Confidence Intervals: Neyman-
Pearson Theory; Uniformly Most Powerful Tests and Uniformly Most Accurate
Confidence Intervals for Distributions with Monotone Likelihood Ratio; Systematic
Use of Sufficiency and Conditioning to Eliminate Nuisance Parameters
in Exponential Families; Use of Invariance to Eliminate Nuisance Parameters
in Group Families; Maximin Tests and the Hunt-Stein Theorem; Permutation
Tests.
IV. Elementary Large Sample Estimation Theory: Asymptotic Relative Efficiency;
Maximum Likelihood Estimation; Chi-squared tests; Rao, Wald, and
Likelihood Ratio Tests; Delta Method; Asymptotic Distribution of Quantiles
and Trimmed Means; Differentiability of Statistical Functionals; Robustness
and Influence. Rank, Permutation and Randomization Tests; Jackknife, Bootstrap
and Sample Reuse Methods.
V. Asymptotic Optimality Theory of Estimators, Tests, and Confidence Intervals:
Contiguity, Quadratic Mean Differentiability, Expansions of the log likelihood
ratio. Convergence to a Gaussian experiment, Asymptotic Minimax Theorem,
Convolution Theorem, Asymptotically Uniformly Most Powerful Tests,
etc.
VI. Density Estimation: Kernel Density Estimation; Bias versus Variance Tradeoff;
Choice of Bandwidth and Kernel.
VII. Time Series: First and Perhaps Second Order Autoregressive Processes;
Conditions for Stationarity; Use of Maximum Likelihood in Time Series With
Asymptotic Theory.
VIII. Other Possible Topics (Covered Briefly): Sequential Analysis; Optimal Experimental
Design; Empirical Processes With Applications to Statistics; Edgeworth
Expansions With Applications to Statistics.
Homework 1, tentatively due Monday, October 3. From TPE, Chapter
1: 1.8, 1.11, 4.13, 4.16, 5.10, 6.2, 6.3.
Homework 2, tentatively due Monday, October 10. From TPE, Chapter
1: 6.9, 6.32, 6.33, 6.35, 7.23. From TPE, Chapter 2: 1.15, 1.17.
2