PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
2 Statistical Inference
2.1 Basic definitions and terminology
Probability is concerned partly with the problem of predicting the behavior of the RV
X assuming we know its distribution.
Statistical inference is concerned with the inverse problem:
Given data x 1 , x 2 , . . . , x n with unknown CDF F (x), what can we conclude about
F (x)
In this course, we are concerned with parametric inference. That is, we assume F
belongs to a given family of distributions, indexed by the parameter θ:
where Θ is the parameter space.
Examples
I = {F (x; θ) : θ ∈ Θ}
(1) I is the family of normal distributions:
θ = (µ, σ 2 )
Θ = {(µ, σ 2 ) : µ ∈ R, σ 2 ∈ R + }
then I =
{
F (x) : F (x) = Φ
( x − µ
σ 2 )}
.
(2) I is the family of Bernoulli distributions with success probability θ:
Θ = {θ ∈ [0, 1] ⊂ R}.
In this framework, the problem is then to use the data x 1 , . . . , x n to draw conclusions
about θ.
Definition. 2.1.1
A collection of i.i.d. RVs, X 1 , . . . , X n , with common CDF F (x; θ), is said to be a
random sample (from F (x; θ)).
Definition. 2.1.2
Any function T = T (x 1 , x 2 , . . . , x n ) that can be calculated from the data (without
knowledge of θ) is called a statistic.
Example
The sample mean ¯x is a statistic (¯x = 1 n
n∑
x i ).
i=1
77