Theory of Statistics - George Mason University

3.2 Statistical Inference: Approaches and Methods 247
Example 3.14 least squares in a linear model
Consider the linear model (3.7)
Y = Xβ + E,
where Y is the random variable that is observable, in the least squares setup
of equations (3.66) and (3.67) we have
Sn(β ; y, X) = y − Xβ2. (3.69)
In the case of the linear model, we have the estimating equation
sn(β ; y, X) = X T y − X T Xβ = 0. (3.70)
This system of equations is called the “normal equations”.
For the estimand g(β) = l T β for some fixed l, the least squares estimator
is l T (X T X) − X T Y , where M − denotes a generalized inverse of a matrix M.
See page 420 and the following pages for a more thorough discussion of the
linear model in this example.
Example 3.14 illustrates a very simple and common application of estimation
by fitting expected values; the expected values are those of the observable
random variable. The next example is a somewhat less common situation of
defining which expected values to focus on.
Example 3.15 estimation in a stable family; the empirical CF
Most members of the stable family of distributions are quite complicated.
In general, there is no closed form for the CDF of the PDF, and none of
the moments exist or else are infinite. The family of distributions is usually
specified by means of the characteristic function (see equation (2.30)),
ϕ(t) = exp (iµt − |σt| α (1 − iβ sign(t)ω(α, t))).
Because the characteristic function is an expected value, its sample analogue,
that is, the empirical characteristic function can be formed easily from a sample,
x1, . . ., xn:
ϕn(t) = 1
n
e
n
itxi . (3.71)
The empirical characteristic function can be computed at any point t.
The expected values would be fit by minimizing
i=1
Sn(x, r, µ, σ, α,β) = ϕn(t) − ϕ(t)r
(3.72)
for given r at various values of t.
While this is a well-defined problem (for some given values of t) and the
resulting estimators are strongly consistent (for the same reason that estimators
based on the ECDF are strongly consistent), there are many practical
Theory of Statistics c○2000–2013 James E. Gentle