Theory of Statistics - George Mason University

3.1 Inferential Information in Statistics 211
An additive model has the advantage of separability of first order expectations
of the two components no matter what assumptions are made about
joint probability distributions of elements of the one component and those of
the other. Note a questionable requirement for this separability: the variance
of the residual component must be constant no matter what the magnitude
of the expectation of the systematic component. Despite these issues, in the
following, we will concentrate on models with additive random effects.
In the case of the black-box model (3.8), both the systematic and random
components are embedded in the box. The objectives of statistical analysis
may be to identify the individual components or, more often, to determine
“average” or “most likely” output Y for given input x.
Because the functional form f of the relationship between Y and x may
contain a parameter, we may write the equation in the model as
Y = f(x; θ) + E, (3.12)
where θ is a parameter whose value determines a specific relationship within
the family specified by f. In most cases, θ is a vector. In the usual linear regression
model, for example, the parameter is a vector with two more elements
than the number of elements in x,
where θ = (β0, β, σ 2 ).
3.1.1 Statistical Inference: Point Estimation
Y = β0 + x T β + E, (3.13)
Statistical inference is a process of identifying a family of distributions that
generated a given set of observations. The process begins with an assumed
family of distributions P. This family may be very large; for example, it may
be the family of distributions with probability measures dominated by the
counting measure. Often the assumed family is narrowly defined; for example,
it may be the family of univariate normal distributions. In any event, the
objective of statistical inference is to identify a subfamily, PH ⊆ P, that
contains the population from which the data arose.
Types of Statistical Inference
There are various types of inference related to the problem of determining
the specific Pθ ∈ P or else some specific element of θ. Some of these are
point estimation, hypothesis tests, confidence sets, and function estimation
(estimation of the PDF, for example). Hypothesis tests and confidence sets
are associated with probability statements that depend on Pθ.
Beginning on page 12 and later in Chapter 2, we have distinguished families
of probability distributions as either parametric or nonparametric. In
Theory of Statistics c○2000–2013 James E. Gentle