Theory of Statistics - George Mason University

138 1 Probability Theory
• Fn(x) is the ECDF of an iid sample of size n from distribution with CDF
F.
• f(x) = dF(x)/dx.
Foundations
I began this chapter by expressing my opinion that probability theory is an
area of pure mathematics: given a consistent axiomatic framework, “beliefs”
are irrelevant. That attitude was maintained throughout the discussions in
this chapter. Yet the literature on applications of probability theory is replete
with interpretations of the meaning of “probability” by “frequentists”, by “objectivists”,
and by “subjectivists”, and discussions of the relative importance
of independence and exchangeability; see, for example, Hamaker (1977) and
de Finetti (1979), just to cite two of the most eloquent (and opinionated) of
the interlocutors. While the airing of some of these issues may just be furious
sound, there are truly some foundational issues in application of probability
theory to decisions made in everyday life. There are various to statistical inference
that differ in fundamental ways (as alluded to by Hamaker (1977) and
de Finetti (1979)) in whether or not prior “beliefs” or “subjective probabilities”
are incorporated formally into the decision process.
While the phrase “subjective probability” is current, that concept does not
fall within the scope ot this chapter, but it will be relevant in later chapters
on statistical applications of probability theory.
There are, however, different ways of developing the concept of probability
as a set measure that all lead to the same set of results discussed in this
chapter. I will now briefly mention these alternatives.
Alternative Developments of a Probability Measure
Probability as a concept had been used by mathematicians and other scientists
well before it was given a mathematical treatment. The first major
attempt to provide a mathematical framework was Laplace’s Théorie Analytique
des Probabilités in 1812. More solid advances were made in the latter
half of the 19th Century by Chebyshev, Markov, and Lyapunov at the
University of St. Petersburg, but this work was not well known. (Lyapunov
in 1892 gave a form of a central limit theorem. He developed this in the
next few years into a central limit theorem similar to Lindeberg’s, which
appeared in 1920 in a form very similar to Theorem 1.58.) Despite these developments,
von Mises (v. Mises) (1919a) said that “probability theory is not
a mathematical science” (my translation), and set out to help to make it
such. Indicating his ignorance of the work of both Lyapunov and Lindeberg,
von Mises (v. Mises) (1919a) gives a more limited central limit theorem, but
von Mises (v. Mises) (1919b) is a direct attempt to give a mathematical meaning
to probability. In the “Grundlagen” he begins with a primitive concept of
collective (or set), then defines probability as the limit of a frequency ratio,
Theory of Statistics c○2000–2013 James E. Gentle