Modeling and Multivariate Methods - SAS

Appendix A Statistical Details 675
Key Statistical Concepts
In business, you want to maximize revenues and minimize costs. In science you want to minimize
uncertainty. Uncertainty in science plays the same role as cost plays in business. All statistical methods fit
models such that uncertainty is minimized.
It is not difficult to visualize uncertainty. Just think of flipping a series of coins where each toss is
independent. The probability of tossing a head is 0.5, and –log( 0.5)
is 1 for base 2 logarithms. The
probability of tossing h heads in a row is simply
p
=
1-- h
2
Solving for h produces
You can think of the uncertainty of some event as the number of consecutive “head” tosses you have to flip
to get an equally rare event.
Almost everything we do statistically has uncertainty, –logp , at the core. Statistical literature refers to
uncertainty as negative log-likelihood.
The Two Basic Fitting Machines
Springs
h
= –log 2 p
An amazing fact about statistical fitting is that most of the classical methods reduce to using two simple
machines, the spring and the pressure cylinder.
First, springs are the machine of fit for a continuous response model (Farebrother, 1987). Suppose that you
have n points and that you want to know the expected value (mean) of the points. Envision what happens
when you lay the points out on a scale and connect them to a common junction with springs (see
Figure A.6). When you let go, the springs wiggle the junction point up and down and then bring it to rest at
the mean. This is what must happen according to physics.
If the data are normally distributed with a mean at the junction point where springs are attached, then the
physical energy in each point’s spring is proportional to the uncertainty of the data point. All you have to do
to calculate the energy in the springs (the uncertainty) is to compute the sum of squared distances of each
point to the mean.
To choose an estimate that attributes the least uncertainty to the observed data, the spring settling point is
chosen as the estimate of the mean. That is the point that requires the least energy to stretch the springs and
is equivalent to the least squares fit.