Modeling and Multivariate Methods - SAS

116 Fitting Standard Least Squares Models Chapter 3
Restricted Maximum Likelihood (REML) Method
Unrestricted Parameterization for Variance Components in JMP
Note: Read this section only if you are concerned about matching the results of certain textbooks.
There are two different statistical traditions for parameterizing the variance components: the unrestricted
and the restricted approaches. JMP and SAS use the unrestricted approach. In this approach, while the
estimated effects always sum to zero, the true effects are not assumed to sum to zero over a particular
random selection made of the random levels. This is the same assumption as for residual error. The
estimates make the residual errors have mean zero, and the true mean is zero. But a random draw of data
using the true parameters is some random event that might not have a mean of exactly zero.
You need to know about this assumption because many statistics textbooks use the restricted approach. Both
approaches have been widely taught for 50 years. A good source that explains both sides is Cobb (1998,
section 13.3).
Negative Variances
Note: Read this section only when you are concerned about negative variance components.
Though variances are always positive, it is possible to have a situation where the unbiased estimate of the
variance is negative. This happens in experiments when an effect is very weak, and by chance the resulting
data causes the estimate to be negative. This usually happens when there are few levels of a random effect
that correspond to a variance component.
JMP can produce negative estimates for both REML and EMS. For REML, there are two check boxes in the
model launch window: Unbounded Variance Components and Estimate Only Variance Components.
Unchecking the box beside Unbounded Variance Components constrains the estimate to be nonnegative.
We recommend that you do not uncheck this if you are interested in fixed effects. Constraining the variance
estimates leads to bias in the tests for the fixed effects. If, however, you are interested only in variance
components, and you do not want to see negative variance components, then checking the box beside
Estimate Only Variance Components is appropriate.
If you remain uncomfortable about negative estimates of variances, please consider that the random effects
model is statistically equivalent to the model where the variance components are really covariances across
errors within a whole plot. It is not hard to think of situations in which the covariance estimate can be
negative, either by random happenstance, or by a real process in which deviations in some observations in
one direction would lead to deviations in the other direction in other observations. When random effects
are modeled this way, the covariance structure is called compound symmetry.
So, consider negative variance estimates as useful information. If the negative value is small, it can be
considered happenstance in the case of a small true variance. If the negative value is larger (the variance ratio
can get as big as 0.5), it is a troubleshooting sign that the rows are not as independent as you had assumed,
and some process worth investigating is happening within blocks.