Modeling and Multivariate Methods - SAS

Appendix A Statistical Details 673
The Usual Assumptions
Assumed Model
Most statistics are based on the assumption that the model is correct. To the extent that your model may not
be correct, you must attenuate your credibility in the statistical reports that result from the model.
Relative Significance
Many statistical tests do not evaluate the model in an absolute sense. Significant test statistics might only be
saying that the model fits better than some reduced model, such as the mean. The model can appear to fit
the data but might not describe the underlying physical model well at all.
Multiple Inferences
Often the value of the statistical results is not that you believe in them directly, but rather that they provide
a key to some discovery. To confirm the discovery, you may need to conduct further studies. Otherwise, you
might just be sifting through the data.
For instance, if you conduct enough analyses you can find 5% significant effects in five percent of your
studies by chance alone, even if the factors have no predictive value. Similarly, to the extent that you use
your data to shape your model (instead of testing the correct model for the data), you are corrupting the
significance levels in your report. The random error then influences your model selection and leads you to
believe that your model is better than it really is.
Validity Assessment
Some of the various techniques and patterns to look for in assessing the validity of the model are as follows:
• Model validity can be checked against a saturated version of the factors with Lack of Fit tests. The Fit
Model platform presents these tests automatically if you have replicated x data in a nonsaturated model.
• You can check the distribution assumptions for a continuous response by looking at plots of residuals
and studentized residuals from the Fit Model platform. Or, use the Save commands in the platform
popup menu to save the residuals in data table columns. Then use the Analyze > Distribution on these
columns to look at a histogram with its normal curve and the normal quantile plot. The residuals are not
quite independent, but you can informally identify severely non-normal distributions.
• The best all-around diagnostic tool for continuous responses is the leverage plot because it shows the
influence of each point on each hypothesis test. If you suspect that there is a mistaken value in your data,
this plot helps determine if a statistical test is heavily influenced by a single point.
• It is a good idea to scan your data for outlying values and examine them to see if they are valid
observations. You can spot univariate outliers in the Distribution platform reports and plots. Bivariate
outliers appear in Fit Y by X scatterplots and in the Multivariate scatterplot matrix. You can see trivariate
outliers in a three-dimensional plot produced by the Graph > Scatterplot 3D. Higher dimensional
outliers can be found with Principal Components or Scatterplot 3D, and with Mahalanobis and
jack-knifed distances computed and plotted in the Multivariate platform.