Community Needs Assessment Process

follow up an exploratory analysis with a confirmatory analysis in the same dataset. An
exploratory analysis is used to find ideas for a theory, but not to test that theory as well.
When a model is found exploratory in a dataset, then following up that analysis with a
confirmatory analysis in the same dataset could simply mean that the results of the
confirmatory analysis are due to the same type 1 error that resulted in the exploratory
model in the first place. The confirmatory analysis therefore will not be more informative
than the original exploratory analysis.
Stability of Results
It is important to obtain some indication about how generalizable the results are. While
this is hard to check, one can look at the stability of the results. Are the results reliable
and reproducible? There are two main ways of doing this:
Cross-validation: By splitting the data in multiple parts we can check if an
analysis (like a fitted model) based on one part of the data generalizes to another
part of the data as well.
Sensitivity analysis: A procedure to study the behavior of a system or model
when global parameters are (systematically) varied. One way to do this is with
bootstrapping.
Page 63 of 137