GAO-12-208G, Designing Evaluations: 2012 Revision

Case Studies
Typical Designs for
Outcome Evaluations
Chapter 4: Designs for Assessing Program
Implementation and Effectiveness
In some circumstances, an evaluator may want to use case studies to
explore certain issues in more depth than can be done in more than a few
locations. In single case study evaluations, especially, much attention is
given to acquiring qualitative information that describes events and
conditions from several points of view. The structure imposed on the data
collection may range from the flexibility of ethnography or investigative
reporting to the highly structured interviews of sample surveys. (For more
on the evaluation insights to be gained from ethnography, see GAO
2003.) Case studies are often used to provide in-depth descriptive
information about how the program operates in the field. If the objective of
the case study is to describe aspects of an issue, provide context, or
illustrate findings developed from a more broadly applied survey, then
selecting a nongeneralizable sample of cases may be appropriate.
Case studies can also supplement survey or administrative data to
explore specific questions about program performance, such as
understanding variation in program performance across locations (for
example, rural versus urban settings), or to identify factors key to program
success or failure. The criteria used for selecting cases are critical to
one’s ability to apply their findings to the larger program. To heighten the
value of the information they provide, cases should be selected carefully
to represent particular conditions of interest (for example, sites with low
versus high levels of performance) and with certain hypotheses in mind.
However, most often, case studies will generate hypotheses rather than
answers to questions such as what factors influence program success.
(For more on case study methodology, see GAO 1990, Stake 1995, or
Yin 2009.) For example,
• To identify the causes of a sudden decline in control of an agricultural
pest, evaluators might conduct field observations in the localities most
affected to assess how well key components of the pest eradication
and control program were executed or whether some other factor
appeared to be responsible.
Outcome evaluations address questions about the extent to which the
program achieved its results-oriented objectives. This form of evaluation
focuses on examining outputs (goods and services delivered by a
program) and outcomes (the results of those products and services) but
may also assess program processes to understand how those outcomes
are produced. Outcome evaluations may address questions such as
Page 34 GAO-12-208G