After-School Initiative's Toolkit for Evaluating

Mitigation of burden on participants and staff: For an evaluation to be sustained within a program, the evaluation must be manageable
for both the program staff collecting the data and the program participants providing the data. The survey question sets are short so that
after-school programs can measure multiple outcomes without undue burden on youth participants. The survey is easy to administer and
involves minimal training of program staff. Finally, the survey questions are straightforward so data analysis and reporting are not complex.
Measuring relevant results: Because every ASI program has unique goals and activities, not all will achieve the same outcomes. NRC
encourages program staff to use logic models to determine the outcomes that best fit with the services they provide. Measuring outcomes
in an area where little or no services are provided not only wastes valuable staff time but may also provide less than desirable results.
Sensitivity to change: Many ASI programs attempt to foster significant, positive changes in the lives of their youth participants. These
changes often occur incrementally over time, so it is important to be able to measure small changes. However, small changes are often
difficult to measure. Many of the methods commonly used to measure change violate evaluation principles for youth, (e.g., using 4 or more
scale options and using higher literacy adjectives to differentiate scale points), making it more difficult to demonstrate small increments of
change. For this reason, “post-only” questions are provided in this toolkit. “Post-only” implies that program staff administers surveys at
the end of the program or after a “significant” amount of intervention has occurred (i.e., school-based program staff may want to
administer in December and May every year). “Post-only” surveys do not include survey administration before programs begin (as there
would be in a pre-post survey). “Post-only” surveys ask program participants to report whether they have changed at the end of a program,
instead of measuring outcomes before and after an intervention and comparing the results to detect changes.
Augmenting the use of existing question sets: There are scores of surveys designed to measure many of the same outcomes proposed
for measurement in this toolkit. This toolkit was not developed to replace the use of such surveys because many of the surveys used in
academia and clinical settings have been found to be valid and reliable ways of measuring youth characteristics. If this toolkit does not
include enough questions to provide the details desired to measure a specific program focus, staff may want to consider augmenting the
question sets and creating a longer, more specific survey.
Toolkit for Evaluating Positive Youth Development 3