After-School Initiative's Toolkit for Evaluating

More documents

Recommendations

Info

Toolkit for Evaluating Positive Youth Development 4 Benefits of Using the Toolkit This toolkit offers ASI program staff the evaluation materials necessary to measure many of the outcomes they plan to achieve. The toolkit contains question sets and tracking forms that are straightforward in measuring a multitude of youth outcomes in a simple fashion. Other benefits of using the toolkit include: Specific question sets for the After-School Initiative programs: This toolkit was developed specifically for ASI programs, taking into account the types of services provided and the youth served. Many question sets in the existing literature were developed for other purposes and for other populations. Timesaving: It is easier to use existing question sets than to reinvent the wheel. Often, after-school programs do not institutionalize evaluation systems because staff gets bogged down creating and/or selecting the best measurement question set. By selecting outcomes of interest to a program, staff can create a customized survey with relatively little work. Pooled data: ASI programs will have a way to compare their results to other programs in the initiative. After analyzing the data, NRC will send each grantee a customized report that compares the responses of youth served by the grantee’s program to responses of all youth served by the After-School Initiative.
Using This Toolkit As part of their evaluation technical assistance to ASI grantees, NRC provides an evaluation liaison for each ASI program. The evaluation liaison serves as a consultant to guide and assist program staff to develop a program logic model, identify the program’s youth development outcomes, develop a survey and survey administration protocol, and understand and use the results of the survey. With their evaluation liaison, staff at each after-school program should review the outcome domains and question sets provided in this toolkit and determine whether they are applicable to their program. In addition, staff should consider whether the question sets reflect the youth development outcomes that individual programs are working toward according to their logic model. Once outcome domains and question sets are chosen, NRC, through the evaluation liaisons’ communication with ASI programs, will customize surveys for programs electronically. Liaisons will help ASI programs determine the most appropriate dates for survey administration and develop protocol for administration. Then, after program staff administers the surveys to their youth participants, completed surveys will be sent to NRC for analysis. Finally, within a few months of administration, evaluation liaisons will meet with their assigned ASI programs to review the data results and discuss ways of using their data for program improvement. Recruitment, Incentives and Consent Determine who should be recruited to complete the survey: Every youth in fourth through ninth grades participating in after-school programs funded by ASI is eligible to be a survey respondent. Whenever possible, all eligible participants should be asked to complete the survey. If resources permit, it is easier to survey all participants to avoid designing special data collection methods that survey only a representative sample of participants. If resources do not permit, the evaluation liaison should help program staff determine a sampling method. Appendix II provides instructions for developing a sampling scheme for those programs unable to survey all participants. Decide if incentives will be offered: Program staff needs to decide if incentives will be offered as part of the survey administration process. Appendix III provides more information on the use of incentives to increase participation in evaluation. Obtain consent and assent for the evaluation: The consent and assent forms are designed to protect evaluation participants by informing them of their rights as well as any risks and benefits that may occur as a result of participation in the evaluation. Participants must understand what the evaluation is about and what they are being asked to do. Program staff is required to have a signed current consent and assent form on file for each participant in the evaluation. Before an ASI program collects data from youth – even if it is anonymous – program staff must have a consent form signed by a parent or guardian of each youth on file. The consent form is intended to describe the risks and benefits of participation in the evaluation and to give Toolkit for Evaluating Positive Youth Development 5
Page 1 and 2: THE AFTER-SCHOOL INITIATIVE’S Too
Page 3 and 4: Table of Contents INTRODUCTION ....
Page 5 and 6: Introduction The After-School Initi
Page 7: Mitigation of burden on participant
Page 11 and 12: Guarantee anonymity or confidential
Page 13 and 14: • Who is responsible for gatherin
Page 15 and 16: The following is a checklist of tas
Page 17 and 18: The Survey Administration The surve
Page 19 and 20: Problem situations during survey ad
Page 21 and 22: Outcome Domain Outcome Indicators A
Page 23 and 24: Academic Success Many after-school
Page 25 and 26: Arts and Recreation Many youth prog
Page 27 and 28: Cultural Competency As communities
Page 29 and 30: Life Skills Life skills are profici
Page 31 and 32: Positive Life Choices Although the
Page 33 and 34: Section 2: Program Quality Toolkit
Page 35 and 36: ASI requirement: All program qualit
Page 37 and 38: Section 3: Youth Background and Pro
Page 39 and 40: Section 4: Alternative to Post-only
Page 41 and 42: Review of positive youth developmen
Page 43 and 44: Reliability of Toolkit Domains Doma
Page 45 and 46: Appendix III. The Use of Incentives
Page 47 and 48: [ASI program staff may delete some
Page 49 and 50: Child Assent to Participate in the
Page 51 and 52: [ASI program staff may delete some
Page 53 and 54: Asentimiento de jovenes para partic
Page 55 and 56: Outcome Domain Community Involvemen
Page 57 and 58: Outcome Domain English Post-only Qu
Page 59 and 60:
Outcome Domain Sense of Self Englis
Page 61 and 62:
7) Who will assist the survey admin
Page 63 and 64:
Toolkit Outcome Domains Toolkit Out
Page 65 and 66:
Toolkit Outcome Domains Toolkit Out
Page 67 and 68:
Toolkit Domains Toolkit Outcome Ind
Page 69 and 70:
TABLE 3 - Toolkit Outcome Domains L
Page 71 and 72:
Toolkit Domains Toolkit Outcome Ind
Page 73 and 74:
Outcome Domain Advanced Question a
Page 75 and 76:
Outcome Domain Advanced Question I
Page 77 and 78:
Background Information Advanced Que
Page 79 and 80:
CSAP Tracking Form Student ID Readi
Page 81 and 82:
Attendance Tracking Form Student ID
Page 83 and 84:
Appendix IX. Complete Post-only Too
Page 85 and 86:
[Academic Success - For ASI grantee
Page 87 and 88:
[Life Skills - For ASI grantees cho
Page 89 and 90:
17) What grade are you in? ________
Page 91 and 92:
Page 93 and 94:
[Life Skills - For ASI grantees cho
Page 95 and 96:
17) ¿En qué grado estás? _______
Page 97 and 98:
Flowchart for Selecting Post-only o
Page 99 and 100:
Outcome Domain Post-Only Question P
Page 101 and 102:
Outcome Domain Post-Only Question P
Page 103 and 104:
Appendix XI. Complete Pre-Post Tool
Page 105 and 106:
Page 107 and 108:
[Positive Core Values - All questio
Page 109 and 110:
[NAME OF ASI PROGRAM/AGENCY] ENCUES
Page 111 and 112:
[Cultural Competency - For ASI gran
Page 113 and 114:
[Questions a and b are required if
Page 115 and 116:
REFERENCES 1 The International Coun
show all

After-School Initiative's Toolkit for Evaluating

Create successful ePaper yourself

Delete template?

Save as template?