Deeper_Dive_into_Afterschool

Program Evaluation and ImprovementThe Communities Organizing Resources to Advance Learning (CORAL) Initiative in Californiabegan as an effort by the James Irving Foundation to improve the academic achievement ofstudents in the lowest-performing schools. The program hired an outside evaluator, SRIInternational, to assess if they were meeting their goals and determine their impact on programparticipants. The evaluation found that the program was falling short in a number of areas.Implementation, number of participants, quality of enrichment activities and cost of theprogram were all flagged as problematic issues for the program. 109Based on the evaluation, CORAL revisited their programming approach and concluded that theyneeded to implement a more targeted educational focus. CORAL revised their goals, adopting anew, more tailored focus. In turn, the program’s curriculum was also improved to reflect theirnew goals. Along with the implementation of the revised curriculum, Public/Private Ventureswas brought on board to evaluate CORAL by measuring student participation, engagement andoutcomes.“We talk about the idea ofcontinuous improvementprocesses…It’s that ongoing lookat the program from either theagency running the program oranother intermediaryorganization to continuallyreflect on and think aboutwhether the quality elementsthat they’re looking for arepresent.”— Christina A. Russell,Researcher, Policy StudiesAssociates, Inc.A continuous program improvement cycle—asdiscussed briefly in Section II of the report—was alsoimplemented as a part of CORAL’s dedication to raisestudent achievement. To start the process, CORALestablished well-defined goals that were explicit innature and dedicated a senior staff member tosupervise and manage the quality improvementprocess. The cycle consisted of three steps: 1) stafftraining and professional development; 2) datacollection, program monitoring and staff coaching;and 3) data analysis. 110In the first part of the cycle, staff trainings weredeveloped to further the program’s goals. Frequentstaff trainings, scaffolding staff trainings and paringnew staff with experienced staff were variousstrategies employed by CORAL. Next, senior staffobserved the quality of the programming, monitored staff use of practices taught during stafftrainings, and documented their observations. Step three of the cycle included the senior staffand/or an outside evaluator reviewing and analyzing the data collected, such as thedocumented observations and student participant attendance rates and skills. Senior staff thendetermined what changes needed to be made to the curriculum, what skills the staff neededmore help with, and the types of trainings that would be most useful to staff and support theoverall goals of the program. The cycle began again with staff development, and findings fromthe data analysis were incorporated into the new staff trainings.29