Framework-08-25-14_web

22CHAPTER 03: FRAMEWORK DEVELOPMENT PROCESSPriority Outcomes, Indicators and MeasuresFigure 6: Criteria for Selecting MeasuresAligned to Priority Outcomes 21• Should be reliable and consistentover space and time• Verifiable and replicable• Make use of available data/be easilymeasured• Measure what is important tostakeholders• Be diverse enough to meet therequirements of different users• Be limited in numberBased on their field of expertise in the four sectors, workgroupparticipants identified an initial list of at least three outcomesfor each sector, which was further edited and prioritized. Usingan agreed-upon criteria, the groups identified indicators ormeasures for at least three priority outcomes (where feasible) ineach sector for the following levels:• Program-level outcomes• Research-level outcomes• Policy outcomesThe framework focuses on these three levels to 1) provideguidance for program sites who want to expand their programevaluation efforts, 2) build capacity in the field for developingcommon language and outcomes at each level, 3) supportgrowth in the research field connecting program activitiesto researching associations, and causality between programactivities and outcomes in the sectors, and 4) suggest areas forpolicy development that will support programs.The differences between where program evaluation leaves offand research begins can be hard to distinguish. In general:Program evaluation - uses systematic methods to collect,analyze, and report information about activities undertaken inorder to improve, or further develop implementation. It servesas a feedback loop for program coordinators, partners, andparticipants. The program level outcomes prioritized in thisframework are intended to be easiest to measure, requiring theleast amount of resources to measure, and are site-specific. Theproject team, in scanning the literature and discussing currentfarm to school practice determined that long term outcomeevaluation, surveillance, and monitoring to examine systemicchanges is beyond the scope of evaluation efforts undertakenby an individual farm to school site – these practices needadditional support from researchers.Research - While researchers use similar methods as evaluatorsfor conducting interviews, surveys, or quantitative datacollection, unlike evaluation, the focus of research is to advancetheory or what is known about a specific topic. The researchlevel outcomes prioritized are intended to build on evaluationefforts at the program level. The project team learned fromreviewers that some schools and districts are partnering withexternal evaluators or support organizations to examine manyEVALUATION FOR TRANSFORMATION