A Framework for Evaluating Systems Initiatives

More documents

Recommendations

Info

IV. Developing Systems Initiative Theories of Change Theories of change are now part of mainstream evaluation practice; almost every systems initiativeevaluation is now based on one. Theories of change illustrate the pathways by which change isexpected to occur and the role that initiatives play in producing that change. They show how initiativestrategies or activities connect to interim outcomes that then set the stage for long-range impacts.Because theories of change enable evaluation stakeholders to clarify what initiatives are doing inrelation to their intended outcomes and impacts, they typically are developed early on to inform bothinitiative strategy development and evaluation design.As theories of change have become ubiquitous, they have taken on many forms. Some aredepicted visually while others are narratives. Some are linear while others are circular. Some arebroad while others are quite specific. The point here is not to prescribe a specific formula forhow a theory of change must look. Rather, it is to say that the concept that underlies a theoryof change—making explicit stakeholder notions about the relationships between an initiative’sstrategies, interim outcomes, and long-term impacts, and producing testable assumptions regardingthose relationships—is always useful, especially for complex initiatives where it can be difficult tounderstand the many strategies in play.A Theory of Change Menu for Systems InitiativesThe figure on the next page was developed to aid theory of change development for systems initiatives.It is constructed around the five systems initiative focus areas. For each focus area, the figure offersa broad description of initiative activities, possible outcomes of those activities, and what thoseoutcomes, if achieved, may produce in terms of impacts for system beneficiaries. The actual outcomesand impacts selected for a theory of change will depend on the systems initiative at hand. For example,an initiative focused on both a system’s context and its infrastructure would have a theory of changefeaturing content from both of those columns.The figure is simply a menu. Actual theories of change should detail how initiative strategies willconnect to and produce their intended outcomes and impacts. For an initiative focused on context,for example, it is important to describe or illustrate how outcomes such as public will or leadership willbe built. In other words, an initiative with multiple areas of focus might have both a broad theory ofchange that includes outcomes and impacts across focus areas, and a more detailed version that breaksout and specifies the theories of change within each focus area.The activities, outcomes, and impacts associated with each focus area are outlined in more detail belowand then illustrated using examples of real-life initiatives. Again, note that systems initiatives typicallyfocus on more than one of these areas simultaneously.7A Framework for Evaluating Systems Initiatives
A Theory of Change Menu for Systems Initiative CONTEXTCOMPONENTS CONNECTIONS INFRASTRUCTURESCALE• Improving the• Establishing• Creating strong• Developing the• Ensuring apolitical contexthigh-performanceand effectivesupports systemscomprehensiveImpacts Outcomes Activitiesthat surrounds thesystem so itproduces the policyand fundingchanges neededto create andsustain it• Recognition ofsystem need• Shared vision• Leadership• Public engagement• Media coverage• Public will• Political will• Policy changes• Initiatives typicallyare not expectedto demonstratehow context-relatedoutcomes causallyconnect toprograms andservices withinthe system thatproduce resultsfor systembeneficiaries• New systemprograms orservices• Expanded programreach or coverage• Improved programquality• Increasedoperationalefficiancy• Beneficiaryoutcomes thatprecede impacts• Better impacts forbeneficiaries relatedto specific programsor practiceslinkagesacross systemcomponents thatfurther improveresults for systembeneficiaries• Shared goals• MOUs acrosssystems• Shared standards• Cross-systemtraining• Sharedcompetencies orskills standards• Shared datasystems• Referrals/follow ups• Seamless services• Better impacts forbeneficiaries whereor whenconnections aremade compared towhen they are notneed to functioneffectively andwith quality• Cross-systemgovernance• Less categoricaland more flexiblefunding• Leveraged use offunding• Mechanismsfor two-waycommunication• System-wide useof data• Practitioner supports• Initiatives typicallyare not expectedto demonstratehow infrastructureoutcomes causallyconnect tosystem is availableto as many peopleas possible so itproduces broadand inclusiveresults for systembeneficiaries• System spread• System depth• Systemsustainability• Shifts in systemownership• Beneficiaryoutcomes thatprecede impacts• Better impacts forbeneficiaries acrossa broad spectrumof domains andon a system-widepopulation levelbeneficiary impactsbeneficiary impacts(e.g., on communityor state indicators)8A Framework for Evaluating Systems Initiatives
Page 1 and 2: A Framework For Evaluating Systems
Page 3: Purpose of this PaperThis paper dra
Page 6 and 7: III. Defining Systems Initiatives:
Page 10 and 11: 1 ContextInitiatives focused on con
Page 12 and 13: • New programs or services develo
Page 14 and 15: 4 InfrastructureSometimes a system
Page 16 and 17: OutcomesOutcomes for systems initia
Page 18 and 19: An Evaluation Design Menu for Syste
Page 20 and 21: Because they interact with the poli
Page 22 and 23: ExamplesThe Build Initiative in Pen
Page 24 and 25: 4 Evaluating InfrastructureSystems
Page 26 and 27: Evaluation MethodologiesEvaluating
Page 28 and 29: VI. Applying the Framework: The Bui
Page 30 and 31: Identifying Evaluation Options: The
Page 32 and 33: 2) Base evaluation decisions on the
Page 34 and 35: 8) Hold some systems initiatives, b
Page 36: www.buildinitiative.orgThe Heinz En

A Framework for Evaluating Systems Initiatives

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?