Guide for the Development of Results-based Management and ...

RMAF Guide August, 2001contribution of the policy, program or initiative to these achieved outcomes (oftencalled summative evaluations).The first step in developing an evaluation strategy involves identifying the issues and associated questionsthat need to be addressed during the periodic evaluations. The identification of the evaluation issues andquestions provides a guide for the development of the strategy that ensures all essential issues will beaddressed during later evaluation. A key benefit to the identification of issues at this stage is that these arethen used to elaborate a set of data requirements and data collection strategies, which, on implementation,helps to ensure that information necessary for evaluation is available when it is needed. As such, theevaluation strategy needs to be linked to the ongoing performance measurement strategy as someevaluation data requirements will be met through ongoing performance measurement activities.Evaluation issues are the broad areas which need to be explored within an evaluation while evaluationquestions are the more specific research questions that need to be answered in order to be able to addresseach evaluation issue. Some issues and questions might only be relevant during a formative evaluation,others only in the context of a summative evaluation and yet some might be relevant during both. Everypolicy, program or initiative will have its own unique set of evaluation issues as well as intended schedulefor evaluation. For example, initiatives of a limited timeframe may require a formative evaluation afterbeing in place for only a year, whereas longer-term policies, programs or initiatives may require aformative evaluation after two or three years of operation.b. ProcessAs presented in Appendix B: Evaluation Standards for the Government of Canada, of the Treasury BoardEvaluation Policy, there are three primary issue areas for evaluation that need to be considered:Relevance - Does the policy, program or initiative continue to be consistent withdepartmental and government-wide priorities, and does it realistically address an actualneed?Success - Is the policy, program or initiative effective in meeting its intendedoutcomes, within budget and without unwanted negative outcomes? Is the policy,program or initiative making progress toward the achievement of the final outcomes?Cost-Effectiveness - Are the most appropriate and efficient means being used toachieve outcomes, relative to alternative design and delivery approaches?22