Evaluating Country Programmes - OECD Online Bookshop

More documents

Recommendations

Info

$CQE=U]^\]Z: KAZAKHSTAN - OECD Online Bookshop$

OECD 1999 Country Programme Evaluation: A State of the Art Review Box 2.4. Examples of different categorisation systems used to judge country programme performance in the UNDP Different UNDP evaluations have used different systems. Sri Lanka: self-evaluation of overall programme performance using scoring In evaluating components of the Sri Lanka programme, the evaluators asked country programme staff to score UNDP performance in 13 "sub-themes" of the programme (in terms of contribution to, for example, "high and sustained economic growth" or “a leaner, more efficient and service-oriented public administration") according to the following guideline: 5. Made substantial progress. 4. Made some progress. 3. No difference made. 2. Negatively affected progress somewhat. 1. Negatively affected progress substantially. The scores given were then averaged for each sub-theme (UNDP/Sri Lanka, 1994: 17, 46-7). Myanmar: external evaluation using classification In an earlier evaluation of the programme in Myanmar, 51 projects were each classified as satisfactory, unsatisfactory or partially satisfactory/partially unsatisfactory (UNDP/Myanmar, 1992: 29). In this case, the judgement was made by the evaluation team rather than the implementing staff. Broad conclusions about country programme effectiveness were guided by the proportion of projects which were judged (for example) successful, unsuccessful or partially successful. If projects are classified or scored, the key question is who provides the classification or scoring. An individual evaluator or small team of evaluators making their own decisions about project performance (effectiveness or impact) has the advantage of greater consistency (i.e. their subjective judgements are at least consistent, so that “good” means the same in all cases). However, evaluators will generally not be able to acquire the depth of knowledge required to make an informed judgement of each and every element within the country programme portfolio, at least not within the timeframe of the typical CPE. More commonly, they ask those involved in the country programme to classify or score those projects with which they are familiar. 79
Evaluating Country Programmes 80 This approach is seen in the IDB’s recent client surveys (IDB/Ecuador; IDB/ Dominican Republic). In each CPE, around one hundred “clients” (staff of the partner country government financial institutions who worked on IDB-financed projects) completed a long questionnaire. This asked them to categorise and prioritise IDB aid instruments and to suggest possible alternatives. An evaluation of EU aid to Ethiopia used pairwise ranking to identify aspects of donor performance and matrix scoring to rate EU performance against other donors. Groups of stakeholders (Ethiopian officials, donor staff, NGO and relief agency staff and EU Delegation Advisors) were invited to compare pairs of donors and specify an aspect of performance in which one performed better than the other. They then rated EU country programme performance on each of these criteria from one (below average for donors to Ethiopia) to three (above average) using matrix scoring. Comparators and counter/factuals Having established the conceptual nature of criteria by which country programme strengths and weaknesses are to be judged and identified means to obtain and present information on these performance criteria, it is then necessary to establish a basis for comparison. Should the achievements of the country programme be compared to other country programmes executed by the same donor; to the country programmes of other donors in the same country (UNDP, 1997: 3); or against a counter-factual (what would have occurred in the absence of the country programme)? There are strengths and weaknesses to each approach. If it is possible to find another donor with a similar programme operating in the same country, this may be a major focus of the evaluation. This may be hard given that country programme mix, priorities, target areas or target groups may differ considerably between donors operating in any given country. Often, however, it is possible to identify a suitable comparator for certain activities or sectors. This should be attempted whenever possible. Few of those CPEs we reviewed attempted any such comparison. The counterfactual case (what would have happened without the country programme) is extremely hard to determine. The World Bank notes that relating country performance to Bank assistance is: “a difficult topic, especially when the Bank’s influence on policy reform is shared with the Fund and Government, when its lending role is overtaken by international capital markets and other donors,…and when there are numerous external shocks.” (World Bank 1999: 4) The temptation is to compare the country situation “before” and “after” the period of donor assistance and use this to make broad conclusions about how the situation “without” aid would have compared to that “with”. This is not a satisfactory OECD 1999
Page 1 and 2:
ORGANISATION FOR ECONOMIC CO-OPERAT
Page 3 and 4:
ORGANISATION FOR ECONOMIC CO-OPERAT
Page 5 and 6:
Evaluating Country Programmes 4 The
Page 7 and 8:
Evaluating Country Programmes 6 SDC
Page 9 and 10:
OECD 1999 Glossary of Acronyms and
Page 11 and 12:
Introduction OECD 1999 Chapter 1 Co
Page 13 and 14:
Rationale, purpose and use OECD 199
Page 15 and 16:
OECD 1999 Country Programme Evaluat
Page 17 and 18:
Page 19 and 20:
Page 21 and 22:
DONOR INFLUENCES Evaluation skills
Page 23 and 24:
Page 25 and 26:
Page 27 and 28: OECD 1999 Country Programme Evaluat
Page 41 and 42: 13.00-14.30 Lunch break 14.30-15.30
Page 45 and 46: OECD 1999 FINLANDE/FINLAND Ms. Eeva
Page 47 and 48: OECD 1999 PORTUGAL Mr. Mario Mendes
Page 49 and 50: Mr. Wolfgang GRUBER Director, Proje
Page 53 and 54: Evaluating Country Programmes 54 It
Page 55 and 56: OECD 1999 56 Mali . . . . . . . . .
Page 57 and 58: Evaluating Country Programmes 58 -
Page 59 and 60: Evaluating Country Programmes 60 po
Page 61 and 62: Evaluating Country Programmes 62 Bo
Page 63 and 64: Evaluating Country Programmes 64 -
Page 65 and 66: Evaluating Country Programmes 66 In
Page 67 and 68: Evaluating Country Programmes 68 Ti
Page 71 and 72: Evaluating Country Programmes 72 Re
Page 73 and 74: Evaluating Country Programmes 74 pe
Page 75 and 76: Evaluating Country Programmes 76 va
Page 77: Evaluating Country Programmes 78 (e
Page 81 and 82: Evaluating Country Programmes 82 ou
Page 83 and 84: Evaluating Country Programmes 84 Wi
Page 87 and 88: Evaluating Country Programmes 88 Ap
Page 89 and 90: OECD 1999 Appendix 2.3 Complete Lis
Page 91 and 92: OECD 1999 93 Complete list of known
Page 93 and 94: OECD 1999 95 Complete list of known
Page 95 and 96: OECD 1999 The CPE review instrument
Page 97 and 98: OECD 1999 Appendix 2.5 List of Dono
Page 101 and 102: OECD 1999 Checklist of questions (c
Page 103 and 104: Country programme evaluations - See
Page 105 and 106: Summary OECD 1999 Chapter 3 Country
Page 107 and 108: Background OECD 1999 CPE in the Net
Page 109 and 110: CPE in the Netherland’s Developme
Page 111 and 112: OECD 1999 CPE in the Netherland’s
Page 115 and 116: Methodology OECD 1999 CPE in the Ne
Page 121 and 122: Evaluating Country Programmes 124 P
Page 123 and 124: Evaluating Country Programmes 126 G
Page 125 and 126: Evaluating Country Programmes 128 R
Page 127 and 128: Evaluating Country Programmes 130 3
Page 129 and 130:
Evaluating Country Programmes 132 P
Page 131 and 132:
Evaluating Country Programmes 134 Y
Page 133 and 134:
Evaluating Country Programmes 136 N
Page 135 and 136:
Evaluating Country Programmes 138 O
Page 137 and 138:
Evaluating Country Programmes 140 p
Page 139 and 140:
Evaluating Country Programmes 142 R
Page 141 and 142:
Chapter 5 Country Assistance Evalua
Page 143 and 144:
Country Assistance Evaluation in th
Page 145 and 146:
OECD 1999 Country Assistance Evalua
Page 147 and 148:
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Traditional tools: relevance, effic
Page 155 and 156:
Measuring sustainability OECD 1999
Page 157 and 158:
Page 159 and 160:
Rating country strategies and assis
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
OECD 1999 Chapter 6 French Experien
Page 171 and 172:
OECD 1999 French Experience with Co
Page 173 and 174:
OECD 1999 French Experience with Co
Page 175 and 176:
French Experience with Country Prog
Page 177 and 178:
OECD 1999 Chapter 7 Issues for Coun
Page 179 and 180:
Issues for Country Evaluation: The
Page 181 and 182:
Issues for Country Evaluation: The
Page 183 and 184:
OECD 1999 Issues for Country Evalua
Page 185 and 186:
Introduction OECD 1999 Chapter 8 Co
Page 187 and 188:
OECD 1999 Country Assistance Strate
Page 189 and 190:
Page 191 and 192:
Country Assistance Strategies as a
Page 193 and 194:
Page 195 and 196:
Page 197 and 198:
Page 199 and 200:
Page 201 and 202:
Page 203 and 204:
Introduction OECD 1999 Chapter 9 Th
Page 205 and 206:
Page 207 and 208:
Page 209 and 210:
Page 211 and 212:
Executive summary OECD 1999 Chapter
Page 213 and 214:
OECD 1999 Country Programme Plannin
Page 215 and 216:
Page 217 and 218:
Page 219 and 220:
Page 221 and 222:
Page 223 and 224:
Page 225 and 226:
Page 227 and 228:
OECD 1999 Chapter 11 Real Progress:
Page 229 and 230:
Real Progress: Fifty Years of USAID
Page 231 and 232:
Page 233 and 234:
OECD 1999 Real Progress: Fifty Year
Page 235 and 236:
Page 237 and 238:
Page 239 and 240:
Page 241 and 242:
Page 243 and 244:
Page 245 and 246:
Page 247 and 248:
Page 249 and 250:
Assessment OECD 1999 Real Progress:
Page 251 and 252:
Page 253 and 254:
Page 255 and 256:
Sectoral successes: activities with
Page 257 and 258:
Evaluation of macro-level performan
Page 259 and 260:
Page 261 and 262:
Page 263 and 264:
Page 265 and 266:
Page 267 and 268:
Page 269 and 270:
OECD 1999 Appendix 11.1 Sectoral an
Page 271 and 272:
OECD 1999 Bibliography Real Progres
show all

Evaluating Country Programmes - OECD Online Bookshop

Create successful ePaper yourself

Delete template?

Save as template?