¿MISION CUMPLIDA? EVALUACIÓN DEL PROGRAMA DE ... - Novib

¿Misión cumplida? Evaluación Programa ON-LAC
Appendix: Methodological criteria and limitations
The judgment criteria for this evaluation were based on the OECD Guidelines (2002)
formulated for development interventions, in particular those applicable to the relevance
and effectiveness of the ON-LAC Programme. Relevance refers to the extent to which the
Programme intervention has corresponded to the requirements of target groups, country
needs, global priorities and donor policies. This criterion has been applied over the time
span of the period 2003 – 2009, covered by this evaluation, although it should be
recognised that a large part of the Oxfam Novib policies had been put in place before the
start of this period. In this report, the relevance of the ON-LAC Programme has been
assessed through an analysis of economic, social and political trends in a number of Latin
American countries over the last decade. Related information refers to the area of human
development and socio-economic inequality, but also to variables traditionally important
to the intervention strategy of Oxfam Novib. At the national level, these include per capita
income and income distribution, food security, gender development, as well as the
functioning of democracies and civil security. Within the analytical framework defined and
in conformity with OECD standards, country information allows for assessing the
relevance of the Programme at different spots in time, especially at the start and end of the
evaluation period.
Effectiveness refers to the extent to which the Programme objectives are achieved or
expected to be achieved, taking into account their relative importance. The weight of this
criterion has greatly increased in the course of the evaluation period itself, given the
pressure for accountability on the use of public resources for development cooperation. By
use of both existing methods (logical framework) and new instruments (Toolbox), the ON-
LAC Programme has been reframed with a view toward making its objectives specific,
measurable, achievable, relevant and time-bound (SMART). However, when applying
these criteria, the evaluation encountered a set of limitations, including the following:
• Result chains within projects, where inputs, activities and outputs are more difficult
to structure when interventions are planned at different levels. The issue in particular
relates to Oxfam Novib’s financing modalities, which have largely been geared
towards (strategic) institutional rather than specific (measurable) project support.
• Homogeneity of results, which is likely to decrease with an increasing variety of
activities by counterparts, the number of projects themselves and the involvement of
counterparts in various Aims of the Oxfam Novib strategy.
• Aggregating or combining data: the validity of cross-section or internationally
comparable results is likely to decrease with an increased scope of intervention and
lesser tangibility of results produced in different national settings. This hampers the
step from project to programme assessments.
• Availability and reliability of information: the quality of data becomes more
heterogeneous as the procedures and instruments for their collection vary in scope,
source and period. Data interpretation then becomes dependent on diverse methods
of data gathering.
• Partner policies. Neither last nor least, the counterparts which have been
collaborating with Oxfam Novib in implementing the Programme are likely to affect
the measurement of the effectiveness of contributions, since the latter have
amalgamated autonomous counterpart policies with a variety of inputs and in
diverging scale from other sources.
The above factors have reduced the scope for assessing the effectiveness of the
Programme. By consequence, the evaluation in this regard cannot lead to a discrete
verdict. Effectiveness should instead be viewed on the basis of the plausibility that Oxfam
Novib’s strategic objectives have been achieved over the period, taking into account the
18