Dialogue in Pursuit of Development - Are you looking for one of ...

183
Another important arena in which to look for and understand the subtle
way in which blocks are built up is that of evaluation. Let me again use a
practical example. In 1992, Sida had decided that it would be interesting to
carry out an impact evaluation of a public administration masters degree
program that was being supported in the Central American region. As part
of the assignment, I spent some time in the Sida headquarters in Stockholm
talking to colleagues from different divisions, including the evaluation group
and also digging through many different documents in the library. One of
the things that very quickly struck me was the presence of different and at
times antagonistic versions of what evaluation was, yet at the same time
how these differences were somehow accepted and tolerated within that
strange, relatively warm and open hearted, alliance of views that was Swedish
aid at that time. As a way of passing this picture to my colleagues in
Central America I suggested that the different approaches could be grouped
into three main strands, all of which had their own worldviews and were
products of identifiable circumstances (Spink, 1993). The three were: ‘being
useful’; ‘what happened – accountability and learning’; and ‘what does it
mean’. All of them were present, with different supporters and different
arguments, in different parts of Sida at that time; they were some of the
versions that were circulating about ways of attributing value.
‘Being useful’ referred to those approaches to evaluation that are more
concerned with understanding whether actors are able to do what they see
as important to them, whether what they are trying to do seems to make
sense in the circumstances, whether they were able to do it and whether it
was as useful as was thought. Key here is that there is no a priori assumption
about what ‘useful’ means and consequently of what criteria should be used
to evaluate projects. The process is interactive and based on an underlying
concern with whether the aid being provided was useful; which implies
thinking about both the aid and the way it was provided and the use that
those involved were able to make of it. These are points that open discussion
rather than close it, for they ask questions of both donors and those
receiving aid; they require both to be open about what they are hoping for
and what they see as important. The result is a kind of ‘muddling through’
oriented to the construction of a joint value frame rather than precise and
measurable goals, performance targets and check lists.
‘What happened’ approaches to evaluation and analysis begin with the
assumption that evaluation is a technical matter. Things take place that can
be measured and there are a variety of methods for doing this which can be
looked at in terms of strengths and weaknesses. Evaluation will need to take
into consideration a number of questions that will require discussion. For
example: what data is or can be made available; what the possibilities for
working with ‘end users’ are; what resources are available on both sides;
how much evaluation is needed and at what points in the process. However,
once these questions have been resolved it will be the responsibility of