Participatory Evaluation of our 2008 - Action Against Hunger

ut none of the monitoring conducted in 2008 included data necessary for reporting or analysis.
Most of the programme monitoring was in fact conducted on behalf of FAO, concentrating on crop
growth, with data collection and entry completed in the field and a spreadsheet emailed to FAO
(unfortunately this did not include yield data). ACF also monitored the programme, concentrating
entirely on quantitative cumulative progress of for example number of groups formed and bank
accounts opened, types of enterprises selected, acres planted, number of people trained,
additional assets constructed, number of household animals and symbolic assets, etc. Some of the
above might have contributed to an evaluation, but certainly not to analysing changes in
purchasing power, for example, which was the strongest statement of impact forecast in the
proposal and inception report. Conducting this evaluation closer to the programme conclusion may
have facilitated greater access to some of these data, but this proved to be surprisingly difficult.
Aside from the fact that hard copy files had been stored already, there appeared little staff
knowledge about what might be found in the files – such as whether they contained copies of
group profitability analyses or work plans. Some of the electronic data were found, but it was
evident that – like the monitoring system on the whole – it was not stored in a manner that reflected
its anticipated access or use. Nor were there final data against which the baseline could be
contrasted. For these reasons the evaluation had to be based on predominantly external data and
a retrofit framework. This made the exercise possible but has compromised its quality and
relevance in that there are very few conclusions that can be drawn on the basis of FFS programme
monitoring in 2008.
Greater investment in initial assessment for design followed by a more strategic commitment to
baseline data collection and monitoring will ensure that information can be utilised more effectively
during implementation and evaluation. This can ensure a more efficient use of staff time in the
field, reduction in the costs associated with monitoring (like additional photocopying, vehicle travel,
data entry, etc), and ultimately better reporting. Planning for M&E also helps field staff understand
the content, purpose, and use of the information they are collecting. Recent ACF experience
following the FFS programme suggests that greater field staff involvement in all aspects of
monitoring has improved the accuracy and reliability of their data collection. Greater foresight in
data collection, management, and use will also help support ACF’s planned surveillance activities
and encourage greater comparability of data within and across departments.
The method of initial assessment and baseline data collection, depending on how approached,
might even be able to contribute to the originally intended participatory nature of FFS M&E. Any
one of the RRA tools used during the evaluation fieldwork could generate participant perspectives
on a range of issues pertaining to practices, production, gender, assets, transmission, etc. The
specific criteria they use, the amounts they indicate, the priorities they establish could all be
channelled into more meaningful indicators and monitoring. The same could prove true for a
modified version of matrix scoring for profitability analysis and enterprise selection, as the tool by
necessity would establish a range of criteria and preferences that could be evaluated for
achievement at the end of the programme.
4.11 Targeting based on vulnerability and integrated criteria
It is difficult to establish precisely how the three programme parishes were selected, or how the
participating communities therein were identified. At household level the process and criteria
become clearer and more effective.
Geographic priorities were established partly on the basis of FSL field staff experience and where
they ‘knew’ vulnerability to food insecurity to be higher. As the programme map illustrates (Figure
1), there was geographic overlap between the FFS parishes and other ACF activities in nutrition
and WASH. Of particular interest are water user committees and village health teams, as they too
reflect elements of group approaches to management and problem solving. The FFS programme
locations were also all within the catchment area of ACF feeding centres. The evaluation team
therefore expected more apparent connections between the FFS groups and those from WASH
and nutrition, however in practice these were surprisingly few. Although not explored in depth
during each focus group session, there was always a question or two about other ACF activities or
Action Against Hunger Uganda - 42 - Farmer Field School Evaluation