Design, Monitoring, and Evaluation – Capacity Assessment

• On one occasion, respondent generated codes had to be altered to fit in with the codes
used by other Country Offices. Likewise, there were some alterations in the ordering of
responses that had to be modified prior to data entry
• The sample is too small for some of the analysis; hence only numbers and not
percentages are presented by country.
• The spread of projects by region / Country Offices tends to give weightage to the results
from select regions / Country Offices
• Sector information for projects is currently unavailable to use as an analysis filter,
consequently all the projects are viewed as a whole.
• Tremendous spread of projects by beneficiaries reached and lack of information on this
for all projects doesn’t allow assignment of weights.
• It appears that at times the questionnaire has been filled up by individuals rather than as
part of a group exercise. (See section 3.6: Numbers ranged from one individual to 50
project staff participating in the DME CA).
• Definitive response categories considered – only responses that were an unqualified
‘Yes’. This may tend to present a pessimistic picture, by not counting “possibly” or “to
some extent”.
• Tremendous variation in projects within a country in terms of their Design, Monitoring,
and Evaluation make it difficult to paint a true Country Office-specific picture.
• These were self-assessments by project staff. The answers represent their own
perceptions. In some cases, outside experts facilitated DME CA sessions. It is hoped
that they helped to provide some objectivity. Nevertheless, all of these results need to be
interpreted as “according to those participating in these DME CA exercises”.
• It is apparent that CARE staffs have a variety of interpretations of some DME terms.
One example is “registry” for counting participants/beneficiaries. While many projects
say their M&E MIS uses a “registry” it is doubtful they all do keep track of individuals,
especially given the size of some projects (with as many as 16.5 million beneficiaries).
• Narrative summaries were available for a sub-sample of the 23 Country Offices.
3. Results
Some of the key results of this analysis were that 84% of projects have a log frame; 20% of
projects were based on a full Household Livelihood Security (HLS) assessment; 73% of projects
say their final goal is addressed at the household impact level; 45% of projects have some form
of evaluation plan; 43% of projects had a quantitative survey as a baseline; and in 65% of
projects, staff were measuring or processing outcome data. It is evident that the DMECA has led
to the systematization of reflection and varying levels of analysis of the state of DME in the
Country Offices.
Summary results were reported in the Executive Summary. Detailed results are divided into the
following sub-sections: Concept for Projects (3.1); Diagnosis, Design, Goals, and Indicators
(3.2); Monitoring and Evaluation (3.3); Information Processing and Use (3.4); Training and Use
of DME Skills (3.5); and DME CA Process (3.6).
DME CA Global Synthesis 3