A guide for practitioners

More documents

Recommendations

Info

Monitorable:Monitorability of indicators refers to whether an indicator can be independently validated or verified.Indicators should also be reliable and valid to ensure that what is being measured at one time is what itcan be measured at later time and that what is measured is actually what is intended to measure. If anyone of these five criteria is not met, formal performance indicators will suffer and will be less useful.In addition to the CREAM criteria, some authors suggested different sets of criteria. Roche (1999)claimed that when indicators are used more as specific examples of change, different characteristicsbecome important. In this context, he outlined another set of characteristics for indicators using theacronym SPICED (subjective, participatory, interpreted, cross-checked, empowering, and diverse).Below we give brief description of these criteria.1. SubjectiveInformants have a special position or experience that gives them unique insights which may yield avery high return on the investigators time. In this sense, what may be seen by others as ‘anecdotal’becomes critical data because of the source value.2. ParticipatoryIndicators should be developed together with those best placed to assess them. This involvesbeneficiaries, local staff and other relevant stakeholders.3. InterpretedLocally defined indicators may not mean much to other stakeholders, so they often need to beexplained.4. Cross-checkedThe validity of assessment needs to be cross-checked, by comparing different indicators and progress,and by using different informants, methods, and researchers. This characteristic is more critical whenthe bias of sources is polarized.5. EmpoweringThe process of setting and assessing indictors should be empowering in itself and allow groups andindividuals to reflect critically on their changing situation.6. DiverseThere should be a deliberate effort to seek out different indicators from a range of groups, especiallymen and women. This information needs to be recorded in such a way that these differences can beassessed over time. This characteristic is more important when the intervention has differential effecton gender, age, ethnicity etc.6.2 Types of indicatorsIndicators are classified into quantitative, qualitative, proxy indicators and pre-designed indicators.Below we give brief description of each of these types of indicators.34
Quantitative vs. qualitative indicatorsIn establishing M&E system, it is recommended to start with a simple and quantitatively measurablesystem rather than inserting qualitatively measured indicators upfront. Quantitative indicators arediscrete measures or indicators that are expressed numerically (number, mean, median, percentile, andratio). Examples of such indicators are number of farmers who adopt a technology, ratio of women-tomenin extension service, percent of farmers who adopted a given technology, percent of farmers abovea certain income level, percent of farmers who contribute to collective action etc. Outcome indicatorsare often expressed as the number or percent of something. However, it should be stressed that usingboth numbers and percentages for a given indicator provides more complete information than just usingnumber or percentage. The quantitative indicators directly measure the status or change of specificvariables. For example, crop yield in kg, kilometre of irrigation canal constructed are quantitativeindicators which provide direct numerical results.Qualitative indicators, on the other hand, are variables that measure an issue based on qualitativeassessment. Qualitative indicators are measures of an individual or group’s judgement and/or perceptionof congruence with established standards, the presence or absence of specific conditions, the qualityof something or the opinion about something (CIDA 2009). Many of the qualitative indicators useadjectives such as successful, adequate, equitable, good, effective, participatory, empowered and wellfunctioning. Qualitative indicators can also use scales or ranks, such as highly, moderately, poorly,adequately etc.Qualitative indicators are collected by asking people to express their opinion, judgement, perception orexplain what they have observed. They are reported using qualitative assessments such as congruencewith, quality of, extent of and level of. Qualitative indicators provide insights into changes in institutionalprocesses as well as attitudes, beliefs, motives and behaviours of individuals. A qualitative indicatormight measure perception, such as the level of farmers’ satisfaction with regard to a new extensionapproach. Qualitative indicators might also include a description of behaviour, such as the level ofmastery of a newly learned skill. The advantage of qualitative indicators is that they can capture thingssuch as perception and quality which are difficult to express in numbers (quantitatively). However, datacollection, measurement and analysis for qualitative indicators is more time consuming, and likely tobe less accurate and prone to subjectivity.For the sake of simplification, information from qualitative indicators can be quantified. For example,opinion of people can be categorized and counted to express it numerically. But making quantitativeinformation qualitative is not possible as it is difficult to extract opinion from a number. In general,there should be a balance about the number of indicators between qualitative and quantitative.Qualitative indicators provide more in-depth information. These two types of indicators can be used incomplementary way. For example, we can take opinion of target group on something by listing theiropinion, but also we can use a more quantitative approach and ask the target group to report on theissue quantitatively.Proxy indicatorsIt may not always be possible to develop precise and direct indicators. Some outcome indicatorsmay be difficult to measure directly. In such instances, one can strive to use approximate measures.However, use of indirect or proxy measures should be limited only to situations when data for directindicators are not available, when data collection may be too costly or beyond the available budget,when data cannot be collected at desired intervals, or when data collection is not feasible at all. In35
Page 1 and 2: Improving Productivity and Market S
Page 3 and 4: Authors’ affiliationsBerhanu Gebr
Page 5 and 6: 9 Reporting and using monitoring an
Page 7 and 8: AbbreviationsANOVABoARDCBOCIDADACHD
Page 9 and 10: 1 IntroductionAgricultural developm
Page 11 and 12: 2 Results-based monitoring and eval
Page 13 and 14: OutputsOutputs are direct products
Page 15 and 16: (outputs), and/or the change in acc
Page 17 and 18: to systematically collect relevant
Page 19 and 20: Ultimate outcome(long-termimproveme
Page 21 and 22: 3 Relationships between the concept
Page 23 and 24: outputs) and outcome targets is cru
Page 25 and 26: the quality of new or renewed proje
Page 27 and 28: intervention had not taken place?
Page 29 and 30: 3.3 Differences and complementariti
Page 31 and 32: 4 Participatory monitoring and eval
Page 33 and 34: 4.3 Fundamental principles of PM&EF
Page 35 and 36: how the results are to be used. End
Page 37 and 38: involved in selecting the results s
Page 39 and 40: 4.5.6.Note that implementation focu
Page 41: contributions to the achievement of
Page 45 and 46: learning, reporting and decision-ma
Page 47 and 48: Targets are established for each in
Page 49 and 50: possibility when primary data colle
Page 51 and 52: an initial review of the literature
Page 53 and 54: Field experimentsIn rare cases, con
Page 55 and 56: 8.5.1 Qualitative vs. quantitative
Page 57 and 58: 9 Reporting and using monitoring an
Page 59 and 60: management in M&E implies capturing
Page 61 and 62: Clear roles and responsibilitiesCle
Page 63 and 64: addressed. First, there are issues
Page 65 and 66: Annex 1 Glossary of common M&E term
Page 67 and 68: Note: An activity is economical whe
Page 69 and 70: LLessons learned: Generalizations b
Page 71 and 72: RReach: The beneficiaries and other
Page 74: ISBN 92-9146-245-4Federal Democrati

A guide for practitioners

Create successful ePaper yourself

Delete template?

Save as template?