planning a governance assessment - United Nations Development ...

Indicator checklist
How one chooses to measure or assess an aspect of governance will directly impact the type of results that will be reported.
Below is a checklist of questions to consider when selecting an existing or developing a new indicator.
• What sources of data are available for this indicator?
• What type of data will this indicator produce?
• Can the results be disaggregated?
• At what level does it assess governance: macro or micro?
• What form of governance is examined: de facto or de jure?
• Will it be an objective, perception-based or a proxy indicator?
• How should it be measured: by text (qualitatively), numerically (quantitatively) or by both?
• Will it capture the local context and/or vulnerable groups?
Conclusions, costs and benefits concerning indicators
In conclusion, it is probably best to try and strike a balance between “core” (existing) and “satellite” (new country- or
culturally-specific) indicators when developing a country-led assessment. In the recent Mongolian assessment only 11
percent of indicators used fell into the “satellite” category. It also is recommended that both de jure and de facto forms of
governance be assessed, and the macro and micro levels of governance be measured as directly as possible. Using the
checklist provided above is a good place to start, whether using an existing indicator or developing a new one.
Participation in indicator identification by an expanded group of stakeholders will likely increase the amount of ownership
people feel in the process and increase interest in the results. However, the larger the group, the longer and more costly the
process is likely to become. When it comes to the more technical side of developing the indicators, a smaller skilled group
is best.
Developing new indicators requires skill. Many previously used indicators have been tested for reliability and validity, which
can save time and money. New indicators should be developed only where gaps exist. Regardless of the approach, the cost
of new indicators and the time required to develop them must be seriously considered. Brainstorming in a small or large
meeting of stakeholders is the easy part; meeting high professional standards for this type of work (testing for reliability and
validity) is the harder. Like most factors concerning the assessment process, it may come down to how much time, money
and human resources are available.
Many existing indicators are based on widely accepted universal standards and treaties, such as the human rights
accords. These universal standards are often advocated by researchers, civil society and marginal groups within countries.
If ownership of the selection process is broad-based and national expertise is trusted, there may be greater acceptance for
adopting and adapting frameworks based on such standards. Strong ownership may well lead to less focus on indicators
being old or new, and a greater focus on what the assessment should aim to measure.
So, when should you develop new indicators and when should you use existing ones? This is a much-debated point. Where
there is a legitimate need for new indicators, that need should be met. However, when existing indicators are sufficient, then
new indicators should not be created. What is important here is that there is strong stakeholder ownership of the indicator
selection process. Nationally led assessments should capture the local context, vulnerable groups, and national priorities.
We do need to invest in better measurement to push our understanding of how governance matters, especially for groups
and issues that have not been included in external assessments. If financial and human resources are indeed limited,
it is better to measure fewer aspects of governance well, than many poorly. Simply stated, better measurement leads to
better results. The more reliable and valid the indicators, the more likely that the results will be viewed as professional
and legitimate.
Planning a Governance Assessment: A Guide to Approaches, Costs and Benefits
27