BDS market development guide.pdf - PACA

63
• How well?—the quality dimension to performance: indicators should offer insight into
the quality of the change they have helped to bring about.
• Is it feasible to measure? there really is a difference between what we would like to learn
and what can be learned: indicators need to be set that can be measured given the
practical limitations of resources and expertise available.
• Are there “intermediate indicators”?- a plausible pathway to the bottom line: indicators
should reflect changes closer to the point of consumption of BDS and not rely only on
bottom-line changes in SME performance that may be influenced by many other factors.
• Precise?—”tightness” in definition: without real specificity in the definition of indicators,
interventions lack focus.
• Is it business-like?—the overarching need to “think business”: indicators need to reflect
SMEs and providers own priorities and business realities. Anything else undermines the
general business-like message of BDS.
Big Problems that Don’t Go Away
The world of small businesses that BDS interventions seek to improve in some way is
inherently complicated. SMEs do not exist by themselves; they compete in markets against
other businesses, have relationships with a range of suppliers, are influenced by government
actions, and are subject to the whims of market trends. Their world is complex. Indeed, often
in small informal businesses where the distinction between business and household is
blurred, this complexity is greater than in the formal sector.
This complexity poses great challenges for monitoring and evaluation. A question illustrates
this point: if an SME has changed positively apparently as a result of a project, how do we
know that?
• This change would not have happened without us; all the other myriad influences on a
business have not caused change, rather than our project alone; and
• This change is not at the expense of another SME nearby; it does not merely displace
economic activity from one place to another without any real net addition.
These issues of attribution are always relevant in business development situations, and
(regrettably) there is no magical fix to call on to address them 42 .
42 See Oldsman, E; "Evaluation as an effective management toool"; Paper presented to conference on "Business
services for small enterprises in Asia: developing markets and measuring performance"; Hanoi, 3 rd -6 th April,
2000 for a more detailed analysis of fundamental challenges inherent in rigorous approaches to evaluation.
Chapter Five—How do We Assess Our Performance?—
Monitoring and Evaluation BDS Interventions