Guidelines for Impact Monitoring & Assessment in Microfinance ...

portals.wi.wur.nl

Guidelines for Impact Monitoring & Assessment in Microfinance ...

Birgit Schäfer

IMPACT MONITORING & ASSESSMENT Section IN MICROFINANCE 41

PROGRAMMES

Economic Development and Employment Promotion

Guidelines for Impact Monitoring &

Assessment in Microfinance

Programmes

1


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

2


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Birgit Schäfer

Section 41

Economic Development and Employment Promotion

Section Financial Systems Development

Guidelines for Impact Monitoring &

Assessment in Microfinance

Programmes

September 2001

3


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

4

Published by:

Deutsche Gesellschaft für

Technische Zusammenarbeit (GTZ) GmbH

Postfach 5180, 65726 Eschborn

Internet: http://www.gtz.de

Division 41 – Economic Development and Employment Promotion

Author:

Bärbel Schäfer

Responsible:

Roland Gross

Layout:

seifert media inform, 65929 Frankfurt


Table of Contents

IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Abbreviations ........................................................................................................................ 7

PART 1: Introduction ............................................................................................................ 9

1.1 SCOPE .................................................................................................. 9

1.2 UNDERSTANDING IMPACT AND IMPACT MEASUREMENT.......... 15

What Does Impact Mean? ................................................................................ 15

What Do Impact Monitoring and Impact Assessment Mean? .......................... 17

From Proving Impact to Improving Intervention................................................ 19

1.3 QUALITY ASSURANCE IN GERMAN TECHNICAL

COOPERATION .................................................................................. 22

A New Conceptual Approach to Quality Monitoring ......................................... 22

Implementation of the Quality Management System........................................ 24

Quality Management Tool: The Impact Model.................................................. 25

1.4 A MULTIDISCIPLINARY APPROACH ............................................... 28

Bridging the Attribution Gap.............................................................................. 28

The Analytical Framework: A Systems Perspective......................................... 30

The Methodological Framework: Knowledge Creation..................................... 31

The Operational Level: Quality Assessment and Process Orientation............. 38

Composition of the Team.................................................................................. 40

A Step by Step Approach ................................................................................. 42

PART 2: Implementation Steps ......................................................................................... 47

2.1 Introduction........................................................................................ 47

2.2 STEP 1: IDENTIFICATION OF CORE ISSUES.................................. 51

Analysis of Programme Effectiveness................................................... 52

Financial Market Conditions ............................................................................. 52

Programme History: Financial Institution and Instruments............................... 52

Programme Outreach ....................................................................................... 53

Performance Indicators..................................................................................... 56

Analysis of Environmental Condititions................................................ 57

Government Policies......................................................................................... 58

The Natural Environment.................................................................................. 59

Physical, Social and Institutional Infrastructure ................................................ 59

Economic Infrastructure and Socio-economic Context .................................... 60

Analysis of the Household Level ........................................................... 61

The Composition of a Household ..................................................................... 61

The Concept of Household Socio-economic Portfolios .................................... 62

The Concept of Vulnerability of Livelihood ....................................................... 63

2.3 STEP 2: FORMULATION OF CORE HYPOTHESES......................... 67

Identification of Impact Domains ........................................................... 68

Levels of Analysis ............................................................................................. 68

5


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

6

Core Impact Hypotheses...................................................................................69

Core Impact Hypotheses at the Family/Household Level .................................70

Core Impact Hypotheses at the Individual Level (women/men)........................71

Core Impact Hypotheses at the Enterprise/Farm Level ....................................73

Core Impact Hypotheses at the Community Level............................................74

2.4 STEP3: FORMULATION OF INDICATOR SETS................................ 76

Identification of Impact Indicators ......................................................... 77

Quality Criteria (I) ..............................................................................................77

Types of Indicators ............................................................................................78

Mediating Variables ...........................................................................................79

Quality Criteria (II) .............................................................................................81

Selection of Indicators ............................................................................ 82

Levels of Measurement .....................................................................................82

Core Indicators at the Family/Household Level ................................... 85

Core Indicators at Individual Level..................................................... 856

Core Indicators at the Enterprise Level (off-farm activities) ................. 87

Core Indicators at the Farm Level........................................................ 88

Core Indicators at Community Level.................................................... 89

2.5 STEP 4: ADAPTION OF SURVEY METHODS................................... 90

Application of Survey Tools ................................................................... 91

TOOL 1: Impact Survey ....................................................................... 92

TOOL 2: Client Monitoring System .................................................... 109

TOOL 3: Client Exit Interview............................................................. 117

TOOL 4: Case Study 1: Client History/Loan Use &

Saving Strategies................................................................ 122

TOOL 5: Case Study 2: Client Empowerment ................................... 126

TOOL 6: Roundtable: Client Satisfaction ........................................... 130

2.6 STEP 5: DATA ANALYSIS .............................................................. 137

Aggregation of Data Sets ...................................................................... 138

Data Logging ...................................................................................................138

Data Analysis and Presentation of the Results: Quantitative Data .................142

Data Analysis and Presentation of the Results: Qualitative Data ...................154

Documentation Sheet......................................................................................158

Summary Matrix: .............................................................................................162

ANNEX 1: Sources and Further Reading........................................................... 163

ANNEX 2: Rapid Appraisal Methods: Exampel ................................................. 171

ANNEX 3: Interiew Guide .................................................................................... 173

ANNEX 4: Sampling Methods ............................................................................. 175


Abbreviations

IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

AFCE Associations Feminines de Credit et d’Epargne

AIMS Assessing the Impact of Microenterprise Services

ANOVA Analysis of Variance

BMZ Bundesministerium für Wirtschaftliche Zusammenarbeit und

Entwicklung

CDE Centre for Development and Environment

DCE Department Credit Epargne

DCEG Dakar Consulting & Engineering Group

DIE Deutsches Institut für Entwicklungspolitik

FENU Le Fond d’Equipements des Nations Unies

FI Financial Institution

GATE German Appropriate Technology Exchange

GTZ Deutsche Gesellschaft für Technische Zusammenarbeit

(German Technical Cooperation)

IA Impact Assessment

IADB Inter-American Development Bank

IDRC International Development Research Centre

IFPRI International Food Policy and Research Institute

IIEP International Institute for Environment and Development

IM Impact Monitoring

IRAM Institut de Recherches et d’Applications des Méthodes de

Développement

MARP Méthode Accélérée de Recherche Participative

M&E Monitoring and Evaluation

MFI Microfinance Institution

MIS Management Information System

MUTEC Mutuelles d’Epargne et de Crédit

NEGD Non-equivalent Group Design

NGO Non-governmental Organization

OESP Office of Evaluation and Strategic Planning

PCM Project Cycle Management

P&D Planning and Development Department

PIM Participatory Impact Monitoring

PIMU Policy Implementation Monitoring Unit

7


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

PLA Participatory Learning and Action

PMR Promotion des Mutuelles Rurales

PNGT Plan National de Gestion Territoire

PRA Participatory Rural Appraisal

ROSCA Rotating Savings and Credit Association

RRA Rapid Rural Appraisal

SEEP The Small Enterprise Education and Promotion Network

SHG Self-help Group

SPSS Statistical Package for the Social Sciences

SS Sum of Squares

UNRISD United Nations Research Institute for Social Development

UNSO United Nations Office to Combat Desertification and Drought

ZOPP Zielorientierte Planung (Objective-oriented Project Planning)

8


Part 1: Introduction

SCOPE

by Roland Gross 1

and Birgit Schäfer 2

INTRODUCTION

“Only partially obscured by the mountain of paper

the development business generates, a

striking tension has appeared. On the one hand,

most observers feel that the world seems more

uncertain. On the other hand, development decision

makers insist on higher levels of certainty

before they release funds and undertake programmes.”

3

“‘Microfinance’ became a buzzword in the 1990. No other development topic attracted

comprarable attention from donpors, and no other tool was regarded more

effectice in fighting poverty”. 4 Having access to microfinance programme services

means having access to productive resources through loan and savings products.

From a socio-political point of view, being a member of a self-help group or cooperative

movement, or being a client of a village bank or financial NGO, means accepting

the institutional social structure in place to undertake the given activities.

This can lead to attitude changes in daily life. Poverty has been viewed primarily as

a lack of income, which has reduced poverty to economic deficiency, without an

understanding of how poor people live.

Despite the many success stories of microfinance institutions in numerous countries,

donors are still keen on knowing how and to what extent microfinance programmes

have contributed to the reduction of poverty. Since donor money is mostly tax payers

money there is a strong interest of donors to clearly see that the flow of funds to

microfinance programmes have served their purpose. Many case studies have

proved that microfinance programmes or microfinance institutions have successfully

managed to offer financial products on a sustainable basis. But is it enough to say a

1

Roland Gross is a Senior Economist in GTZ’s Planning and Development Department (P&E),

Section Financial Systems Development

2

Birgit Schäfer is a Research Fellow at University of Hohenheim, Department of Development Policy

3

Samoff, J., “Chaos and Uncertainty in Development, In: WORLD DEVELOPMENT 24(4), 1996, pp

611-633, Cited in: Preuss/Steigerwald 1966:1.

4

Steinwand, D., “The Alchemy of Microfinance”, Berlin 2001

9


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

sustainable microfinance institution alone will reduce poverty? Indeed, impact

measurement in microfinance should not stop at the institutional level. Programme

interventions serve multiple ends. Impact measurement seeks to measure and explain

induced changes that occur at the client level in terms of their quantity, quality,

and direction and addresses how to achieve meaningful programme results. The

latter requires analysing programme results on the basis of understanding the complexity,

diversity, and contingency of the livelihood of the rural and urban poor.

The purpose of these guidelines is to overcome the practical restrictions of impact

measurement mentioned above by focusing on management-oriented measurement

systems that emphasize comprehensive evaluation and place impact monitoring

(IM) and assessment within the larger framework of on-going organisational activities.

Therefore, this analysis will seek to provide responses to the question “How

can we improve positive impact and promote transformation?” rather than the

question ”Was there any impact?” In this context, we likewise ask:

10

• Who are the programme clients?

• Are clients benefiting from participation in the programme?

• When does impact occur?

• What happened?

The reinforcement and consolidation of internal monitoring and evaluation (M&E)

capacities goes hand in hand with our approach. On the basis of a flexible, systems

analytical framework, it is possible to monitor the complexity and dynamics of the life

of the target population. We use a multidisciplinary approach that includes aspects

of scientific experimental models, qualitative models, and participant-oriented models

based on the underlying operational principles of process evaluation in order to

plausibly explain and attribute changes in the life of the target population. We concentrate

on a practitioner-oriented approach, considering options as well as limitations.

We are thus able to look not only at the impact of programme interventions at

the client level, but also at the quality of programme implementation and the services

provided. Based on market research, it is also possible to make recommendations

on adapting financial products and institutional design, complementary actions,

and cooperating with other programmes.


These Guidelines 5 :

INTRODUCTION

• Offer relevant information that will help users make informed and effective

decisions in order to design IM and IA suited to prevailing contextual factors

and objectives;

• Build on and improve existing M&E procedures, and help users understand

and appraise the impact of projects on human well-being (current M&E

mainly focuses on performance indicators such as financial and institutional

sustainability criteria);

• Measure the effectiveness of microfinance programmes according to key

concepts in poverty reduction: strengthening physical, human and social

capital; increasing the standard of living; improving access to and control

over productive resources; and enhancing knowledge about and participation

in individual rights and power;

• Help to design a less-costly, application-orientated M&E process that is context

specific and with which it is possible to reach a high level of data reliability;

• Provide information for decision making, project design and mid-term corrections

by proving impact (accountability) and improving intervention (project

management);

• Help users avoid undesirable or negative programme impact;

• Identify various MFI stakeholders and make them more aware of their ownership;

• Help to indicate necessary changes in microfinance policies to ensure that

efficient dissemination and transparency exist between the project, the MFI

and the donor; and,

• Show donors the effectiveness of their input in response to their goal to

alleviate poverty.

5 A view on impact monitoring in projects focussing on economic development and employment

promotion in a broader sense than here is provided in Vahlhaus, M., Guidelines for Impact

Monitoring in Economic and Employment Promotion Projects with Special Reference to Poverty

Reduction Impacts (Parts I and II)

11


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Principal Users of the Guidelines:

Our intention is to demonstrate operational methods to prove impact and improve

intervention. It is challenging to find an acceptable balance between the quality

(credibility, objectivity and validity) of impact measurement and its costs (financial

resources, time and expertise). We have taken the normally small budgets of projects

and institutions to conduct IA and IM into account. We have therefore devised

practical tools and methods that will be easily accepted by project staff. IM and IA

must be understood as important project management tools in technical cooperation

programmes based on an interactive learning process. These guidelines primarily

focus on aspects of microfinance programmes. Principal users of the guidelines are:

12

• Programme coordinators and project managers responsible for initiating the

monitoring process, assessing results, and organizing the presentation, dissemination

and storage of information.

• Practitioners, experts, consultants and researchers conducting IA and IM.

Advice about which methods to select in specific situations is included.

How to Use the Guidelines:

Part 1 is designed to help users organise and understand IM and IA procedures

with respect to the objectives of microfinance programmes. It provides basic

information on the importance of IM and IA for programme management and

takes into account the methods and implementing procedures developed and

applied by German Technical Cooperation (GTZ). Special attention is given

to explaining the main concepts and analytical requirements underlying the

multidisciplinary approach elaborated in Part 2.

Part 2 contains step-by-step guidelines on how to develop and implement comprehensive

and integrated IM and IA. It provides users with basic analytical tools

and methodologies and encourages them to adapt these to fit their needs. It

can be selectively used, supported by the user‘s own methods and

instruments, and adapted to meet specific needs and requirements. Part 2

consists of the following steps:

STEP 1: Identification of core issues

STEP 2: Formulation of core hypotheses

STEP 3: Formulation of indicator sets


STEP 4: Selection and adaptation of survey methods

STEP 5: Data analysis

INTRODUCTION

Knowledge Base of the Guidelines:

These guidelines are based on the experience of diverse microfinance programmes

in IM and IA. They were developed through the inter-institutional collaboration of the

University of Hohenheim-Stuttgart, Department of Agricultural Economics and Social

Sciences in the Tropics and Subtropics and GTZ, Division of Financial Systems Development

and Banking Services. They are based on the applied research of two

GTZ microfinance programmes:

→ PADER-NORD – COTE D’IVOIRE, Korhogo

“Programme d’Appui au Développement Rural de la Région Norde, Département

Crédit-Epargne (DCE)”, (AFCE: Associations Féminines de Crédit et

d’Epargne) ; GTZ / ANADER 6

→ PMR - RFA/ NIGER, Niamey

“Promotion des Mutuelles Rurales”, (MUTEC: Mutuelles d’Epargne et de

Crédit); GTZ / Ministère de Finances, des Reformes Economiques et de la

Privatisation

Results are taken from impact studies of these two programmes and from impact

studies and project documents of other microfinance programmes. Study of relevant

literature and project documents has provided valuable insight to enrich the guidelines.

Our intention was not to reinvent the wheel, but to present a comprehensive

methodology and applicable instruments for IM and IA in microfinance. Hence, several

of the basic methodologies and tools presented in Part 2 of the guidelines are

currently being used in microfinance programmes and have proven effective. The

guidelines exemplify basic methodologies so that users can adapt them to local

conditions.

Acknowledgements:

GTZ wants to thank all those, who have contributed to this book. First and foremost

Mrs. Birgit Schäfer who has not only collected and evaluated the data over almost

6 German support to this project has ended in July 2001, before full sustainability of the microfinance

institution could be achieved. It is hoped that other donors can continue the very successful

programme in Northern Cote d’Ivoire.

13


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

two years, but has also drawn up the main parts of this study; Dr. Michael Hamp

who has given a lot of conceptual input during the early stage of the survey, all other

colleagues (special thanks to Dr. Brigitte Klein and Martina Wiedmaier), who have

given their valuable comments throughout the study. Mrs. Laura Elser for patiently

editing text and graphics, our project staff in Niger and Cote d’Ivoire as well as the

numerous farmer women and farmers in both countries for making their precious

time available. We hope that these guidelines contribute to improve on ongoing and

future microfinance programmes which in turn should make the lives of their clients

easier.

14


UNDERSTANDING IMPACT AND IMPACT MEASUREMENT

What Does Impact Mean?

INTRODUCTION

Microfinance is complex; not only in terms of how to build viable and demand-oriented

institutions, but also in terms of how to evaluate the implications of institutional

structure in impact measurement. Let us now address the question of what impact

means and why organizations should perform impact monitoring (IM) and impact

assessment (IA). 7

Implementing Impact Monitoring and Assessment

Means Dealing With Multiple Realities

First, let us look at responses to the question ”What changes have you encountered

in your daily life since becoming a member of your village bank?” which are drawn

from impact studies in two microfinance programmes in rural Western Africa. 8 The

responses vividly illustrate the wide range of direct programme outcomes that individuals

perceive due to their association with an MFI. Most go beyond typical development

goals.

“What changes have you encountered in your daily life since becoming a

member of the village bank/credit group?”

→ ”I can prepare better meals for my family.”

→ ”I went from being a day labourer to being self-employed.”

→ ”I don’t have to borrow from family members and friends to buy raw materi-

als for my small business.”

→ ”I had to pay major medical expenses for my wife, but the loan allowed me

to keep my business operating.”

→ ”Since I can support my family, my husband has withdrawn his financial

support more and more.”

→ ”I take part in community life and have made new friends.”

7

The definitions given are related to practitioner-oriented IM and IA. Compare the discussion given in

Sebstad (1998), a discussion paper prepared for the second virtual meeting of the CGAP Working

Group on impact assessment methodologies.

8

AFEC: Associations Féminines d’Epargne et de Crédit: GTZ - PADER-NORD/Ivory Coast; and

MUTEC: Mutuelles d’Epargne et de Crédit: GTZ - PMR-R.F.A./ Niger.

15


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

16

→ ”Nothing has changed, because my husband used the loan.”

→ ”I increased the size of my livestock.”

→ ”I have my own savings now.”

If we only use the development objectives that were defined during project planning

as impact indicators, we might not capture the actual state of well-being of the target

population. We must consider that technical cooperation projects operate in dynamic

settings, and that the target population is a heterogeneous group of individuals. It is

therefore not surprising that different clients of MFIs anticipate different changes and

have different perceptions of how project interventions have influenced their lives.

The lives of the target population in microfinance programmes reflect complex and

dynamic realities that are influenced by individual perceptions, personal interactions

and often-restrictive environmental conditions.

Changes and Innovations are Results of Complex

Social Interaction Processes

A basic problem in microfinance is that poverty has been considered primarily as a

lack of income. This has reduced poverty to mere economic deficiency and neglects

understanding how poor people survive. If we understand the development process

as an interactive and complex social transformation process with multiple players

involved, impact is linked to individual, social and organizational relationships and

learning processes. Causes and rationales of changes in a person’s and family’s

life, or even in a community, do not necessarily include tangible and determinable

patterns. Taking into account the complex character of human, social, economic,

and political transformation processes, changes can often not be attributed to a single

factor. As life itself is a dynamic process, impact also changes its character over

time – in terms of its values, degrees, patterns, and directions.

It is not feasible to provide a universal definition of what impact really means. Instead,

we will review the discussion in a general manner and determine analytical

categories.


INTRODUCTION

Impact involves economic, political, human and social transformation proc-

esses (changes):

• that are project outcomes at the client/beneficiary level;

• that can be plausibly related to project interventions;

• that can be direct or indirect, short-, medium-, or long-term, predictable

or unpredictable;

• that are linked to dynamic and often interrelated individual, social and or-

ganisational relationships and learning processes; and,

• that can be material changes, changes of behavioural patterns, changes

in individual perception, or changes in relation to the social environ-

ment.

Domains of impact refer to

• scope (physical, financial, human and social capital), whereas,

Dimensions of impact refer to

• analytical units (individual, household, enterprise, farm, and community

levels).

What Do Impact Monitoring and Impact Assessment Mean?

Impact Monitoring Is Not Impact Assessment

Impact Assessment Complements Impact Monitoring

Impact Monitoring (IM) tracks client progress and, when considered with other

client outreach data (gender, sector, etc.) and financial data, can yield good portraits

of clients who enter and stay in a programme and those who leave. It also

assists programme managers to know more about the key characteristics of the

clients that the programme reaches. It is important to note that we understand the

limited capabilities of a client monitoring system relative to those of impact evaluation.

Tracking change in key indicators is a method to obtain information that suggests

programme impact, but does not explore impact.

17


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Impact Monitoring generally refers to a regular, systematic observation process

and the on-going collection of information over time on the basis of pre-defined im-

pact indicators to ensure that a project is implemented in a timely and efficient man-

ner.

Impact Assessment (IA) complements IM. IA results can be used to redefine IM

indicators. However, IA serves a different purpose than IM. IA usually compares

clients with non-clients (often using new clients as a reference group) and provides

information to plausibly associate changes in selected impact variables to a

programme. It has implications for the use of the information collected through IM.

Impact Assessment estimates the value, degree and/or pattern of change that can

be plausibly associated with an intervention based on the information obtained

through monitoring. IA allows for a better understanding of interventions and results

in recommendations for improvement.

Thus, IA should be able to:

18

• Identify which clients are receiving more benefits and which are receiving

less, and why;

• Provide information on the different sectors in which clients are working;

• Improve institutional understanding of client preferences in terms of products

and services; and,

• Assess the key socio-economic characteristics of programme clientele to de-

termine whether the programme is reaching its intended target populations.

Clients Set Their Own Indicators

We can only incorporate the spectrum of impact perceptions in the monitoring and

evaluation process if the target population is actively involved in all aspects of IM

and IA. Because impact changes over time, we must repeat client interviews. Clients

are the subjects rather than the objects of impact evaluation. At the same time, the

problem of subjectivity in client perception must be treated carefully. Each client not

only has their own life history, but also their own programme participation history.

Individual clients might have special reasons for program outcomes that differ from

those of their neighbour, who, from our point of view, is living in the same, or at least


INTRODUCTION

similar, surrounding. The type of MFI also determines the nature of impact at the

client level. Thus, each MFI should identify its own goals and indicators, based on its

own experience and mission.

From Proving Impact to Improving Intervention

A widespread opinion among practitioners is that separate efforts to measure impact

are not necessary because microfinance programmes are self-evaluating: the success

of a programme is best measured by its continued growth and its ability to

keep clients. In other words, if the programme serves clients efficiently and profitably,

it is obviously doing a good job, and the clients demonstrate that they value the

programme by their on-going participation. This line of reasoning, which uses institutional

and financial viability indicators as proxies for quality measurement, tells us

more about the financial institution than about the clients. Hence, in the past, impact

evaluations in microfinance have tended to be contrary to reality, strictly separating

institutional and financial performance evaluation and impact assessment. We argue

that sustainable institutions rely on sustainable clients - and vice versa. Hence, we

focus on the interrelated and cumulative process. The purpose of practitioner-led IM

and IA is to combine the twin goals of proving impact and improving intervention.

Strengthening Internal Management Capacities

The better the needs of the programme users are met, the better the desired results

or objectives will be achieved. Thus, we can expect that programme interventions

generate welfare effects in the sense of lasting improvements in living conditions at

the client level. It is indispensable for comprehensive IM to know the ”whys” of both

clients and institutions in order to assess whether objectives are met, on which level

and in which direction. Client feedback helps explain the level of observed impact,

and also serves as a useful market research tool for programme services. The following

questions concern the relevance, effectiveness, and impact of programme

interventions and are related to issues at the client level.

Impact? Have project interventions resulted in lasting improvements in living

conditions at the client level?

Effectiveness? Are the desired results or objectives being achieved?

Relevance? Does the programme meet client needs?

19


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

In this context, we need to consider institutional performance in terms of its pertinence

to meeting client needs and combating poverty. Do the services provided by

the programme meet client needs? Have the objectives been satisfactorily met? If

not, what corrections are necessary to ensure that interventions have positive impact

on clients?

20

• In this context, we must check the following quality criteria at the client level:

From the point of view of the programme, what impact is anticipated on the

target group?

• To what extent do activities and results contribute to intended impact and

goals?

• What impact do programme clients perceive, and how do perceptions differ

according to the work sector and gender of the client?

• To what extent do results correspond with the target population’s needs?

• What do clients value about the programme (products, services)?

• Which clients receive more benefit, which less, and why?

On the basis of this operational model, we obtain findings at each level that you can

use to adapt and improve interventions and become more effective. The findings

should be integrated into existing project management activities. IM and IA incorporate

frequent feedback loops in which analysis, planning and decision-making are

repeatedly reassessed in light of experience gained. Our main goal is to institute IM

as part of the on-going M&E framework in order to strengthen the internal management

capacity of the implementing organisation. IM should be based on the principles

of process monitoring. 9

9 GTZ/ NARMS (1996)


Table 1: The Programme Outcome Model 10 (Example: Village Bank)

INPUTS

Resources dedicated

to or consumed by

the programme

Money

Staff and their time

Volunteers and

their time

Facilities

Equipment and

supplies

ACTIVITIES

What the programme

does with the inputs

to fulfil its mission(s)

Supports the creation

of a sustainable

FI with credit

delivery and savings

facilities on

the basis of the

linkage banking

approach

Holds regular

meetings

Offers functional

literacy courses

Creates a network

structure

10 The illustration is adapted from United Way (1999).

OUTPUTS

The direct products

of programme

activities

RESULTS

Number of clients/

members by gender

Ratio of rural to

urban clients

% of total targeted

clientele served to

total population in

the intervention

area

Number of active

borrowers by gender

Average outstanding

loan

portfolio (in %

change from last

year)

Value of average

savings account

by gender

INTRODUCTION

OUTCOMES

Net effects for

participants during

and after the

programme

IMPACT

Short-term:

Increased income

Increased business

skills

Changed attitudes,

values, know-how

Medium-term:

Modified behaviour

Increased resource

bases

Long-term:

Improved living

conditions

Altered status

21


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

QUALITY ASSURANCE IN GERMAN TECHNICAL COOPERATION

A New Conceptual Approach to Quality Monitoring 11

What principles underlie IM and IA in German technical cooperation? GTZ has undergone

numerous changes in organizational structure, instruments and procedures

in order to ensure the quality of its work. Faced with changes in quality criteria for

the implementation of technical projects and the provision of advisory services and

given the dynamic setting in which development cooperation currently operates,

evaluation methods and instruments must be accordingly adapted.

Impact monitoring is an integral part of GTZ’s quality management system. Conventional

monitoring procedures in quality assurance were based on quality at entry.

Efforts were geared toward a comparison of the preliminary situation with targets

fixed to achieve specific development objectives. Project implementation proceeded

according to a logical framework. Experience has shown, however, that detailed

planning and implementation of a project does not automatically result in greater

success, and is not necessarily correlated with the overall social and economic development

process of partner countries. Detailed logical plans have often failed to

capture unforeseen obstacles and opportunities due to their lack of flexibility in implementation

and adaptation. Not enough attention was given to the relevancy of

project results for the clients. M&E was integrated into a central control culture that

weakened local staff activities and responsibility for quality work and reduced monitoring

to mechanical accountability measurements.

Taking these operative problems into account, GTZ now regards quality as a relative

concept in the process of quality assessment, which includes a multitude of instruments

and procedures. A critical component of the conceptual changes in quality

management implied that within each target population, client groups are normally

heterogeneous.

11 GTZ (1999)

22


INTRODUCTION

Graphic 2: Benchmarks in the BMZ/GTZ Evaluation System 1980 - 2000 12

It is against this background that GTZ reoriented its procedures in quality management

in 1997, using client satisfaction, which depends on the usefulness of project

results, as a benchmark. Now, the decisive indicator of success is not whether

planned results have been achieved, but what results have been achieved – and

whether these results meet client needs. 13 Additionally, the broad spectrum of stakeholders

implies a wide range of interests, and hence, attention must be carefully

directed to the systematic evaluation of results and impact. In 1998, GTZ redefined

its corporate identity and determined five guiding principles for its staff members:

• Client orientation;

• Results orientation;

• Efficiency, flexibility;

• Responsibility; and,

• Accountability.

12 Source: V. Steigerwald (1999).

13 T. Kuby (2000), Internal Evaluation Team, GTZ:

2000

client feedback

self-evaluation/learning

monitoring as negotiation what aid works

quantitative/qualitative

objective/subjective budget constraints

orientation towards impact

process monitoring quality management

participatory monitoring decentralization

1995 Quality at entry

PCM, ZOPP ownership

accountability does aid work?

1990

reporting, not steering participatory (rural) appraisal

piles of data

monitoring units 4 th generation evaluation

indicator banks

poverty alleviation participation

1985

monitoring function

project management

system target group orientation

ZOPP clearer project design

1980

23


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Implementation of the Quality Management System

Due to the aforementioned paradigm shift in quality assessment, GTZ now translates

theoretical monitoring into concrete institutional restructuring by selecting

measures that achieve the optimal mix of self-evaluation, internal evaluation and

external evaluation. Evaluation is now decentralized and relies on the competence

and willingness of the staff to take responsibility for the quality of their work. This

has resulted in a shift from operational rules to guidelines, thus leaving room for selfinitiative

on the part of project staff.

Graphic 3: GTZ Evaluation Levels

GTZ promotes effective cooperation among staff and organizational units through

flexible organizational structure and flexible procedures. Recently, an Internal

Evaluation Office replaced internal team evaluation to perform impact monitoring. 14

14 Since April 2000, the newly created Internal Evaluation Office performs this task. This institutional

restructuring ensures the main evaluation principles – independence, credibility and usefulness –

upon which BMZ considerations are based.

24

FOCUS TASK

GTZ Performance and Evaluation System

GTZ Portfolio Business

Strategies

Countries Portfolios

and Sectors

Performance/

Usefulness

of Projects

and Programmes

BMZ

Federal Audit

Office, Public

Accountant

Internal Evaluation

Office

Operative Units

(P&D, Regional Departments,

Country Offices)

Projects & Programmes

Operational

Self-Evaluation

External

Evaluation

Internal

Evaluation


INTRODUCTION

The new office is schematically between external and internal evaluation, though

directly linked to the GTZ managing directorate, and thus independent of operative

management. The internal evaluation office looks at GTZ’s overall portfolio and the

effectiveness of business strategies; it assists with the self-evaluation of operational

units and cooperates closely with external evaluation. The office acknowledges that

a central control unit cannot work independently when dealing with thousands of

projects in more than 100 countries. Therefore, quality assurance must be linked to

on-site responsibility in order to ensure an interactive learning process. Expected

results should create more synergy between the different elements of the quality

evaluation system and develop an evaluation culture.

Quality Management Tool: The Impact Model

The new impact model used by GTZ for internal evaluation takes the aforementioned

challenges into account and acknowledges the fundamental methodological

problem of any IA: the existence of an attribution gap. This refers to the difficulty

of attributing specific effects (project outcomes at the client level – impact) to

specific causes (project interventions).

25


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Graphic 4: GTZ Impact Model 15

What does this mean?

As already mentioned, we consider programme impact to be all net effects of a project

that are not directly under the control of project management. The attribution

gap cuts the impact chain into two levels: one depicting direct effects of project intervention

and the other depicting indirect effects that fall under aggregate development

goals. Impact cannot be accurately depicted as a linear function whereby effects

are clearly traced, or attributed to a specific cause. Factors that might additionally

influence project outcomes can be grouped into two main categories, as outlined

below.

15 Compare T.Kuby (2000).

26

Monitoring and self

Evaluation of projects

inputs

highly aggregated

development progress

possible indirect benefits

of the project Project-independent

evaluation

Attribution Gap

use of outputs

and services

outcome and services

activities

direct benefits of project

output and services

Facts about

the development

Facts about

the project


Mediating Variables

INTRODUCTION

Factors that enhance or constrain opportunities for change but are not directly linked

to programme intervention, such as client gender, number of household members,

and price of enterprise inputs.

External Factors

Phenomena that cause or lead to changes, irrespective of the programme, such as

an increased level of household income due to macroeconomic conditions and other

increases not associated with client activities.

Consequently, we need to conduct an intervention assessment to accurately understand

the processes of change and their causes. Impact measurement can plausibly

associate changes in selected impact variables with programme interventions. In the

international debate about practitioner-oriented impact measurement, plausibility is

considered as an acceptable standard for attribution and is able to bridge the attribution

gap. We use a combination of scientific and humanistic approaches to check

the validity of information and provide added confidence in the findings and client

responses.

GTZ management has clearly stated that the self-evaluation process of a project

must include the monitoring of direct benefits. Nevertheless, GTZ’s own portfolio

management and public accountability rely upon information about the general development

effectiveness of individual projects, which goes beyond direct benefits.

Project results should be analysed with a development perspective on the basis of

project-independent evaluation methods. There is no contradiction if data collection

and analysis are conducted by qualified staff and, as much as possible, in collaboration

with external experts – ideally local ones – who should be brought in to verify

the accuracy of the data obtained and analysed. Local experts external to interventions

possess in-depth knowledge about local socio-economic and cultural conditions

and are able to determine if the interview partner digresses from the subject.

27


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

A MULTIDISCIPLINARY APPROACH

Bridging the Attribution Gap

Microfinance is not only faced with the inherent difficulty of plausibly ascribing individual

changes at the client level to specific project interventions, it is also faced with

the consequences of the fungibility of money. What does this mean? Foremost,

money – such as credit, savings, or surplus income – can be used for multiple purposes

such as consumption, production, and investment. Determining the use of

money within a household socio-economic portfolio is an elusive undertaking because

MFIs often operate without bookkeeping and household and business budgets

are not strictly separated. At the same time, interventions result in empowerment

due to financial self-sufficiency and social intermediation services, such as group

formation, leadership training, cooperative learning, and self-governance. These

types of impact are bound to individual characteristics, subjective perceptions, and

the cognitive faculty of the clients and can be grasped most readily by proxy indicators.

The result is that a number of intangible attributes are difficult to measure with

conventional impact measurement tools. From the outset, we must understand the

strategies that clients follow to earn their living and assure their well-being. We need

to find the right balance between a flexible analytical framework and strict operational

models in order to attain both dynamic and heterogeneous development and

systematic IM and IA.

Therefore, the construction of the bridge over the attribution gap should follow the

same criteria as the work of an architect who closely collaborates with his clients.

We must design a solid base by establishing a sound analytical framework in order

to implement operations (monitoring and evaluation) through appropriate tools

(methodologies). The quality of our work is determined by the credibility criteria of

data collection and processing. The better the validity of our information, the more

confidence we and other stakeholders will have in our bridge. In practitioner-oriented

impact measurement, we need to make choices within the methodological hierarchy

of purpose and rigor and thereby ensure that the trade-offs between data accuracy

and cost-effectiveness are kept to a minimum. This process requires all

stakeholders to have a good understanding of and agreement about the primary

assessment objectives, the methods to be used, and the resources available to

28


INTRODUCTION

conduct IM and IA. The reliability and objectivity of any evaluation depends on the

following five issues:

• Hypotheses, indicators, design and findings based on an in-depth understanding

of the clients, the intervention, the programme mission, the impact

processes, and the possible effects of external factors;

• The sampling methodology, which must be random in order to provide representative

results;

• The quality of the data collection tools;

• The quality of the data collection process, including interview techniques,

staff skills and supervision; and,

• The quality of the analysis.

When a concept is translated into a functioning and operating reality (operationalisation)

one must be concerned about the quality of the translation. Various levels of

validity, which are essential for obtaining a high level of confidence in data accuracy

and reliability, are described below. The left side of the table describes types of validity.

These provide a unifying theoretical base of the criteria necessary for effective

application of impact monitoring and assessment. The right side presents the respective

operations. 16

Table 2: Levels of Validity in Impact Measurement

THEORETICAL BASE PRACTICAL OPERATIONS

INTERNAL VALIDITY

the approximate truth

about inferences regarding

cause and effect or causal

relationships

EXTERNAL VALIDITY

the degree to which the

conclusions in your study

would hold for other organisations/persons

in

other places and at other

times

16 compare Trochim (2000)

DESIGN

the structure of the study, which explains how all of the

major variables (samples or groups, measurements, programmes,

and assignment methods) work together to address

the central hypotheses

SAMPLING

the process of drawing representative samples, i.e.,

selecting units (clients) from a population of interest (MFI

clients) so that sample results may be generalized for the

population from which the units were selected

29


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

30

CONSTRUCT VALIDITY

the degree to which inferences

can legitimately

be made from the operationalisations

in the assessment

to the theoretical

constructs on which

they were based

CONCLUSION VALIDITY

the degree to which conclusions

made about the

relationships in the data

are reasonable

MEASUREMENT

the process of observing and recording the observations

collected; fundamental types of measurement and their reliability

fall into four broad categories in terms of quality,

consistency and repeatability

Categories of measurement include:

Survey research: the design and implementation of interviews

and questionnaires;

Scaling: methods of developing and implementing scales

that associate qualitative constructs with quantitative metric

units (nominal, ordinal, interval and ratio);

Qualitative research: the range of non-numerical measurement

approaches;

Unobtrusive measurement: a variety of measurements that

do not intrude on or interfere with the context of the research

(indirect measurements; content/text document analysis and

secondary analysis of data).

ANALYSIS

Data analysis involves three major steps: cleaning and organizing

the data (data preparation); describing the data

(descriptive statistics); and, testing hypotheses and models

(inferential statistics).

The Analytical Framework: A Systems Perspective 17

Due to the difficulties specified above in implementing the dynamics and processes

of household socio-economic portfolios (production, investment, and consumption

activities) in the context of programme objectives, we will try to capture these variables

in a process-oriented analytical framework in order to understand causality

rather than control for it. It is practically impossible to determine exactly where

money goes, i.e., to separate the systems within a household into which a programme

client channels their money. Hence, we see fungibility of money as a

vital client strategy to manage their affairs, thereby acknowledging the linkages

between the different systems and analytical units: the individual, the family,

the enterprise(s), the farm, and the community.

By applying general categories and concepts to the description and analysis of relevant

elements in specific local social fields, we are able to formulate appropriate

impact hypotheses and define indicator sets in a systemic manner. This gives us

17 See steps 1, 2, and 3 in the guidelines.


INTRODUCTION

enough flexibility to consider the dynamics of the local setting and programme conditions

when assessing intervention.

Since we explore various approaches in detail in Part 2 of the guidelines (steps 1, 2

and 3), we will only briefly outline them in this section. In the first step, we identify

contextual core issues that are related to the analysis of programme effectiveness

(scale and depth of programme outreach), the analysis of environmental conditions

(external factors), and the analysis of the client. We use two main concepts: the

Household Socio-economic Portfolio 18 and the Vulnerability of Livelihood. With

these concepts, we are able to take the aforementioned difficulties into account and

set the cornerstone for sound IM and IA.

In the second step, core impact hypotheses are formulated on the basis of step 1.

We identify impact domains for all levels of analysis of the target population (individual,

household, business/farm, and community) with respect to important mediating

variables and gender issues. In the third step, core impact hypotheses are operationalised

by identifying meaningful indicator sets that fulfil the methodological requirements

for each analytical level.

The Methodological Framework: Knowledge Creation 19

Impact measurement in microfinance encompasses a broad methodological spectrum.

At one extreme, large-scale, expensive and rigorous IA has been carried out

to prove that microfinance results in certain prescribed changes at the client and

household levels. Project monitoring and evaluation are overburdened to demonstrate

the relationship between interventions and highly-aggregated economic and

social development goals through quantitative measurement in such cases. At the

other extreme, projects rely solely on qualitative data. However, qualitative data

alone cannot provide the degree of confidence in conclusions that a fully-resourced

scientific approach can yield. We have sought a middle ground that keeps the tradeoffs

between data credibility, applicability, and cost-effectiveness to a minimum. The

methodological question in IM for microfinance is no longer ”What is the optimal

method?” but ”What mix of methods is most appropriate and how should we

18 Compare the analytical framework designed in Chen/ Dunn (1996).

19 Compare Hulme (2000).

31


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

design an optimal methodological mix?” Thus, our goal is to create knowledge

on the basis of triangulation rather than to use a methodological blueprint.

We distinguish four main categories of methodological approach in impact measurement:

scientific, qualitative, participatory learning and action (PLA), and management-oriented,

with emphasis on the latter. There is no inherent incompatibility

between these approaches – each brings something valuable to the IM agenda - so

we can take from each of the four categories as needed. Practitioner-oriented IM

and IA focus on the points outlined below.

Management-oriented systems models:

32

• emphasize comprehensiveness in evaluation, placing evaluation within a larger

framework of organizational activities

include aspects of the models below

Scientific experimental models:

• plausibly explain causality of impact in relation to programme intervention

• How can external factors that influence development policies and

goals, as well as project interventions be controlled?

• How can a control group be devised to ascertain what would have happened

without programme intervention?

Qualitative/anthropological models:

• demonstrate the importance of observation, the need to retain the phenomenological

quality of the evaluation context, and the value of subjective human

interpretation in the evaluation process (naturalistic or fourth-generation

evaluation)

Participant-oriented models:

• encourage a two-way learning process by allowing the target population to

determine their own indicators and development goals

In practitioner-oriented impact measurement, problems are complex, so methodologies

should be varied. Thus, it is desirable to consider the different methods and

inherent paradigms as a continuum with two poles: one quantitative and one qualitative;

and to consider this polarization as an attempt to guide your thinking about

aspects that might otherwise be left out.


INTRODUCTION

Quantitative impact monitoring and assessment typically uses random sampling surveys

and structured interviews to collect data – primarily quantifiable data – and analyses it using

statistical techniques.

Data include numbers and statistics

Data are predictable and controllable

Goal is to know – to prove

Often deductive in nature

On the basis of a quasi-experimental design (comparison with a control group) this

approach seeks to prove impact: how the living conditions would have been different if the

client had not received programme services.

Qualitative IM and IA typically apply purposeful sampling and semi-structured or interactive

interviews to collect data. Most data rely on people’s opinions, attitudes, preferences,

priorities, and/or perceptions about a subject. Data is normally analysed using sociological

or anthropological research techniques.

Data include words, pictures, and graphs

Results describe qualities

Intended to enhance meaning, challenge values

Usually field- or case-based

Goal is to understand the impact chain

Often inductive in nature

Following any one of several sampling strategies, the evaluation team can select

microfinance clients who represent the following in order to document why an individual or

group either follows or does not follow a specific pattern. 20

Extreme or deviant cases (failure/success);

Obvious cases that clearly demonstrate the phenomenon you want to assess;

Maximum variation (urban/ rural);

Typical cases that demonstrate an average outcome;

Typical high- and low-performing clients;

Cases that make a specific point dramatically;

Client recommended clients (snowball or chain method) to identify who knows the

20 SEEP (2000)

most about particular phenomena.

33


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

The sequencing of quantitative and qualitative techniques has implications for how

the two methodologies can be integrated. The best knowledge creation can be

achieved through incorporating various methodologies 21 described below.

34

• Use quantitative survey data to determine the households and communities to be

studied through a qualitative approach.

• Use a quantitative survey to design the interview guide of a qualitative survey.

• Refine understanding of intra-household differences, especially of gender dif-

ferences and poverty by selecting a sub-sample and interviewing them intensively

with open questions.

• Use qualitative methods to determine the stratification of a quantitative sample.

• Use qualitative methods to determine the design of a quantitative survey ques-

tionnaire. Omissions would produce a form of specification error, (the relationship

between the variables measured would differ if a given variable were taken into

account rather than ignored; multivariate analysis). Often, omissions are not

random given the physical distance between researchers and their respondents.

• Use qualitative work to pre-test a quantitative survey questionnaire in the pre-

liminary design phase, identifying core hypotheses and indicators according to the

perceptions of the population, integrating qualitative questions.

• Use qualitative data to refine the poverty index: variables and their ranking for the

quantitative poverty index can emerge from qualitative work.

Examine, explain, verify, disprove, and/or enrich information from one approach with

that from another.

Examine: Generate hypotheses from qualitative work and test them with quantitative

methods.

Explain: use qualitative work to explain unanticipated results from quantitative data (at a

later stage of the survey, when data are collected and analysis begins).

Verify or disprove quantitative results through qualitative methods: many aspects of rural

livelihood are not monetized (compared to urban households).

Enrich: use qualitative work to identify issues or obtain information on variables not obtained

by quantitative surveys: perception variables (reflecting attitudes, preferences, or priorities).

21 Carvalho and White (1997)(2).


And merge the findings of the two approaches.

INTRODUCTION

• In order to derive one set of policy recommendations to improve project

intervention.

Guided by low-cost criteria, credibility and usefulness of results, a mix of quantitative

and qualitative instruments can meet a combination of management and accountability

objectives. Basic characteristics of quantitative and qualitative approaches

are listed in the following table. 22

Table 3: Characteristics of Quantitative and Qualitative Approaches

CHARACTER-

ISTICS

Interview

format

QUANTITATIVE APPROACH QUALITATIVE APPROACH

• Structured and formal, pre-de-

signed questionnaire that can be

modified and adapted after the pre-

test

Tools • Sample survey with control group of

newly-entered clients

Sampling 23 • Probability sampling

• Randomisation: each unit or

population has an equal chance of

being selected

• Stratified randomisation: population

is divided into different groups or

classes (according to socio-eco-

nomic aspects) called strata; a

sample is drawn from each stratum

at random

22 Compare Carvalho/ White (1997)

• Systematic randomisation: one unit

is selected at random and then ad-

ditional units are selected at evenly-

spaced intervals until the desired

sample size has been reached

23 Sampling methods are explained in detail in Part 2, ANNEX 4.

• Open-ended, semi-struc-

tured, and interactive

• Rapid Appraisal

• Participant Observations

• Case Studies

• Participatory Learning and

Action

• Purposive sampling

• Non-probability sampling:

sampling is accidental

when a person is selected

by accident because they

are available, or arrive at

your doorstep

• Snowballing: a key infor-

mant names other people

who should be contacted

by the surveyor to under-

stand specific aspects of a

situation under study

• Common sense sampling:

inclusion of a wide range

35


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

36

CHARACTER-

ISTICS

QUANTITATIVE APPROACH QUALITATIVE APPROACH

Sample size • 50 (non-clients) - 150 (clients) from

Geographic

coverage

each subgroup

• Wide, typically selected from the

whole intervention area

Average time Depends on time available and pre-

Statistical

analysis

existing information:

• Preparation: 1 week for gathering

background information and initial

survey design; 1 week for staff

training and pre-testing the ques-

tionnaire, and 1 week for final sur-

vey design (including sampling)

• Interviews take an average of 45 –

60 minutes: maximum 5 surveys

per day/ interviewer x 5 interviewers

= 25 interviews + 5 interviews by

supervisor = 30 interviews/day

per week: 30 interviews x 5 days =

150 interviews/week

• 2 weeks data treatment, entry, and

analysis (2 persons); 1 week report

writing

• Statistical analysis is important

• Double-difference method: before/

after (baseline data) and with/with-

of people or a variety of

different situations in the

study sample in order to

avoid error through sam-

ple bias by ensuring suffi-

cient diversity

• Quota or proportionate

sample: reflects the distri-

bution of socio-economic

groups based on the rela-

tive distribution of these

groups in the population

Depending on the type of

tool:

• 5 – 10 interviews per sub-

group (case studies) or 10

– 15 group interviews

• Small: typically a few re-

gions or communities

Depends on the type of tool:

• Group interview: (if staff is

already trained in applied

methods and instruments)

60 minutes (RRA) to 120

minutes (PRA/ PLA)

• Individual interview (key

person): (by trained staff)

20 to 30 minutes

• Descriptive statistics

through classification and

grouping of the answers


CHARACTER-

ISTICS

INTRODUCTION

QUANTITATIVE APPROACH QUALITATIVE APPROACH

out (clients/non-clients) comparison

with project and control group to

plausibly attribute effects

• Triangulation is employed

(simultaneous use of sev-

eral different sources and

means of gathering and

interpreting information)

• Systematic content analy-

sis and gradual aggrega-

tion of data from different

analytical levels

Use of Control Groups 24 : A Quasi-experimental Design

Control groups are employed to show plausible associations of cause and effect that

can indicate change. While control groups might be the most common and most

scientifically-reliable way to bridge the attribution gap in IA, they also produce logistical

and ethical difficulties for microcredit programmes. On the logistical side, it is

costly and time consuming to track data on people who do not receive credit and

other financial services. An ethical dilemma arises when you look at using impact

assessment as an ongoing management tool. It would be wrong to continuously

withhold credit from people who might be able to use it just to prove the value of the

credit. We therefore normally work with reference groups to avoid these difficulties.

People trained in statistics will quickly recognize weaknesses in each of the following

approaches. The statistical problem of selection bias arises as a threat to internal

validity. A selection threat is any factor other than the program that leads to posttest

differences between groups. Whenever outcomes differ between groups because

of prior group differences, there could be a selection bias. For management

purposes, however, the following methods can give enough of an indication of causality

to provide useful information for decision making. The first method of forming

reference groups is the most feasible and applicable for IA in microfinance.

• Develop comparisons between new applicants and clients who have been

with the organization through several loan cycles, using new applicants as a

proxy control group.

24 Cheston (1999), Mosley (1998), for detailed description: Mohr (1995).

37


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

38

• Use data generated by the government statistical office to develop profiles of

the average household in a community and how it has changed over time,

and compare this to changes among clients.

• Encourage a university to have students carry out ongoing research of people

outside of the programme who could serve as a control group.

The Operational Level: Quality Assessment and Process Orientation

Impact measurement tools used by management on a regular basis need to be integrated

into existing management activities. Optimal entry-points are project orientation

and planning phases. Integrating impact measurement tools from the beginning

will identify and involve various stakeholders in IM and allow relevant baseline information

about the core issues identified to be gathered before the project has

started. Impact indicators, in addition to performance indicators, can be incorporated

in the project planning matrix or logical framework. In order to strengthen the internal

management capacities of a microfinance programme, a management information

system (MIS) needs to be established. 25 An effective MIS will enhance the programme’s

effectiveness and efficiency by tracking information on operations in an

accurate, timely, and comprehensive manner. An MIS includes all the systems used

for generating information that guide management in its decisions and actions to

provide adequate or better services to their clients. A system that gathers data on

client impact, the accounting system and portfolio management should be institutionalised.

Client impact tracking is even less standardized than portfolio management

and presents major challenges to MFIs since few off-the-shelf systems and

software packages are available. In order to determine the information needs of an

MFI, it is necessary to identify the users of the information and evaluate the needs of

each user group. Precise data inputs and information outputs must be sufficiently

defined. An integral part of this process is information presentation, frequency and

timeliness.

Internal monitoring and evaluation capacities should be systematically strengthened.

Staff members should collect baseline information on important financial and social

indicators to form a sample population. On a periodic basis, the programme should

25 For a detailed description of the requirements of an effective MIS for an MFI, see Ledgerwood

(1999) and Mainhart (1999).


INTRODUCTION

analyse this data to determine trends among its clients. Managers should then explore

the issues raised by the analysis through client surveys or focus groups. Next,

programmes should compile such data and reflect on indications for the design and

strategy of their activities and services. On an annual basis, programmes should

publish a report describing program impact, the levels of transformation desired and

achieved, and any changes that they will be making in their program as a result. 26

The integration of IM and IA as a systematic management tool in on-going project

steering has been rare. The collection and evaluation of information about clients

has been neglected from the very start of most interventions. For the majority of

projects, socio-economic baseline data are not available. Hence, for projects that

have collected data during the loan application process, a consistent follow-up to

reliably obtain and evaluate information is often missing or fragmentary. However,

single steps and tools from these guidelines can be used selectively during any

phase of the project cycle, and incorporated into on-going planning, re-orientation

and implementation processes.

Guiding principles for comprehensive IM and IA:

Clarifying objectives and parameters to be monitored:

• Identification of the minimum data needed, including qualitative data

• Establishment of benchmarks with plans to monitor them at periodic intervals

• Establishment of an effective performance monitoring system (financial and

institutional viability) for additional outreach and scale information at the cli-

ent level to differentiate client groups (mediating variables)

• Specification of impact hypotheses and indicators

• Inclusion of a reference group as a point of comparison

Planning a mix of internal activity and external support:

• Staff members primarily collect and analyse data.

• Outside experts – ideally local ones - could be involved to verify the accuracy

of the data obtained, suggest improvements in the information collection

process, and assist management in thinking through the implications of the

impact analysis on program activities.

26 Compare the impact audit approach described in Chester et al. (1999).

39


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Implementing the overall M&E design:

40

Impact measurement tools should be incorporated in the regular data collec-

tion and monitoring process of the programme.

• The data collected and analysed should remain consistent over time and

location, allowing for trend analysis and comparisons of different geographic

areas; this leads to a systematic evaluation process

• Results should be reviewed on a regular (normally annual) basis and be-

come a key part of the planning process in order to improve intervention.

• The participation of stakeholders in all stages of the project cycle should be

ensured.

• Capacity building measures (restructuring, planning, training) in the field of

M&E should be implemented for partner organizations and programme staff.

Management Information System (MIS):

• The needs of the key information users must be identified in order to design

an appropriate system.

• The key indicators the users need to monitor the programme and perform

their job well must be identified.

• Additional information needed by users to understand the programme, the

organisation’s performance and achievement of overall goals must be identi-

fied.

• The requirements of a computerized MIS that can serve the essential func-

tions must be identified. (i.e., software, hardware, and a central database

that receives information from decentralized branches).

• A price must be fixed that the programme is willing to pay if it intends to har-

vest the benefits of a good MIS, including the costs of ongoing data entry,

maintenance, up-front development and purchase, employment of trained

personnel, training, etc.

Composition of the Team

It is worthwhile to mention the composition of the IM and IA team, because evaluation

always takes place within a political and organizational context that requires

considerable skill. On the one hand, capacity for team work, management capability,


INTRODUCTION

political dexterity and sensitivity to multiple stakeholders are needed. On the other

hand, high-quality field work, well-documented research, quantitative and qualitative

research techniques, incorporation of lessons from previous IA studies, and documented

use of triangulation are required. Field staff (credit promoters, etc.) should

not interview clients with whom they have personal relations. The most effective way

to avoid bias is to assign staff to zones other than those in which they work. Nevertheless,

it is just as important to consider the difficulties strangers will have locating

the clients to be interviewed. To avoid this problem, regular programme staff who

know each area can serve as guides.

In general, the following aspects should be taken into account when forming an

M&E team: 27

Areas of technical competence:

• Multi-disciplinary teams with experience in data collection, analysis and

setting up IM-systems are desired and required for triangulation.

• Gender orientation: a team consisting of both men and women facilitates a

gender-sensitive approach.

• Coordination capacity: systematic and well-documented data collection

and data use are important.

In-country work experience:

• Local team members. To assure the integration of IA as a tool in the programme

cycle, local staff should be trained, if necessary, e.g., in participatory

survey instruments.

• Language proficiency of local team members ensures that they are accepted

by the target population and that they understand the local dialect

and are familiar with the local socio-cultural setting.

Participation of various partners and stakeholders:

• Fair and objective perceptions. Internal and external views should be integrated

to triangulate information and reach a high level of unbiased results.

Outside experts could be involved to verify the accuracy of data, suggest im-

27 Compare CDE/GTZ/KIE (1998).

41


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

42

provements in data collection, and assist management with recommendations

concerning how to implement measures in response to IA findings.

Facilitation skills:

• An atmosphere of mutual trust and confidence ensures a sustainable

monitoring process. Transparency between interviewers/staff members and

the local population is necessary. Respondents will only give representative

answers to sensitive questions if they are motivated, especially in the case of

a reference group or drop outs.

• Communication capacity. The interactive procedures of impact monitoring

require the ability to address and resolve conflicts between stakeholders.

Analysis skills:

• Clear documentation of information and straight aggregation of data

are necessary to plausibly explain patterns of change and to affirm confidence

in results. Transparency in documentation and assessment is important.

Information should be saved for future planning. Specific analysis skills,

especially in statistical calculations, should be acquired by one or two people

on the team.

A properly-composed data collection team is fundamental to obtaining valid data. It

is essential to train and guide local staff members to carry out IM and IA processes.

At the local level, the target population should be actively involved in the process

from identification of core issues and hypotheses, to selection of impact indicators

and interpretation of results. The degree of qualification, experience and sensitivity

of the surveyors has an irreversible influence on data validity.

A Step by Step Approach

We are coming to the end of Part 1. How can we apply a comprehensive IM and IA

model based on the concepts and methods described? We have seen that in order

to analyse the complexity of the livelihood of our clients and to plausibly explain

changes due to programme intervention, we need to implement comprehensive IM

and IA. These processes are primarily built on a programme’s internal monitoring


INTRODUCTION

and evaluation systems and should thus substantially strengthen internal management

capacities.

The following illustration suggests that IM and IA are dynamic processes with a

good deal of interplay between the various stages.

Graphic 5: Steps in Impact Monitoring and Assessment

1. Identification

of Core Issues

5. Data Analysis

Portfolio

Performance

Programme

Outreach

4. Selection and

Adaption of

Survey Methods

The practitioner-oriented impact monitoring and assessment tools presented step by

step in Part 2 of the guidelines are based on the preceding theoretical and conceptual

discussion. We consider the paradigm of impact measurement as a comprehensive

tool of internal management for MFIs.

The implementation process consists of five main steps:

2. Formulation of

Core Hypotheses

3. Formulation of

Indicator Sets

STEP 1: IDENTIFICATION OF CORE ISSUES:

Baseline data about financial and institutional viability of a microfinance

programme with a focus on the client level are gathered.

43


IMPACT MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

44

Environmental conditions and potential factors of influence at all

operational units are assessed.

Decisive individual and household income-generating and survival

strategies are studied.

STEP 2: FORMULATION OF CORE HYPOTHESES

On the basis of the findings of step 1, impact domains for each level

of analysis are identified (referring to positive and negative impacts).

Contextual factors and gender-specific aspects are analysed and reflected

in the formulation of core impact hypotheses.

STEP 3: FORMULATION OF INDICATOR SETS:

Based on the outputs from step 2, main indicator sets for each level

of analysis and impact hypotheses are identified.

Impact indicators integrated in such a way that programme changes

can be assessed.

Mediating variables are selected to disaggregate information for programme

management and strategic policy use.

STEP 4: ADAPTION OF SURVEY METHODS

A comprehensive and integrated multidisciplinary approach is

adapted, consisting of:

A cross-sectional comparison of clients to non-clients using a survey

instrument addressing all key hypotheses;

In-depth interviews of a small sample of clients (esp. women) on either

empowerment or loan use and business development;

A survey of ex-clients about their assessment of programme impact

and programme services; and,

Focus group interviews with clients about their satisfaction with the

programme.

STEP 5: DATA ANALYSIS

The main principles of preparation, analysis and documentation for

quantitative and qualitative information are presented.

These principles and basic instruments guide users to develop their

own assessment instruments.


INTRODUCTION

Let us conclude with the proverb of the famous philosopher Aristotle that applies to

all kinds of implementation stages within the development process.

"Well begun is half done." – Aristotle

This refers to a truth that applies to any monitoring and assessment process: the

better the information, the more efficient the programme can manage its resources,

and the better the resources are utilized, the more effective the programme results

will be with respect to:

• obtaining and ensuring viable financial institutions in the field of microfinance;

and

• optimising programme outputs, i.e., obtaining positive impact at the client

level.

In other words, the quality of the IM and IA processes has an influence on the success

of the microfinance programme. The better the instruments are designed and

applied, the more correct and valuable the information will be. While we recommend

an integrated approach to IM and IA for microfinance, we realize that it is not

enough to prove whether or not the programmes funded by donors did any good.

We also do not insist on the application of rigorous scientific methods to measure

impact. Rather than these extremes, our goal in Part 2 is to guide you step by step

to develop impact measurement tools for ongoing management purposes. We will

focus on the question ”How can we improve positive impact and promote transformation?”

rather than the question ”Was there any impact?”

45


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

46


Part 2: Implementation Steps

2.1 INTRODUCTION

IMPLEMENTATION STEPS

"Well begun is half done." – Aristotle

We begin Part 2 of the guidelines with the same proverb we used to end Part 1. For

an evaluation team, having a clear outline of management needs takes them half

way to ensuring a meaningful evaluation. Expending adequate time and effort in

preparing a good scope of work results in important payoffs in terms of evaluation

quality, relevance and usefulness, and in terms of ensuring transparency for internal

staff and external players. If you devise relevant impact hypotheses and indicators,

develop and implement a sound sampling plan, work through an action plan, implement

and test your measurements, and develop a design structure, then data analysis

is fairly straightforward. You should develop your evaluation plan using a

participatory process.

A good work plan includes the following elements: 28

• Objectives: What is the need for the evaluation, its audience, and purpose?

• Background: What is the history and the current status of the programme?

• Availability of data: What are the existing performance information

sources?

• Scope of IM: Identify relevant assessment questions.

• Specify and adapt M&E tools: Select overall design strategy, prepare data

collection and analysis plan.

• Team composition: Identify adequate and qualified team members.

• Procedural requirements: Specify schedule and logistical needs.

• MIS: Define the reporting and dissemination requirements.

• Budget: Work out a financial plan.

The practitioner-oriented IM and IA tools presented in Part 2 of the guidelines are

based on the theoretical and conceptual discussion of Part 1. The examples pre-

28 Compare also CGAP Paper, Barnes and Sebstad (2000).

47


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

sented demonstrate that impact measurement is a comprehensive internal

management tool for microfinance.

The tools include the operational criteria below. They:

48

• Allow trend analyses over time (longitudinal design);

• Encourage comparison with previous impact data;

• Are easily implemented by project staff using participatory methods;

• Can be incorporated into existing management information systems;

• Build up internal M&E capacities; and,

• Cost the same as monitoring systems for financial and institutional performance.

Guided by the criteria of low cost, credibility and usefulness, a mix of quantitative

and qualitative tools will meet a combination of management and accountability

objectives. Understanding the criteria elaborated in Part 1 is essential to developing

a comprehensive impact monitoring and assessment process. Our goal is that you

will be able to respond to the following questions:

• Can the changes identified be linked to a client’s participation in the programme?

• How is the length of programme participation associated with impact?

• How are loan size and loan terms associated with impact?

• What are the causes of any negative effects encountered?

This will be accomplished in the following five steps:

STEP 1: IDENTIFICATION OF CORE ISSUES

STEP 2: FORMULATION OF CORE HYPOTHESES

STEP 3: FORMULATION OF INDICATOR SETS

STEP 4: ADAPTION OF SURVEY METHODS

STEP 5: DATA ANALYSIS

The steps and accompanying tools are based on the four broad categories of

measurement detailed below using triangulation of information.

• Unobtrusive measurements: a variety of measurement methods that don't

intrude on or interfere with the context of research (indirect measurements,

content/text document analysis and secondary data analysis).


IMPLEMENTATION STEPS

• Survey research: design and implementation of interviews and questionnaires.

• Qualitative research: a range of non-numerical measurement approaches.

• Scaling: major methods of developing and implementing a scale (to associate

qualitative constructs with quantitative metric units).

The basic methods and key features of the instruments are listed in the following

table:

METHODS KEY FEATURES GUIDELINES

SAMPLE SURVEY: GATHER STATISTICAL INFORMATION SYSTEMATICALLY BY

COLLECTING QUANTIFIABLE DATA

Interviews and

Questionnaire 29

• A cross-sectional comparison of clients to nonclients

using a survey instrument that addresses

key hypotheses; measurements predetermine

indicators before and after intervention

in specific intervals; reapplication results in

trend analysis about processes of change,

which is enriched by the qualitative information

obtained

• A computer-based client monitoring system

integrated in the loan application review process

to obtain baseline data

• A standard exit interview given to clients when

they leave the microfinance programme about

their satisfaction.

STEP 4: TOOL 1

STEP 4: TOOL 2

STEP 4: TOOL 3

RAPID APPRAISAL: A RANGE OF TOOLS AND TECHNIQUES ORIGINALLY DEVEL-

OPED AS RAPID RURAL APPRAISAL (RRA)

Text Analysis

and

Secondary Data

Semi-structured

Interviews 30

29

Compare ANNEX 3 and 4.

30

Compare ANNEX 2 and 3.

• Existing records, documents and procedures

are reviewed to gather biographical and other

general information about programme participants

in order to obtain baseline data about

specific subjects

• Analysis of secondary data on institutional and

financial viability is performed to appraise the

programme effectiveness in the context of client

outreach

• Based on interview guides, in-depth interviews

with key informants and focus groups are conducted

to create a knowledge base

STEP 1

STEP 1

STEP 2

STEP 3

STEP 4: TOOL 4

TOOL 5

TOOL 6

49


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

METHODS KEY FEATURES GUIDELINES

Participant

Observations

50

• Observations made by field researchers with

extended residence in a programme community,

using qualitative techniques and miniscale

sample surveys

• Direct observations of the physical surroundings

of the clients, their meetings, activities,

and interactions related to programme interventions

Case Studies • Retrospective interviews used to prepare histories

• Detailed study of a specific unit (a group, locality

or organization) involving open-ended questionnaires

Participatory

Tools 31

• Preparation by (intended) clients of participatory

tools such as timelines, impact flow chart

and preference ranking, village and resource

maps, well-being and wealth ranking, seasonal

diagrams, flux diagrams, problem ranking and

institutional assessments through group processes

assisted by a facilitator and their team

STEP 1

STEP 2

STEP 3

STEP 4: TOOL 6

STEP 4: TOOL 4

TOOL 5

STEP 1

STEP 2

STEP 3

31 Participatory instruments and tools are not described in these guidelines because they are detailed

in exhaustive alternative documentation (Compare: diverse GTZ publications and manuals,

especially IIED 1993, Schönhuth/Kievelitz 1994, CFSDR 1996, Schaefer 1997, CDE/GTZ/KIT 1998,

and Vahlhaus 1999).


OBJECTIVES

ACTIVITIES

RESPONSIBILITY

PARTICIPANTS

IMPLEMENTATION STEPS

STEP 1: IDENTIFICATION OF CORE ISSUES

Gather baseline data about financial and institutional viability

of the microfinance programme with a focus on the client level

Assess environmental conditions and potential factors of influence

for all operational units

Understand individual and household income-generating and

survival strategies

Analyse programme effectiveness: financial market conditions,

programme history, programme outreach and performance

Analyse environmental conditions: government policies, the

natural environment, physical, social, economic and institutional

infrastructures and the socio-economic context

Analyse the household: household composition, household

socio-economic portfolios, vulnerability of livelihood and gender-related

issues

IM Team, with microfinance programme management serving as

survey coordinators

IM Team, programme staff

TIMING Before beginning IM

Selected strategic key informants (local authorities, scientists,

programme officials) and focus groups (client groups)

At the identification/orientation stage

At each planning or re-orientation phase

METHODS/ TOOLS Rapid Appraisal Methods:

- Problem ranking; seasonal calendars, preference ranking,

DATA ANALYSIS

Understanding contextual realities ensures credible formulation

of impact hypotheses and indicators.

etc.

- Combination of semi-structured interviews conducted with

focus groups and key informants; time lines to explore re-

actions to crisis, Flux-diagrams, etc.

Secondary literature and programme documents:

- Gathering of information from existing documentation and

records (balance sheets, income statements, portfolio re-

ports)

Descriptive codes, interview summary sheets and matrixes,

portfolio reports

51


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

ANALYSIS OF PROGRAMME EFFECTIVENESS

We will begin by appraising the efficiency of the financial system as a whole, which

includes financial institutions, financial instruments, and general financial market

conditions. In the next step, you will look at the institutional and financial viability of

FIs closely.

Financial Market Conditions

In order to understand the size, growth, branch system, governance, and supervision

of an MFI, it is necessary to assess how the financial system works in a given

country context. The first step to doing so is conducting interviews with strategic key

informants on the topics listed in the box below.

QUESTIONS

52

• Which external contextual factors affect the supply of micro finance

services and the outreach of the MFI?

• How does the regulatory framework affect micro finance operators?

• What economic and social policies affect micro finance operators?

• Which institutions have direct or indirect influence on micro finance

operations, and what are the consequences?

• Which categories of financial institution exist? Are they operating in

the formal, semi-formal or informal (autonomous) sector? Which

sector is the most effective?

• Who are the suppliers of the micro enterprises operated by MFI cli-

ents: especially those that operate in the same intervention area?

Programme History: Financial Institution and Instruments

To ensure the reliability of the data and information to be collected, the evolution of

a microfinance programme must be carefully checked in order to understand

changes in service delivery and in depth and scale of outreach.


QUESTIONS

• How long has the programme been operating?

IMPLEMENTATION STEPS

• Does the MFI follow a minimalist approach or are programme inter-

ventions based on an integrated approach?

• What financial products and other services does the MFI offer?

• Have there been changes in geographic intervention areas?

• What evolutions has the programme been subject to? Why did they

occur?

• Did any changes in the strategic objectives of the programme take

Programme Outreach

place? What were the consequences (product design, target

population, etc.) at the operational level?

Outreach data at the client level serve as quantifiable proxies to demonstrate the

extent to which an MFI has reached its strategic objectives. They also make the

social costs associated with supporting the institution transparent. Such output-process

indicators result in rich information to differentiate types of clientele when we

disaggregate data and make strategic comparisons among client groups. 32

Two types of strategic objective indicators are usually implemented: 33

• Scale of outreach:

Number of clients served with different types of instruments.

• Depth of outreach:

Type of clients reached and their level of poverty.

Although these indicators are useful, they can sometimes be misleading, because

loans are granted for different terms and uses and may not reflect the income level

of clients. The second type of indicator, depth of outreach, can have very different

meanings depending on which clientele, population groups and sectors are targeted.

How the level of poverty of clients can be measured is a controversial subject in

32 Compare step 3 on Mediating Variables (p. 25).

33 Based on Paxton/Fruman (1998), and Ledgerwood (1999).

53


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

microfinance and goes beyond the objective of these guidelines. 34 Nevertheless, the

level of poverty of clients is usually determined by the average loan size or the average

loan size as a percentage of GDP per capita of new and repeated client loans.

Outreach indicators are both qualitative and quantitative and they are relatively simple

to collect if a regular performance monitoring system is used. In the framework

of IM, it is useful to track these indicators over time and compare them with the

stated goals of the programme.

In order to gather outreach indicators, we first ask questions about the characteristics

of the targeted population.

QUESTIONS

54

Identification of target groups:

• Does the project focus on rural or urban areas, or both?

• Does the project focus on female clients/ beneficiaries, or male, or

both?

Are these women and/or men:

• Self-employed micro-entrepreneurs;

• Small-scale farmers;

• Landless and smallholders;

• Resettled people;

• Indigenous people; or,

• Low-income persons in subsistence or remote areas?

The following list provides a comprehensive overview of possible outreach indicators

that can be easily integrated into an on-going monitoring process.

34 Compare Hatch/Frederick (1999), CGAP (1998)(2), and Ravallion (1994).


Clientele:

• duration of membership in the programme

• # of clients (% women)

• % of total target clientele served (actually)

• # of women as percentage of total borrowers

• # of women as percentage of total depositors

• # of urban/rural branches

• ratio of rural to urban branches

• ratio of rural to urban clients

Loan outreach by gender:

• # of currently active borrowers

• total balance of outstanding loans

• average outstanding portfolio

IMPLEMENTATION STEPS

• real annual average growth rate of loans outstanding during the past 3

years

• loan size: minimum and maximum

• average disbursed loan size

• average loan term

• nominal interest rate

• effective annual interest rate

• value of loan per staff member

Savings outreach by gender:

• total balance of voluntary savings accounts

• total annual savings as a percentage of the annual average outstanding

loan portfolio

• # of current voluntary savings clients

• value of average savings account

• average savings deposit as a percentage of GDP per capita

• value of savings deposit per staff member

• nominal deposit interest rate (per annum)

55


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Performance Indicators

By calculating performance indicators, project management determines the efficiency

and viability of MFI operations. The more sustainability an MFI operates, the

more resounding the impact of the intervention will be at the client level, and vice

versa. Performance indicators are usually calculated in the form of ratios and are

compared over a period of time. Such trend analysis demonstrates whether financial

and institutional performance is improving or deteriorating.

The balance sheet, income statement, and portfolio report serve as the information

sources from which performance indicators are calculated. Given the small

number of MFIs that rigorously measure their financial performance, a standard

range for each of the ratios does not yet exist, especially with a breakdown of portfolio

performance by gender-sensitive criteria. It is also important to take into account

that appropriate benchmarks for one country or sub-region are not necessarily

suitable for other countries or regions.

A description of the most commonly used performance indicators in the field of

microfinance is presented below. 35

Portfolio Quality:

Portfolio quality ratios report information on the percentage of non-earning assets,

which in turn decrease the revenue and liquidity position of an MFI. There are a variety

of ratios to determine portfolio quality. They can be divided into three areas:

repayment ratios, portfolio quality ratios, and loan loss ratios.

Productivity and Efficiency:

Productivity and efficiency ratios provide information about the rate at which MFIs

generate revenue to cover their expenses. Using trend analysis with these ratios, an

35 BMZ/GTZ-indicators see BMZ (1997); for detailed lists of other comprehensive institutional and

financial performance measurement systems see Ledgerwood (1998: 228ff), where several

systems are described in detail: Depth of outreach Diamonds; the CAMEL system, ACCION;

Financial ratio analysis, SEEP; PEARLS system, WOCCU; CGAP).It is important to consider that

most financial and institutional performance indicators refer primarily to lending activities rather than

to savings mobilization and insurance activities. Further reading with applications includes

Saltzman/Salinger (1998) on the ACCION-CAMEL System, CGAP (1998)(1), and Kantor/ Robinson

(1998).

56


IMPLEMENTATION STEPS

MFI can calculate whether they are maximizing their use of resources. Productivity

refers to the volume of output that is generated for a given input (resource or asset).

Efficiency measures the cost of providing services (loans) to generate revenue.

Financial Viability:

Financial viability refers to the ability of an MFI to cover its cost with earned money.

An MFI cannot rely on donor funding to subsidize its operational costs indefinitely.

There are usually two levels of self-sufficiency against which MFIs are evaluated:

operational self-sufficiency (revenue from credit and savings operations and investments),

and financial self-sufficiency (amount of revenue that has been earned to

cover both direct and indirect costs).

Profitability:

Profitability ratios determine an MFI’s net income in relation to the structure of its

balance sheet. They help managers know whether they are earning an adequate

return on the funds invested in their MFI.

Leverage and Capital Adequacy:

Leverage describes the extent to which an MFI borrows money relative to its amount

of equity. Thus, it states the relationship of funding assets with debt versus equity.

Capital adequacy refers to the amount of capital an MFI has relative to its assets.

ANALYSIS OF ENVIRONMENTAL CONDITIONS

For valid impact assessment, it is essential to consider a wide range of potentially

influential external factors. Such factors are context specific and often dynamic so

that even well-designed, identical projects will achieve different levels of performance

if they are implemented in different environments. Contextual factors influence

programme outcomes either directly, through programme operations, or indirectly

through effects on the income-generating activities and social welfare

of clients. They thereby alter programme impact in either favourable or

unfavourable ways. While it is usually not possible to quantify the extent of impact

that each factor has on project results, one can observe whether particular hypothesized

influencing factors are present or absent, and whether changes and trends

occur over a period of time.

57


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Two general issues must be raised for each category of contextual factors:


58

Is the context in which the project is operating relatively favour-

able or unfavourable?

Over a period of time (e.g. the first and second phases of IM),

Government Policies

how have the contextual factors changed and how much can

these changes explain observed changes and project results?

Government policies and regulations may affect microenterprise profitability and

agricultural production as well as microfinance operations. Many of the policy effects

are not directly measurable, and they may vary in direction and intensity depending

on the sector of the economy. 36

QUESTIONS

• Which policies affect the price or availability of goods used by mi-

croenterprises: interest rate controls leading to credit rationing, im-

port duties and quotas or exchange controls?

• Which policies affect the cost of hired labour: minimum wage laws,

labour legislation or labour-based taxes?

• Which policies affect the cost of purchased inputs: import duties,

exchange or price controls?

• Which policies affect output markets: price controls, effective rates

of protection, exchange rates or export taxes?

• Which policies, regulations, and/or enforcement mechanisms affect

the performance of regional and local governments?

• Has the country undergone any Structural Adjustment Programmes

that have influence on microentrepreneurs, the rural economy or

agriculture?

• What is the influence of the actual land tenure system on agricul-

tural production?

36 Compare Snodgrass (1997).


The Natural Environment

IMPLEMENTATION STEPS

In subsistence monetary economies, which exist in most developing countries, most

clients have incomes, either in cash or in kind, from both agricultural and non-agricultural

production. Therefore, agro-ecological risk factors must be studied, especially

when microfinance programmes operate in rural areas and target farmers.

Careful attention should be given to the specific conditions that have prevailed during

and between survey periods.


• Which agro-ecological determinants influence agricultural production

(output, culture)?

• Which climatic conditions prevail?

• Was it the hot/cold, dry/rainy season, when the survey was under-

taken?

• Was the season exceptional compared to other years?

• Have catastrophes occurred (in terms of natural disasters, outbreaks

of disease and/or epidemics, agricultural pests, etc.)?

Physical, Social and Institutional Infrastructure

Access to physical, social and institutional infrastructure strongly influences the profits

of microentrepreneurs and farmers by increasing or decreasing the risks associated

with expected income and product quality. Local socio-political structures and

institutions, as well as local leaders, exert strong influence on decision-making processes

concerning the allocation of productive resources, especially in traditional

societies. In addition, safety networks and institutions of mutual exchange regulate

economic exchange according to socially-accepted normative rules.

QUESTIONS

• Do clients have access to water, electricity, telephones, and/or

roads? What is the quality and cost of such services?

• Do clients have access to medical services, formal or other educa-

tional institutions (primary and secondary schools, religious schools,

etc.)?

59


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

60

• In what ways does the location (remoteness, etc.) of the sub-region

determine the access and use of basic physical infrastructure?

• What role do organizations, traditional and cultural factors (gender,

ethnicity, class, caste, race, age, religion, or linguistic group) play with

respect to political, economic and social interactions among clients?

• Which people are powerful in the local setting? What influence do

they exert, especially regarding decisions that determine scale and

types of economic activities?

• Are safety networks in place? What type? How do they function?

• Are these networks strong, e.g., with a high degree of obligation for

mutual help or are they weak in terms of social or geographic factors?

Economic Infrastructure and Socio-economic Context

Specific knowledge about local economic infrastructure is essential in order to

evaluate the potential and limitations of income-generating activities. The local economy

is strongly embedded in the socio-economic context of the location where the

MFI is operating.

QUESTIONS

• What are the income-generating activities of the clients?

• Which activities do men and women typically undertake?

• What is the profitability of the activities undertaken?

• What is the structure of the markets in which clients sell goods and

services, purchase inputs, obtain capital and recruit labour?

• Are there any non-competitive elements and/or distortions in the mar-

kets that hamper market demand and sales and thus decrease turn-

over and/or increase input prices?

• Do clients have activities in the agricultural sector and/or non-agricul-

tural sectors?

• What role does subsistence farming play in the production activities of

clients?

• Are clients subject to fluctuations in their income? If yes, why?

• Do clients have access to enterprise support services, and/or

agricultural extension services?


ANALYSIS OF THE HOUSEHOLD LEVEL

IMPLEMENTATION STEPS

Any realistic impact assessment in microfinance depends on a good understanding

of how poor people undertake income-generating activities. Economic efficiency is

determined by complex and dynamic social interactions that are embedded in the

local socio-cultural context. As outsiders, we can only understand the prevailing

structures and conditions if we speak with clients and observe their actions and behaviours.

Therefore, clients and other key stakeholders must be actively involved in

determining realistic indicators to measure impact and in interpreting the results of

IM in order to improve programme intervention. Thus, programme management

needs to ensure through selection of appropriate methods and tools that target

groups actively participate in the entire processes of IM and IA.

In order to develop meaningful results that can influence strategic microfinance policies,

we propose an analytical framework based on two main concepts. These concepts

also include more general categories to describe and analyse relevant factors

in local social settings in order to gain an understanding of how poor people manage

their lives. The concepts we use are Household Socio-economic Portfolio and

Vulnerability of Livelihood. These invite systematic use and allow results to be

compared across geographic regions and between different types of MFIs. Furthermore,

the use of these two concepts facilitates the abstraction of complex socioeconomic

interactions and structures into core impact hypotheses and indicator

sets.

The Composition of a Household

Our first step is to find a functional definition of a household with regard to variations

of household structures over time and between and within societies. Because the

income-generating activities of clients are firmly embedded in their household,

especially among poorer families in developing countries, we place the household at

the centre of our analytical framework and utilize the contextual definition as our

guiding operational survey unit.

61


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES


62

What actually constitutes a household?

How can the household be described as a unit of analysis?

Are household members part of a conjugal family (a family or kinship unit, or

a nuclear family) or do they share a common residence, or do they share com-

mon functions such as consumption, production, ownership or investment?

The Concept of Household Socio-economic Portfolios 37

It is virtually impossible to determine exactly where money goes, i.e., to separate the

systems within a household into which a programme client channels their credit,

savings, or surplus profits. Hence, we consider the fungibility of money as a vital

strategy of clients to manage their lives. We capture such strategies and systems

in a process-orientated analytical framework in order to understand causality -

rather than control for it; thereby focusing on the linkages between different systems

and analytical units: the individual, family, enterprise(s), farm, and community.

Recognizing the market and non-market spheres of production and interdependent

consumption structures of poor population groups in developing countries, an integrated

household model, where the household is both producer and consumer, is

applied. We study household socio-economic portfolios from a systems

perspective, which allows us to consider intra- and inter-household patterns of negotiation,

bargaining, and conflict in decision-making over resource allocation to

sustain livelihood. Individual preferences, physical, human, social, and financial resources,

and power relations can be studied with a gender perspective, considering

the wider social unit and networks (Graphic 6).

37 Compare with the analysis of household socio-economic portfolios in microfinance in Chen/Dunn

(1996), and Dunn (1997); on household models compare Doss (1996), Reardon (1997), and

Neubert (2000).


Graphic 6: Household Socio-Economic Portfolios

CULTURE

RELIGION

ENTERPRISE

FARM

SOCIETY

RESOURCE ALLOCATION

HOUSEHOLD

INDIVIDUALS

WOMEN

MEN

FAMILY

CLIMATE

The Concept of Vulnerability of Livelihood

SOCIAL

NETWORKS

& GROUPS

COMPLEMENTARY/ INTERDEPENDENT

INVESTMENT/PRODUCTION/

CONSUMPTION/ACTIVITIES

IMPLEMENTATION STEPS

POLITICS

ECONOMY

In many developing countries, decisions about household resource allocation activities

(production, investment and consumption) must be made under restrictive

conditions. A general climate of uncertainty exists due to the contextual constraints

imposed. 38 Low-income families and individuals are often exposed to acute and

chronic (structural) risk factors, which cause risk aversion. This seems contrary to

the cost-effective and profit maximization strategies normally employed by midlevel

entrepreneurs and commercial farmers. 39

According to the level of risk exposure, low-income clients follow a continuum of

strategic objectives in order to maximize the utility of available resources. This

continuum ranges from aspects of security and income stability to economic growth.

In such an unstable environment, financial and social intermediation services play

an important role in balancing the effects of insecure and restrictive economic, politi-

38

Analysis in step 1, A and B.

39

In on-going works about impact assessment for microfinance, results show that a key area in which

microfinance is expected to have a positive impact is risk reduction through consumption smoothing

and improved ability to undertake forward planning; compare, for example, with AIMS projects,

IFPRI research studies and in Schaefer (2000).

63


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

cal, socio-cultural and natural conditions by gradually reducing the vulnerability of

livelihood.

Using the Flux Diagram (Graphic 7) of the Concept of Vulnerability of Livelihood

as a guiding analytical tool, we should consider the following questions to

select and outline the core issues needed to interpret the non-linear cause and effect

relationships.

QUESTIONS

64

• How can the internal structures and dynamics within the household

be described in terms of decision-making patterns, resource alloca-

tion, negotiation power, production, consumption and reproduction

activities?

• Do these processes have a collective or non-collective character?

• What are the roles, rights, responsibilities, and taboos of women and

men and how are they distributed between genders?

• What are the socially-defined hierarchies?

• Which role do supra-household groupings play; including both social

groups (extended family units and kinship groups) and social net-

works?


Graphic 7: Vulnerability of Livelihood 40

IMPLEMENTATION STEPS

We will see that low-income households throughout the world employ behavioural

coping strategies that are hierarchical in nature. Such strategies are implemented

sequentially relative to the severity and duration of adverse conditions. Coping

strategies are usually primarily based on the following four principles:

• Variety (diversification);

• Overlap and redundancy (securing/maximizing income);

• Flexibility (minimizing high fixed costs); and,

• Reciprocity (mainly in safety networks and groups).

40 Source: Schaefer (2000).

GROWTH

SECURITY

Increased

possibilities

& supporting

mechanisms for using/ combining

available resources

STABILITY

MICROFINANCE PROJECT INTERVENTIONS

HOUSEHOLD SOCIO – ECONOMIC

PORTFOLIOS

Risk Reduction

Strategies (ex ante)

lack of ownership

& unequal access

to physical, human

and social capital

VULNERABILITY

OF LIVELIHOOD

UNCERTAINTY

limitations due to

political, economical

social and cultural

factors

Loss Management

Strategies (ex post)

agro-ecological

risks

65


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

To avoid the negative consequences of a loss (ex ante or ex post), two mechanisms

of risk management, detailed in the following box, are usually pursued by low-income

households.

Income-smoothing strategies to reduce the household’s/individual’s ex ante

exposure to risk:

66

• Engagement in low-risk (less profitable) income generating activities

• Diversification of less capital-intensive income-generating activities (by sector,

season, family member, etc.)

• Holding fixed costs to a minimum (rent, infrastructure, labour, etc.)

• Building insurance mechanisms (savings and assets for liquidation in case of

emergency)

• Risk sharing through spreading mechanisms (relying on kinship relations and

other social networks)

Consumption-smoothing strategies or ex post loss management strategies to stabilize

consumption (adopted according to the stage of destitution of the household):

• Insurance mechanisms (liquidation of assets, use of social safety nets, savings,

etc.)

• Reversible coping strategies (temporary migration, sale of labour services,

reduced consumption)

• Short-term changes in food intake

• Disposal of key productive assets

• Extremely high-cost loans, which can jeopardize future household economic

welfare

• Desperate measures; reliance on charity, distress migration, break-up of

household

In order to assess real benefits, we must be careful in the interpretation of the coping

strategies employed by clients. The objective of an MFI is not that clients

cope better with crisis, but that they gradually reach a higher level of economic

and social security. Therefore, we need to formulate impact hypotheses

and indicators accordingly and interpret their results so that the limitations of

interventions can be discerned.


OBJECTIVES

ACTIVITIES

RESPONSIBILITY

PARTICIPANTS

IMPLEMENTATION STEPS

STEP 2: FORMULATION OF CORE HYPOTHESES

Identify impact domains for each level of analysis based on

the findings of step 1 (both positive and negative impacts)

Analyse contextual factors and gender-specific aspects and

reflect them in the formulation of core impact hypotheses

Define impact domains: identify levels of analysis; identify

core hypotheses and adapt them to local circumstances

IM Team, with microfinance programme management serving as

survey coordinators

IM Team, programme staff

Selected strategic key informants (local authorities, scientists,

programme officials) and focus groups (client groups)

TIMING Before beginning IM

At the identification/orientation stage of a project

At each additional planning or re-orientation phase

METHODS/ TOOLS Rapid Appraisal Methods:

- Semi-structured interviews with focus groups and key informants

to learn how clients assess their well-being; focus

groups are differentiated into new clients and repeated

borrower sub-groups

- Semi-structured interviews with programme officials and

field staff about their client observations

DATA ANALYSIS

Meaningful results depend on an interactive reflection and

interpretation process by all stakeholders involved.

Interview summary sheets

67


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

IDENTIFICATION OF IMPACT DOMAINS

Impact domains vary for each impact assessment according to the perceptions of

the target groups, the objectives of the programme, the local setting, the macro conditions,

and the study design. Therefore, all stakeholders should participate in the

formulation of impact hypotheses and indicators to ensure the relevance of the results.

Levels of Analysis

Our conceptual framework assumes that impact occurs at different levels of the family/household,

and is related to client decisions about the use(s) of their loan or

savings. The impact of intervention at each analytical level involves either direct or

indirect, short-term or long-term, intended or unintended processes of change that

take the transformation stages of clients into account.

We focus on four analytical levels:

1. Family/Household: (production/consumption unit)

At the household/family level, loans invested in income-generating activities contribute

to net increases in family income, asset accumulation, and labour

productivity. Social and economic security is increased through income that is invested

in assets such as savings, stocks and education, which makes it possible

to meet basic needs when the income flow is interrupted.

2. Individual: (women/men)

Individuals increase their well-being, which can be assessed either on the basis

of the commodities they possess, what they do with the commodities, or the utility

(happiness or fulfilment) that the commodities give the person. Change is

measured by client capacity to make decisions and investments that improve

business performance and personal income and, in turn, strengthen the family/household

economic portfolio, which often translates into personal empowerment.

68

Consonant with the concern for financial sustainability, evidence of women’s

higher repayment rates has led many programmes to target women. The

increasing access of women to microfinance services is assumed to initiate a se-


IMPLEMENTATION STEPS

ries of mutually-reinforcing spirals of economic empowerment for asset or wealth

creation, increased well-being for women and their families, and wider social and

political empowerment.

3. Enterprise: (non-agricultural and agricultural activities)

Changes in income, assets, employment, management strategies and volume of

production contribute to the viability, stability and growth of non-agricultural

and/or agricultural income-generating activities.

4. Community:

As a result of wider client social and economic relations, communities develop

economically through increased off-farm and farm activities that provide goods

and services, attract income, and create jobs. Participation in local organizations

based on democratic rules and norms empowers people and mobilizes social

responsibility.

Core Impact Hypotheses 41

In terms of programme design, a hypothesis refers to a presumed correlation between

inputs and outputs, and between these direct effects and their outcomes. We

will propose a list of hypotheses that have demonstrated validity in various programmes

and settings. These represent a starting point that should be tailored

(redefined, reduced, or expanded) in such a way that they reflect each programme´s

focus, clientele, and services. We should stress that the objective of microfinance

programmes is not to help participants cope better with crisis, but to help them

gradually reach a higher level of social security and economic growth. Thus,

we also include hypotheses that refer to negative aspects that could occur due to

programme participation.

It is also important to recall that the greater the number of hypotheses, the more

extensive the evaluation process will need to be. Careful attention should be paid to

the time frame, the dynamic and the dimension of impact processes, i.e., how the

41 A detailed description of different impact hypotheses is given in: Barnes/Little (1996), Chen (1997),

Mayouox (1997), Diagne (1998), Muazzam (1998), Saith (1998); Zaman (1998), Schrieder/Sharma

(1999), and Schaefer (2000).

69


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

length of programme participation is associated with impact and whether loan site

and terms are associated with impact.

We will present impact domains in detail following the presentation of impact

hypotheses.

Core Impact Hypotheses at the Family/Household Level

70

Participation in microfinance programmes leads

to improved long-term economic and social security

through wealth creation.

• Surplus income from income-generating activities and other productive

investments enables households to accumulate assets and thereby diversify

their holdings more effectively. This may result in higher returns for a

given level of risk exposure. Purchase of additional assets expands the

household economic base and allows for more predictability, control and

flexibility by generating a resource base for basic consumption and for future

planning while reducing vulnerability through improved risk management

strategies.

• Households are able to accumulate savings to improve their security

level, especially in a period of stress. Savings, whether in the form of cash or

other physical assets (such as livestock or jewellery) can be used to meet

consumption needs rather than disposing of other productive assets.

Surplus income also allows households to stabilise their income

sources through reinvestment in other income-generating activities, thus aiding

households to cope with seasonal or other short-term periods of stress.

• Households have the possibility of additional expenditures in human capital

such as improved nutrition, health, housing and children’s education, thus

leading to improved household viability and welfare long-term.

Possible Negative Impacts of Microfinance Services on Households:

• Under some circumstances, credit may aggravate the indebtedness of

households if the cash flow of the enterprise(s) is insufficient to cover loan

repayments, and no other sources of income are available. For households

at the margin of survival or in a period of stress, the level of indebtedness

may increase due to lack of other opportunities for productive investment.


IMPLEMENTATION STEPS

Core Impact Hypotheses at the Individual Level (women/men)

Participation in Microfinance Programmes Leads

to Improvements in Personal Well-being and Empowerment

(especially for women)

• Clients increase their economic value to the household and consequently enhance

their ownership of property and assets, which, in turn, reinforces

their control over resource allocation (savings, loans, and income) within

the household economic portfolio.

• Economic development acts as a catalyst. Clients become more financially

self-sufficient and economically independent, and thus experience increased

self-esteem and self-confidence. This leads to improved leverage

in decision-making and increased bargaining power. Clients play an

active role in the household, which, in turn, strengthens their future ambitions

and commitment to long-term activities and investments.

• Clients build their social and human capital due to their access to information

and knowledge through social intermediation. They increase their

mobility and interactions at the household and community levels, thus engaging

in community activities and enlarging and optimising their access to a

broader spectrum of markets and resources.

• If the programme also provides access to complementary services, such as

instruction in business management, clients benefit from increased options

with respect to business operations.

Possible Negative Impacts of Microfinance Services on Individuals:

• Credit may aggravate the indebtedness of individuals beyond their repayment

capacity in those cases where incoming cash flow is insufficient to

cover loan repayment, and/or loan funds are used by somebody else.

• Inter-household tensions may be provoked or intensified if targeted

microfinance interventions exclude access to some household members resulting

in pressures on qualifying individuals.

• An inappropriate programme design that absorbs client time and money incommensurably

may result in income losses for a microentrepreneur or

farmer.

• Because financial sustainability requirements to minimize costs have led

many microfinance programmes to seriously cut complementary services,

71


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

QUESTIONS

72

serious limitations exist to the degree to which some women may benefit. In

this context, we should carefully consider aspects that could cause negative

impacts when formulating impact hypotheses and indicator sets. 42

• Do women clients control loan use?

• Are women engaged in low-paid, traditionally female activities with a

low rate of return that are outside dynamic market structures?

• Does responsibility for household expenditure, household consump-

tion and unpaid domestic work limit the resources and time women

clients are able to invest in economic activities?

• Due to increased economic activity, women spend more time on busi-

ness activities, but are often still responsible for domestic work. Do

women increase their working hours per day so that they develop

health problems due to physical overburden? Do women compensate

for their increased work-load by engaging their children, particularly

their daughters, in house-work? If so, has the school enrolment rate

of the children in client households fallen?

• Is there an increase of domestic tension because women earn money

(possible consequences: divorce, abandonment, domestic violence)?

• Do men cut back on their own income-generating activities or do

women struggle to retain control of their earnings?

• Is there tension between women due to group repayment pressure?

• Do women have the time, power or means to become involved in

42 Compare Mayouox (1997).

wider social or political activities?


Core Impact Hypotheses at the Enterprise/Farm Level

IMPLEMENTATION STEPS

Participation in Microfinance Programmes Leads

to Enterprise Stability and Growth.

Participation in Microfinance Programmes Leads

to Stabilityand Growth of Agricultural Production.

• The viability of enterprises is increased by providing safe and accessible

savings facilities in which clients can accumulate start-up capital. This is particularly

important for poor people who start enterprises with savings. With

credit, new enterprises can be financed within households that dispose of

supplementary income sources to cover loan repayment.

• The production process is increased due to the possibility of having access

to more stable sources of finance through the sustainable provision of

loans and/or savings facilities. This, in turn, allows for a steady and more

predictable supply of inputs and enhances provident business management

strategies.

• Through regulatory reforms and policy interventions, more secure tenure,

better access to services and stable prices for both inputs and outputs

are promoted, which in turn stabilizes business income. Market intermediation

strategies also support entrepreneurs to diversify and expand markets.

• Improved access to larger amounts of capital enables diversification of inputs,

production processes, outputs, and/or assets, thus leading to enterprise

growth through risk spreading, reduction of costs, and increased productivity

and income.

• Enterprise growth through specialization is supported by the provision of

capital for the acquisition of inputs, equipment, tools, and other required

physical assets.

• Microfinance interventions increase incomes and expand employment

opportunities by contributing to the viability, stability and growth of enterprises

through increased resource bases and enhanced production processes.

• Access to financial services contributes to the maintenance of business viability

in light of downward pressures. Savings help to protect clients from

disposing of productive business assets or depleting working capital. Microfi-

73


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

74

nance services support entrepreneurs to reconstruct their productive capacity

and asset base following periods of stress.

• Social intermediation strategies such as group formation increase knowledge

and information about new business opportunities, resources, and

markets, and improve management practices. Additionally, if the programme

implements complementary training programmes, the technical and

managerial skills acquired may accelerate enterprise growth.

Possible Negative Impacts of Microfinance Services on Business/Farm Activities:

• In situations where markets are saturated, microfinance programme

interventions may contribute to reduced market share, lower product prices,

or displacement of other enterprises. Consequently, client profit margin will

be reduced and entrepreneurs/farmers might face indebtedness and/or

liquidity problems.

Core Impact Hypotheses at the Community Level

Participation in Microfinance Programmes Leads

to Economic and Social Development in Communities.

• Through the increased resource bases and enhanced production processes

of client enterprises and/or farms, communities experience a net increase in

local employment and income. Forward and backward linkages are also

developed, which ameliorates the local (and regional) economic infrastructure.

• Social intermediation processes contribute to the formation of social networks

and groups and increase participation in civic organizations.

Through increased mobility and the ability to act independently, clients may

experience increased recognition and respect from others and greater selfworth.

This may lead to political participation and the will to take on more

social responsibility, and hence to the creation of a stronger civil society.


IMPLEMENTATION STEPS

Possible Negative Impacts of Microfinance Services on Community Development:

• If several microfinance programmes operate independently in the same intervention

area, distortion of input and output markets might occur. This leads

towards economic disequilibria at the local and/or regional level.

75


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

OBJECTIVES

ACTIVITIES

76

STEP 3: FORMULATION OF INDICATOR SETS

Identify main indicator sets for each level of analysis and

select impact hypotheses based on the outcomes of step 2.

Operationalise impact indicators in such a way that programme

changes can be assessed.

Select mediating variables to disaggregate information for

programme management and strategic policy planning.

Define impact indicators: assess quality criteria, identify

appropriate types of indicators and analytical approaches,

select and adapt indicator sets, and specify mediating variables.

RESPONSIBILITY IM Team, with microfinance programme management serving as

survey coordinators

PARTICIPANTS

TIMING

IM Team, programme staff

Selected strategic key informants (local authorities, scientists,

programme officials) and focus groups (client groups)

Before beginning IM

At the identification/orientation stage of a project

At each additional planning or re-orientation phase

METHODS/TOOLS Rapid Appraisal Methods:

- Semi-structured interviews with focus groups based on

impact perceptions to learn about the criteria clients use

for assessing well-being

- Semi-structured interviews with programme officials and

field staff about their client observations

DATA ANALYSIS

Indicators should help create a minimum level of

comparability, coherence and consistency between

measurements.

Interview summary sheets


IDENTIFICATION OF IMPACT INDICATORS

IMPLEMENTATION STEPS

The selection of hypotheses and impact variables is intimately connected and cannot

be discussed separately. Indicators are markers that encourage management to

set benchmarks and assess results regularly. Defining impact indicators and required

information at the outset of a programme will ensure that the team collects

needed data in the most consistent and cost-effective manner. In the following section,

we present selection criteria as guidelines that can be creatively applied to

establish a set of preferential indicators that fit the needs and circumstances of each

programme setting. Consistent use of impact indicators will enhance management

capacity to adapt plans.

Quality Criteria (I)

The programme team should select a set of appropriate indicators that will serve to

assess the issues that it has identified as important to monitor. Indicators determine

the type of information that needs to be collected. Before we begin to formulate indicators

for each operational unit, let us look at six important criteria that determine

the quality of an indicator. 43

Good indicators are:

Simple: They are easy to use, simple to understand (in the given context)

and easy to operationalise (are technically feasible).

Valid: They are significant, relevant, and directly linked to programme

objectives and client perceptions.

Reliable: They are robust, accurate, replicable, verifiable and objective so

that if they were measured at different times or places or by different

people, the conclusions would be the same. Indicators must be both

uni-dimensional and operationally precise.

Sensitive: They are capable of demonstrating and capturing change in the

outcome of interest (directions and patterns of change due to a single

intervention).

43 Compare IDRC (1997) and The SEEP Network (2000).

77


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Practical: They are available. Data should be obtained in a timely, cost-effec-

78

tive, and efficient way, and the information provided by the indica-

tors should be worth the effort of collecting, processing, and analys-

ing the data.

Ethical: The collection and use of the indictors should be acceptable to

Types of Indicators

those providing the information.

Measurement indicators have a direct relation to the issue measured. They contain

quantitative information based on precise and replicable measurements and

predetermined, standardized questions. It is difficult to collect reliable absolute values,

particularly with regard to measuring the impact of microfinance programmes

(income status, enterprise profits or net worth) when dealing with low-income

population groups in developing countries due to: 44

• A lack of written records;

• Recall difficulties;

• The fungibility of money;

• The seasonality of income-generating activities;

• Fluctuations in income due to drought, family problems, etc.;

• The sensitivity of information; and,

• The problem of analysis and valuation (the rate of depreciation of fixed assets,

self-supplied inputs, family labour, etc.).

Therefore, proxy - or surrogate – indicators, which have a more indirect relation

to the issue being measured, are often used to assess impact. Proxy indicators emphasize

objects, short- and medium-term impacts, and may have a quantitative or

qualitative character. If correctly identified and applied in a set, proxy indicators can

measure impact more feasibly than direct indicators, and can therefore be more

reliably monitored. For instance, in order to measure profits as accurately as possible,

we must also estimate depreciation and other operating costs and then deduct

them from revenue. In most cases, such a calculation is not possible because the

44

Compare the concepts of household socio-economic portfolio and vulnerability of livelihood

described in step 1 of these guidelines.


IMPLEMENTATION STEPS

figures are not available and the risk of data inaccuracy is high. An alternative

possibility for measuring increased household income is the use of a proxy measure,

such as the value of expenditure for special durable goods, the possession of

specific durable goods, the percentage of clients reporting repairs, improvements or

additions in housing, or the percentage of clients reporting increased income over

the last 12 months. These proxies are often used because quantifying total household

income would require more time, resources, and accuracy than available. 45

Another category of indicators is experiential or anecdotal indicators, which are

based on qualitative and semi-qualitative information. These normally reflect client

perceptions, beliefs and attitudes, and emphasize subjective opinions and long-term

changes. Due to their inductive character, they illuminate the meaning of results,

challenge values, and are indispensable for a good understanding of the linkages

between different types of impact. Due to their subjective character, however, they

should be carefully triangulated against results obtained from other sources and with

other methods. Using the example above of calculating microenterprise profits, an

anecdotal indicator could be the percentage of clients reporting net business income

during the last 12 months.

Mediating Variables

Another category of variables used in impact monitoring is related to the collection of

information to assess the comparability of survey samples. Such mediating variables

enhance or constrain opportunities for change, but are not directly linked to

programme intervention. They refer to the demographic and socio-economic status

of clients, programme services, and community characteristics. Analysis that disaggregates

impact by mediating variables is useful for management purposes, to improve

intervention, and to develop policy recommendations. With the results, we are

able to establish programme links and discern differences in the impact of the variables

under study. For example, if we discover that sample groups are systematically

different in demographic and socio-economic characteristics, this might explain

differences in the results of various indicators. The first step in data analysis is to

compare the client and non-client characteristics of the samples. If differences are

45 For a detailed discussion on indicators referring to enterprise profits and net worth, see Daniels

(1999).

79


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

found, this information must be factored into any subsequent assessment of impact

domains.

Although we have already explored some mediating variables in step 1 it is worthwhile

to summarize different categories of mediating variables below.

Indicators of client demographic and socio-economic status:

80

• Gender

• Age

• Marital status

• Religion

• Educational status/functional literacy

• Gender of head of household

• Household size

• Dependency ratio (# of adult workers/# of dependents or minors)

• % of households with salaried workers

• Ownership of important productive and consumer assets (reflects relative

wealth in the intervention region)

Indicators of Programme Services: (see step 1)

• These indicators were collected from programme records in step 1:

• Length of time utilizing MFI services (new clients, 1-2 year clients, 2 years

and over, etc.)

• Amount of current loan

• Cumulative amount of loans received

• Increase in borrowing since becoming a client

• Amount of current savings deposited with the MFI

• % of savings above the required amount

Community Characteristics: (see step 1)

• Degree of remoteness

• Degree of economic development (proxy indicators: estimated population,

market access, access to transportation, proximity to major roads)

• Degree of social infrastructure (types of school(s) in community, distance to

the closest health centre with qualified staff, etc.)


Quality Criteria (II)

IMPLEMENTATION STEPS

To complete the criteria of what makes a good indicator, we list some general, but

nevertheless useful, principles below that aid in defining appropriate indicators for

impact monitoring.

Context-specificity: Indicators must be translated according to each specific

context.

Values attached: Indicators are expressions of the values of those who

choose them. Both the selection and the acceptance of

an indicator depends on values.

Indicator

combinations:

Indicators often work best and sometimes only in combination

– a single indicator does not necessarily tell you

enough.

Transitory nature: Indicators are transitory and sometimes seasonal. They

should be periodically revised and adjusted. Over the

course of the programme, conditions change, objectives

are altered, or better indicators are discovered.

Direct when possible: An impact indicator should measure the result it is intended

to measure as closely as possible. If using a direct

measure is not possible, one or more proxy indicators

might be appropriate.

Quantitative when

possible:

Disaggregated when

appropriate:

Quantitative indicators are numerical (numbers, percentages

or tonnage, for example). Qualitative indicators are

descriptive observations (opinions, experiences, descriptions

of behaviour). While quantitative indicators are not

necessarily more objective, their numerical precision

lends them to greater ease of interpretation and

comparability of data results. Qualitative indicators should

supplement quantitative indicators with richer information

so that programme results can be better understood.

Disaggregating programme outcomes by gender, years

of client activity, location or other dimensions is important

for management.

81


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

SELECTION OF INDICATORS

Indicators are not universal. Consequently, we cannot determine which indicators

should be selected or how many indicators should be used to measure a given

result. The response to these questions depends on:

82

• The complexity of the result being measured;

• The level of resources available for monitoring impact; and,

• The amount of information needed to make reasonably-informed decisions.

We can, however, state that the quality of measurement is a major issue. Assuring

that the data collection process does not include inaccuracies will help assure the

overall quality of subsequent analyses.

In microfinance, the following types of indicators are generally used:

• Economic indicators that refer to income, expenditure, consumption,

investment and assets;

• Social indicators that refer to education, health, nutrition and risk reduction;

and,

• Socio-political indicators that refer to empowerment and client relations to

the wider environment.

Levels of Measurement

Levels of measurement describe the relationship between the values assigned to

the attributes of a variable. Knowing the level of measurement helps determine how

to interpret the data from a variable. When a measure is nominal, the numerical values

are short codes for longer names. Knowing the level of measurement also

determines what type of statistical analysis is appropriate on the values assigned. If

a measure is nominal, data values cannot be averaged and a statistical significance

test cannot be performed on the data.

There are typically four levels of measurement.

• Nominal measurement: a numeric value names an attribute uniquely. No

ordering of cases is implied.


IMPLEMENTATION STEPS

• Ordinal measurement: attributes can be rank-ordered. Distances between

attributes do not have meaning. The interval between values is not interpretable

in an ordinal measure.

• Interval measurement: the distance between attributes is meaningful and

interpretable. It is therefore useful to calculate the average of an interval variable,

whereas it is not useful to do so for ordinal measurements.

• Ratio measurement: there is always an absolute zero that is meaningful.

Therefore, you can construct a meaningful fraction (or ratio) with a ratio variable.

There is an implied hierarchy in levels of measurement. At lower levels of measurement,

assumptions tend to be less restrictive and data analyses tend to be less sensitive.

Each successive level includes all of the qualities of the one below it and adds

something new. In general, it is desirable to have a higher level of measurement

(e.g., interval or ratio) rather than a lower one (nominal or ordinal).

We need to transform - or operationalise - the identified indicators in order to assess

programme impact. Possible analytical categories are. 46

Describing: Nominal Measurement

Gives us qualitative information that describes one or more aspects. Categories

should be identified to group main issues.

Examples:

To what extent does the client value her/his negotiation power: in business

contracting, family decision-making processes and relations in the wider social

environment?

Does the client think she/he has gone through an empowerment process? If

yes, how can it be described: through increased self-esteem, self-reliance,

self-confidence, civic participation, changes in decision-making processes, financial

self-sufficiency, etc.

46 Compare to the SEEP Network (2000) and Trochim (2000).

83


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Classifying: Ordinal Measurement

Gives us information about non-gradual categories

84

Examples (with codes):

Gender of the client: male (0) / female (1)

Does the client keep business records: yes (1) – no (2)

Scaling or Rating: Interval Measurement

Gives us a gradual description:

Examples (with codes):

Quality of goods can be: very good (5) - good (4) - average(3) - bad (2) -

very bad (1)

Profitability of income generating activities can be: very high (5) – high (4) –

medium (3) – low (2) – very low (1) – negative (0)

Measuring or Counting: Ratio Measurement

Gives us exact numbers:

Examples:

Prices of goods, values of sales in constant $ or EURO

Number of livestock and/or of defined productive assets

Dependency ratio: adult workers in household/minors + dependents

Percentage of clients who have invested in new technologies during the

preceding year

For all of these categories, a focus on absolute values bears the risk of inaccuracy.

Hence, in the analytical section, we focus on degrees, directions, and patterns of

change in order to assess programme impact. In the following tables, we have listed

main indicator sets and the corresponding analytical approach in order to operationalise

impact hypotheses at each analytical level. The indicator sets noted have been

determined to be suitable in previous microfinance impact assessments. For the

impact monitoring of each programme, these should be carefully reviewed and

adapted to programme objectives, urban and rural differentiation, and to a gendersensitive

definition of goods, values, norms, and assets. Survey periods are usually

12 months. Numbers and values should be carefully aggregated according to that

timeframe.


CORE INDICATORS AT THE FAMILY/HOUSEHOLD LEVEL

Participation in microfinance programmes leads to:

ECONOMIC & SOCIAL SECURITY THROUGH WEALTH CREATION

IMPLEMENTATION STEPS

INDICATORS: variables ANALYTICAL APPROACH: attributes and values

Increased Ownership of

Household Durable

Goods

(key household assets – must

be adapted to local setting and

gender aspects: ex.

bicycle/motorcycle/ car;

radio/tape player/ TV;

chairs/tables/ benches/beds;

stove/ refrigerator/other

household utensils)

Increased Effectiveness

in Coping with Shock

Improvements in Human

Capital:

Education

Housing

Nutrition/Food

Security

Health

Children Without Negative

Consequences

• Average value of household income; percentage change (p.c.) in average

value of household income (constant $, €)

• Percentage of clients owning each type of key asset, percentage change

in percent owning each type of key assets

• Average number of each type of asset owned, percentage change in

average number of each type of asset owned

• Percentage purchasing each type of good; percentage change in percent

purchasing each type of good during the preceding year

• Percentage owning no durable goods, percentage change in percent

owning no durable goods in the past year

• Aggregate value of expenditures for special durable goods in the last 12

months (const.$, €)

• Percentage engaging in profitable income-generating activities, percentage

change in percent engaging in profitable income-generating activities

(profitability must be defined according to contextual factors)

• Percentage reporting decrease in (seasonal) migration of family members

due to shocks in the past year

• Percentage reporting decrease in disposal of productive goods due to

shocks in the past year

• Rate of children (girls/boys) enrolled in school (primary/secondary school,

according to age)

• Percentage of children of school age attending school; percentage change

in percent of children attending school (girls/boys, primary/secondary

school, according to age)

• Percentage whose household school expenses for the past year have

increased (average, for each child in school age)

• Percentage making repairs, improvements, or additions in housing;

percentage change in percent making repairs, improvements, or additions

in housings during the preceding year

• Percentage undergoing specific construction (fixed or improved roof/home

expansion/improved water or sanitation system/installation of electricity),

percentage change in percent undergoing specific construction during the

preceding year

• How the diet has improved (qualitative and quantitative aspects) during the

preceding year (descriptive)

• Urban Households: average value of food expenditures/p.c.; percentage

change in average value of food expenditures/p.c. during the preceding

year (const.$, €)

• Percentage relying on food supply cultivated by their family in rural areas;

percentage change in percent relying on food supply cultivated by their

family in rural areas during the preceding year

• Rural Households: Percentage reporting increase in stock of cereals;

percentage change in percent reporting increase in stock of cereals

in the past season

• Percentage experiencing a hungry season (one meal per day) during the

preceding year, percentage change in percent experiencing a hungry

season during the preceding year

• Value of expenditures for medical treatment during the past year (p.c., in

const. $, €)

• Number of children (10 years and under) and older children (11 to 17

years) assisting in business activities in the last 4 weeks

• Number of children (10 years and under) and older children (11 to 17

years) missing school at least once in the last four weeks to assist with

the enterprise or household

85


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

CORE INDICATORS AT INDIVIDUAL LEVEL

Participation in microfinance programmes leads to:

IMPROVEMENT OF WELL-BEING AND EMPOWERMENT

INDICATORS ANALYTICAL APPROACH

Ownership of Property

and Assets (livestock/ vehicles

/luxury items for later disposal

during periods of stress;

for women: clothes, jewellery,

household utensils)

Control Over Resource

Allocation

Enhanced Self-confidence

& Self-respect

Increased Mobility

Leverage in Decisionmaking

Increased Bargaining

Power

Women Without Negative

Consequences

Increased Community

Participation

Knowledge, Know-how,

Skill and/or Ability Creation

86

• Percentage purchasing strategic assets; percentage change in percent

purchasing strategic assets during the preceding year

• Aggregate value of expenditure for special durable goods (see above list)

during the preceding year (const.$, €)

• Percentage reporting whose personal income decreased over the last 12

months

• Percentage reporting who had personal savings during the preceding year

• Percentage whose average value of cash savings increased, percentage

change in percent whose average value of cash savings increased during

the last year

• Women: Ratio of time spent (hours per day/calculated per season) for

productive vs. domestic activities; Ratio of time spent for productive vs. domestic

activities during the preceding year

• No rise in loan delinquency for an MFI (capability to manage credit)

• Gradual increase of a client´s loan amount

• Extent to which the client values her/his negotiation power (descriptive)

• Extent to which client values her/his awareness of options and rights in the

wider social context (descriptive, referring to household power patterns;

negotiating power in business, and in financial and labour markets)

• Percentage of total individual expenditures/household expenditures (p.c.);

percentage change in percent of total individual expenditures to household

expenditures (p.c.)

• Increased ease in purchasing inputs and selling products/services (location

of market)

• Women: Extent to which the client values her own contribution to the

household

• Decrease in percentage of women reporting pressure, tension and/or

violence in family due to programme participation

• Increased participation in local activities (membership in organizations

other than MFI)

• Increased acceptance of responsibility for common local activities (member

of programme committee, etc.)

If complementary non-financial services are provided:

Percentage participating in training/literacy courses/business management/farm

extension, etc.

Percentage practicing improved health care/family planning/nutrition practices


IMPLEMENTATION STEPS

CORE INDICATORS AT THE ENTERPRISE LEVEL (OFF-FARM

ACTIVITIES)

Participation in microfinance programmes leads to:

INCREASE OF ENTERPRISE STABILITY AND GROWTH

INDICATORS ANALYTICAL APPROACH

Increased Resource Base • Value of sales, percentage change in value of sales during preceding year

(constant $, €)

• Value of fixed assets; percentage change in value of fixed assets during

preceding year (const. $, €) (esp. repeat borrowers)

Enhanced Production

Process

• Percentage purchasing minor productive tools/equipment during preceding

year

• Percentage purchasing major productive tools/equipment during preceding

year (esp. repeat borrowers)

• Percentage using improved technology during preceding year

• Percentage who had to shut down their business(es) in the past 12 months

• Average length of period for those who were unable to conduct their

business over a period in the past 12 months

Increased Revenue

• Percentage of clients reporting an increase of net income in their businesses

during the preceding year

Diversification of Income

Sources

• Increase in calculated enterprise profit over the last 12 months (const.$, €),

percentage change in calculated enterprise profit over the last 12 months

(const.$, €)

• Number of paid employees by gender, average number of paid employees

by gender during preceding year

Specialization of

• Number of new paid employees by gender; average number of new paid

Production/ Service

employees by gender during past year

Delivery

• Percentage change in paid employment by gender during the preceding

year

• Number of unpaid employees by gender; average number of unpaid

employees by gender during the past year

• Number of new unpaid employees by gender during the preceding year;

percentage change in unpaid employment by gender during the preceding

year

• Percentage change in unpaid employment by gender during the preceding

year

Improved Financial

Management

• Percentage of working proprietors paying themselves wages out of

business revenue and percentage of clients that adopted this practice

since becoming a client

• Percentage decreasing their liabilities during the preceding year

Improved Business • Percentage buying inputs at wholesale during the preceding year

Practices

• Percentage adding new products/services or improving the quality of their

products/services during last year

• Percentage selling in new markets/locations during the preceding year

• Percentage investing in higher risk (higher profit) activities (esp. among

repeat borrowers) during last year

• Percentage calculating profits based on records of costs and earnings and

percentage of clients who adopted this practice since becoming a client

• Percentage keeping business money separate from money for personal or

family/household expenses and percent who adopted this practice since

becoming a client

87


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

CORE INDICATORS AT THE FARM LEVEL

Participation in microfinance programmes leads to:

STABILITY AND GROWTH OF AGRICULTURAL PRODUCTION

INDICATORS ANALYTICAL APPROACH

Increased Resource Base

Increased Revenue

Increased Productivity

Improved Financial

Management

Improved Business

Practices

88

• Value of crop production (cash crops/cereals/vegetables/fruits and others);

percentage change of crops production in past season (constant $, €)

• Value of sales, percentage change in average value of sales during

preceding year (const.$, €)

• Percentage of clients reporting an increase of net farm income during the

preceding year

• Share (in ha) of crop production for commercialisation to total crop

production; percentage change of share (ha) in crop production for

commercialisation to total crop production in past season

• Average value of fixed assets; percentage change in average value of

fixed assets (const.$, €) (especially for repeat borrowers) during preceding

year

• Value of expenses for high-quality inputs during preceding year for crop

production; percentage change in value of expenses for high-quality inputs

during preceding year for crop production (const.$, €)

• Percentage purchasing major productive equipment during preceding year

(especially repeat borrowers).

• Percentage with paid employees by gender, average number of paid

employees by gender during the preceding year

• Increase in average length of paid employment during main season in the

past year (average working days per labourer)

• Value of livestock at a given time in year; change in value of livestock at a

given time of year (const.$, €)

• Possession of livestock (number of animals according to genus and age),

change in possession of livestock at a given time of year

• Value of expenses for commercial animal husbandry; percentage change

in value of expenses for commercial animal husbandry during the

preceding year (const.$, €)

• Value of sales of animals and animal products; percentage change in

value of sales of animals and animal products during preceding year

(const.$, €)

• Percentage decreasing their liability during preceding year

• Decrease in percentage taking usufruct loans during preceding year

• Percentage selling their products in new markets/locations during the

preceding year

• Percentage diversifying their farming system (new crops, planting year

round, etc.) during the preceding year

• Percentage calculating profits based on records of costs and earnings and

percentage of clients who adopted this practice since becoming a client

• Percentage paying themselves wages and percentage of clients adopting

this practice since becoming a client

• Percentage keeping their farm business money separate from money for

personal or family/household expenses and percent of clients who adopted

this practice since becoming a client


CORE INDICATORS AT COMMUNITY LEVEL

Participation in microfinance programmes leads to:

ECONOMIC AND SOCIAL DEVELOPMENT

INDICATORS ANALYTICAL APPROACH

Development of Forward

and Backward Linkages

Increase of Paid

Employment

Enhancement of Social

Responsibility

IMPLEMENTATION STEPS

• Number of clients who developed a new business during the preceding

year

• Number of clients who reported an increase of net income in their businesses

during the preceding year

• Number of paid employees by gender, average number of paid employees

by gender during the preceding year

• Number of new paid employees by gender; average number of new paid

employees by gender during the preceding year

• Percentage change in paid employment by gender during the preceding

year

• Percentage of clients who purchased their inputs in the local region during

the preceding year

• Percentage of clients who purchased their products in the local region

during the preceding year

• Number of new constructions/renovations/expansions in local infrastructure

• Increased participation in local activities (others than those connected with

the MFI)

• Increased acceptance of responsibility for common activities (number of

members in committees, etc.)

89


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

90

STEP 4: ADAPTION OF SURVEY METHODS

The guiding question is ”How can we improve positive

impact and transformation?”

OBJECTIVES Apply quantitative and qualitative practitioner-oriented impact

monitoring and assessment tools in the context of a

comprehensive impact measurement process.

ACTIVITIES

Apply each tool according to programme requirements, the

information base, and the preparatory work that has been

completed.

Apply the adapted tools to support the M&E process so that:

Identified changes are linked to participation in the

programme;

Length of programme participation is associated with impact;

Loan size and terms are associated with impact; and,

Drivers for dysfunction are identified.

RESPONSIBILITY IM Team, with microfinance programme management serving as

survey coordinators

PARTICIPANTS IM Team, programme staff

Selected strategic key informants, client groups, local authorities

TIMING During each planning or re-orientation phase; a coordinated

and sequenced application of the TOOLS is indispensable

METHODS/ TOOLS TOOL 1: Impact Survey

TOOL 2: Client Monitoring System

TOOL 3: Client Exit Interview

TOOL 4: Case Study 1: Client History/Loan and Savings

Use Strategies

TOOL 5: Case Study 2: Client Empowerment

TOOL 6: Client Satisfaction

DATA ANALYSIS Depends on the TOOL applied, see step 5


APPLICATION OF SURVEY TOOLS

IMPLEMENTATION STEPS

When designing these guidelines, our goal was not to reinvent the wheel, but to

draw on the experience of previous impact assessments in the field of microfinance

and to use this expertise to guide you in selecting and adapting basic instruments

according to your objectives. The six basic tools presented in step 4 guide you to

make necessary adaptations and then design instruments that correspond to the

objectives and scope of your impact measurement, based on the information you

obtained in steps 1, 2 and 3. We present examples of how to analyse data and present

results for each tool. A more detailed description of data analysis is given in

step 5.

We did not include instruments based on the PLA approach, though it is an

appropriate way to implement IM and IA if your main objectives refer to empowerment.

A good example describing the PLA approach called "Learning Friend Diary

for Members of Self-Help Credit and Savings Groups" 47 was written by Helzi Noponen

in partnership with an Indian NGO.

47 See Noponen (1997, 1998) and the PIM Methodology developed by German, Pohl and Schwartz

(1997) for GATE/GTZ.

91


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 1 IMPACT SURVEY

92

TYPE Quantitative, standardized questionnaire

Qualitative semi-structured, open-end questions

Suitable for all types of microfinance programmes; tool should be

adapted according to programme design and client groups; must be

utilized on a regular basis in order to compare the evolution of results

DESCRIPTION

PURPOSE

IM TEAM

SIZE OF

SURVEY

TIME

REQUIRED

FREQUENCY

DATA

ANALYSIS

SOURCE

The Impact Survey is administered in the same way to three randomly-selected

groups: a group of long-term clients (active for three

years or more), a group of short-term clients (active for one to two

years), and a group of new clients (reference group) who have

joined the programme but have not yet received services. Depending

on the clientele, each of these groups must be split into proportionally

stratified samples (subgroups of women and men/rural and

urban clients).

This longitudinal impact study tests multiple hypotheses (step 3)

that correspond to various types of impact on each survey unit by

using a tool that combines practicability with credibility and validity.

Team of four members: team leader who coordinates the survey

and is responsible for logistical and other organizational tasks;

three staff members as surveyors

Each subgroup consists of a completed sample of 30 clients (minimum

size for a simple survey)

45 – 60 minutes (for one complete interview)

Annual or biannual survey round

Computer-based descriptive and univariate statistical analysis,

basic analysis of variance; analysis on a regular basis

The impact survey tool is primarily based on impact assessment

studies that were carried out by two GTZ microfinance programmes

between 1997 and 1999 in Niger and Ivory Coast. Hypotheses and

indicator sets were tested and the most meaningful ones were revised

in order to design a practitioner-orientated impact measurement

tool.

A similar impact survey was designed by Barbara MkNelly of Freedom

From Hunger and tested in Mali. 48

48 Compare MkNelly and Lippold (1998). The SEEP Network also presents a similar questionnaire

(2000).


IMPACT SURVEY

IMPLEMENTATION STEPS

The following questionnaire includes most of the information that responds to impact

hypotheses for quantifiable data. The strength of this standardized impact survey is that the

results obtained plausibly attribute changes to programme interventions because there

are two main groups – programme clients and a control group. 49 The control (or reference)

group should consist of non-clients. However, we advise that you form a control group of

new clients who have just recently entered the programme so that programme effects have

not yet yielded fruit.

Answers can be categorized and quantified, and results can be compared across various

sample groups. Applying the same tools in different MFIs and in different countries allows

organizations to systematise information in order to make strategic decisions related to

global policies. We advise that you conduct a pre-test of the tool before conducting final

interviews. Pre-testing not only trains the people that will conduct and supervise the

interviews, but also allows you to adapt the questionnaire based on a mutual learning

process.

The questionnaire should be given in the same way to all respondents (client and non-client

groups). Answers are expressed largely in terms of numbers corresponding to pre-coded

responses. Indicators and questions related to specific programme and outreach objectives

and corresponding to local settings, must be adapted. For example, MFIs operating in urban

or semi-urban environments should not include detailed questions about agricultural

production. Instead, they should include one question about the nature of exchange between

family members living in town and those living in rural areas.

Hypotheses tested by the tool:

Household/Family Level: Economic and Social Security through Wealth

Creation

Increased ownership of household durable goods: assets, income

Increased effectiveness in coping with shocks: productive investments

Improved human capital: housing, nutrition/food security, health, no

negative consequences for children

Individual Level: Improvement of Well-being and Empowerment

Ownership of property and assets: income, assets

Control over resource allocation: personal cash savings, leverage in

decision-making and increased bargaining power

Increased community participation: participation in local activities and

institutions, political power, social responsibility

Enterprise Level (Off-farm Activity): Increase of Enterprise Stability and

Growth

Increased productive resource base: assets, net worth

Increased revenue: sales volume

Improved financial management

Improved business practices: input supply, marketing, differentiation

between the enterprise and household

Enterprise Level (Farm): Stability and Growth of Agricultural Production

Increased resource base: assets, livestock

Increased productivity: expenses for high-quality inputs, sales volume

49

This is not the case if you apply only tool 2. Monitoring basic indicators and variables allows for

trend analyses if the survey is applied on a regular and repeated basis.

93


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

94

Improved financial management: decreased liabilities

Improved business practices: input supply, marketing, differentiation

between the enterprise and household

Community Level: Economic and Social Development

Development of forward and backward linkages: increased paid

employment

Additional Information:

Strategies to cope with difficult times

Loan use and savings strategies

Programme satisfaction

The following impact survey includes 78 questions, which you should adapt according to

your programme objectives and the prevailing local conditions. The impact survey should be

complemented by the case studies found in tool 3: Loan use and savings strategies and tool

4: Empowerment.


IMPACT SURVEY

IMPLEMENTATION STEPS

TOOL 1: Impact Survey/page 1

Client Identification Number: _______________ Community: ___________________ Code ❐❐

Name of Interviewer:_____________________ _____________ __Date of Interview: ❐❐ ❐❐ ❐❐

CLIENT INFORMATION (complete from programme records)

The objective of this section is to collect information about the respondent’s program experience that might be related

to the degree of impact. Much of the information can be gathered from programme records and filled in before the

interview begins.

Name of group: __________________________ Group’s current loan cycle: _____________________

Date joined the program: ____________________ Months in program: ___________________________

Type of client: participation: about 1 year ❐ more than 2 years ❐ non-client (new client) ❐

Number of loans client has taken: _____________ Is client delinquent? Yes ❐ No


Amount of 1 st loan: ________________________ Amount of current loan: _______________________

Cumulative value of all loans taken: ___________ Current savings amount: ______________________

Ratio of current savings amount/current loan amount: ________________________________________

SOCIO-DEMOGRAPHIC INFORMATION

The codes in brackets (red) refer to the abbreviations used to name the columns in the interview summary

(EXCEL Sheet).

1. Gender of client: 1 = female 2 = male [gender] ❐

2. Age of client: [age] in years

❐❐

3. Marital Status: [civil] 1 = married 2 = separated/ divorced 3 = widow ❐

4 = single

4. Years of school completed: [school]

❐❐

5. Severe sickness in the last 12 months (over 6 weeks) [sick] 1 = yes 2 = no ❐

6. Size of your household: number of total persons : [HHsize]

❐❐

how many of them are adults (18 years of age and older) [HHadul]

❐❐

(school-age) how many of them are children/youth from 7 to 17 years of age [HHchild1]

❐❐

(pre-school age) how many of them are infants up to 7 years of age [Hhchild2]

❐❐

7. How many of your school-aged children currently attend school? [schoolchild]

❐❐

How many of your school-aged girls currently attend school? [schoolgirl]

❐❐

Highest grade in terms of numbers of years in school (girls) [gradegirl]

❐❐

How many of your school-aged boys currently attend school? [schoolboy]

❐❐

Highest grade in terms of numbers of years in school (boys) [gradeboy]

❐❐

8. How many persons in your household are engaged in work that earns income or products?

Number economically active:

95


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 1: Impact Survey/page 2

INFORMATION ABOUT SOCIAL CAPITAL: COMMUNITY LEVEL/INDIVIDUAL LEVEL

The objective of this section is to collect basic information about the respondent’s engagement in local

institutions/organizations to assess her/his grade of civic participation. The table below provides a list

of possible institutions/organizations that you should adapt according to context.

11. Are you a member of one or several local institutions/organizations? If yes, what

function do you hold?

C

o

d

e

96

Type of Institution/

Organization

12 Microfinance program

13 ROSCA

14 Other FI/ group

15 Cooperative

16 Social association

17 Cultural association

18 Religious association

19 Labour union

20 Political party

21 Other:

22 Other:

1 = yes

2 = no

If yes, since:

(month/ year)

count the months

Are you a

member of a

committee,

etc.?

1 = yes

2 = no

If yes: specify If you are not

currently a

member, were

you a member

before joining

the program?

1 = yes

2 = no

ENTERPRISE LEVEL

12. Information on business operations (non-agricultural activities)

The objective of this section is to collect information about the respondent’s business operations (type

of activity, income, profit, labour, etc.)

13. During the past 12 months, did you operate one or several of your own enterprises

other than farming?

1 = yes 2 = no 99 = don’t know ❐

14. In the last 2 months, which of your activities earned the most income? List in

order.

Activity 1__________________________________________________________________

Activity 2__________________________________________________________________

Activity 3__________________________________________________________________

15. In the last 2 months, did you work for someone else on a salaried basis?

1 = yes 2 = no ❐

16. In the last 2 months, in how many of your own enterprises or income-generating

activities were you engaged? Numbers

❐❐


IMPLEMENTATION STEPS

TOOL 1: Impact Survey/page 3

17. Is this enterprise/are these enterprises: Activity 1: ❐

1 = primarily your own Activity 2: ❐

2 = primarily a household activity Activity 3: ❐

3 = a business partnership with other persons than household members

18. Reported net income: Did your net income during the past 12 months…

1 = increase remarkably 2 = increase 3 = stay the same 4 = decrease

99 = don’t know ❐

If the respondent has operated several business activities, you should ask the following questions for

each activity she/he was engaged in during the last 12 months.

Business/ Activity:

Activity no.1 (description)

Code: Years: 19. Number of Employees:

Working

Proprietor: M___ F___

20. Who are your clients?

1 = Neighbours 2 = Retail Stores 3 = Market ❐

4 = Other , describe:

21 What is your supply mechanism, who are your suppliers?

1 = wholesalers 2 = retailers 3 = local market ❐

4 = other, specify:

22. Are there any threats to the following? ❐

1 = client base 2 = production 3 = sales 4 = good supply

5 = other, explain:

23. What is your product cycle for this enterprise? (time from purchasing

inputs to selling most products)

1 = daily 2 = weekly 3 = bi-weekly


4 = monthly 5 = other, specify:

Paid

Employees: M___ F___

Unpaid

Workers: M___ F___

Number of children who

missed school or never attended

school so that they

could help you with this

activity: ___________

24. Sales trend; use the following code to indicate monthly trends:

1 = high 2 = average 3= low 4 = activity was shut down 99 = don’t know

Activity: J F M A M J J A S O N D

Activity A

25. For the same product cycle, what were your sales (cash and credit)?

Ask for minimum and maximum and estimate the average.

Sales per week Sales per 2 weeks Sales per month Total sales during total time

period (calculated over 12

months)

Cash

Credit

TOOL 1: Impact Survey/page 4

97


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

26. How much and what were your costs for the last product cycle?

Ask for minimum and maximum costs and estimate the average for data aggregation.

Expenses/type Cost per Cost per 2 Cost per Total expenses during total time

week weeks month period (calculated over 12

months)

27. For the same product cycle, what were your profits (in cash and in kind)? (after

enterprise expenses and before spending on your family)? Ask for minimum and maximum and

estimate the average.

Profit per week Profit per 2 weeks Profit per month Total profit in time

period (calculated over

12 months)

Cash

In Kind

98

Sales – expenses

per week (profit)

Calculated Profit

Sales – expenses

per 2 weeks (profit)

Sales – expenses

per month (profit)

Calculated profit (sales

– expenses) over 12

months

28. What is the amount of your current available working capital (raw materials, products,

liabilities of clients, etc.) for each activity?

Working Capital Amount:

Raw Materials

Finished products

Stock

Liabilities

Other:

Total:

29. During the last 12 months, did you make any of these changes? (ask the question for

each type of activity)

Type of changes/investments…

1. Added new products

2. Hired more workers

3. Sold in new markets

4. Sold in additional markets

5. Improved product quality

6. Developed a new enterprise

7. Reduced costs with a cheaper source of credit

1 = yes 2 = no 99 = don’t know

8. Bought inputs in greater volume at wholesale prices

9. Purchased small tools/accessories

10. Purchased major equipment and machinery

11. Purchased transport facilities


12. Invested (major) in enterprise site (building, storage

room, etc.)

13. Invested (minor) in enterprise site (chair, sales

counter, etc.)

14. Invested (major) in construction site (electricity,

water supply, telephone, etc.)

15. Other: specify

IMPLEMENTATION STEPS

TOOL 1: Impact Survey/page 5

30. Business inventory of machinery equipment:

Probe for all business equipment and take into account the five most important pieces. Additionally,

you should make a list of important equipment and categorise it if necessary. In your codebook, you

should indicate all equipment mentioned for analysis. After the interview you can aggregate the data for

all enterprise activities.

Description: Condition: Age: Value: Amount

Owned:

Activity A:

SUB –TOTAL A:

31. Business management practices:

In managing your enterprise… 1 =

yes

1. Do you keep records?

2. Do you have a separate budget for

enterprise activities and household

expenses?

3. Do you pay yourself a wage out of your

profit?

4. Do you have a fixed location for

producing/selling your products?

5. Do you have a fixed location for

producing/ selling your products that is

different than where you live?

2 = no 3 =

don’t

know

Did before

entering

programme

Have since

entering

programme

32. Do you currently hold personal cash savings in case of emergency or because you

plan to make a major business investment or purchase?

1 = yes 2 = no 99 = don’t know ❐

33. Do you have additional income?

1 = yes 2 = no 99 = don’t know ❐

34. If yes, from what sources?

1 = rental income 2 = remittances 3 = pension 4 Other: specify _______________❐

FOR FEMALE RESONDENTS ONLY:

Differentiation should be made according to seasonal aspects: dry season – rainy season, etc.

99


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 1: Impact Survey/page 6

35. During the last 12 months, did your daily/weekly workload in terms of business

activities:

1 = increase remarkably 2 = increase 3 = stay the same 4 = decrease

99 = don’t know ❐

36. During the last 12 months, did the scope of your household duties:

1 = increase remarkably 2 = increase 3 = stay the same 4 = decrease

99 = don’t know ❐

If at least one of the two previous questions was answered with yes, continue the interview:

37. Did you have the possibility of alleviating your increased work load?

1 = yes 2 = no 99 = don’t know ❐

38. if yes, how did you alleviate the increased workload? (multiple answers possible)

❐❐❐

1 = my (son)s helped me with the work 2 = my daughter(s) helped me with the work

3 = another family member helped me with the work 4 = I hired labour to help me in the business

5 = I hired someone to take care of domestic activities 6 = my husband/family hired labour for me

7 = other; specify:_______________________________

FARM LEVEL

Before starting with the interview, you should determine the farming systems utilized and the agricultural

season(s) according to context given your timeframe (12 months). You must ensure that expenses

for inputs do not overlap. In some regions, you might not be able to use familiar units of

measurement such as hectares to indicate surface, or kilograms to indicate quantity. In such cases,

you should determine appropriate scale units for conversion. In the preparation phase/after pre-testing

the questionnaire, indicate codes for all relevant crops. We have found that in many settings where

subsistence farming prevails, information about agricultural production is more reliable if you ask the

questions crop by crop (starting with the first question, then the second question, etc.) then aggregate

the data afterwards. Such a process helps respondents to recall the data.

39. Do you have income (cash and/or in kind) from agricultural production?

1 = yes 2 = no 99 = don’t know ❐

39-1. if yes:

If agricultural production is a primary subsistence and/or income-generating activity

of the interviewee, the following questions should be asked:

40. What is the value of your agricultural production, and what are the purposes:

Agricultural Value Total value

Thereof:

production (monthly) (12 months)

% Consump- % Sales % Other

tion

Individual

production

Family production

(what

is consumed

within the

household)

100


IMPLEMENTATION STEPS

TOOL 1: Impact Survey/page 7

41. What is the total surface of your holding?___________________ ha

42. What is the surface of your area under cultivation?

Crop Area Property status Harvest

Types in ha/

or other

unit

1 = owner

2 = tenant

3 = both

4 = family property

5 = belongs to spouse

4 = other

99 = don’t know

Subsistence crops (cereals, vegetables, etc.):

101:

102:

103:


Cash crops (cotton, cacao, coffee, etc.)

120:

121:

123:


Other (fruit trees, etc.)

130

131

132


Quantity

[number

of…in relation

to

respective

unit]

Scale unit

[in sacks,

pots, carloads.)

Conversion

into kg

(calculated

after interview)

Value

[in local

currency,

converted into

€]

43. Reported net income during the last agricultural season: Did your net income:

1 = increase remarkably 2 = increase 3 = stay the same 4 = decrease

99 = don’t know ❐

44. If you had any significant changes in the yield of your agricultural production

compared to the preceding season, can you indicate the causes?

_________________________________________________________________________

____________________________________________________________________

___________________________________________________________________

___________________________________________________________________

45. How did you spend your harvest proceeds? (indicate the codes for each crop in the first

column according to the information in the preceding table).

Crop

C

O

D

E

101

102

103


%

Sale

Type of sale:

1 = collector

2 = local market

3 = regional

market

4 = wholesaler

5 = other

Period of

sales:

1 = just after

harvest

2 = after 6 months

3 = just before

new harvest

4 = don’t know

%

Consumption(household)

%

in

stock

%

social

obligation

%

seeds

%

credit

repayment

%

Other

101


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 1: Impact Survey/page 8

46. Indicate your expenses for agricultural production during the last season:

Crop Seeds fertiliser Herbicide location of Hired la- Hired la-

insecticide machinery, bour: cash bour:

C

etc. means of

in kind

O

transport,

D

ox-team,

E

101

102

103


etc.

102

Other

47. During the last 12 months , did you make any of these changes? (ask the question for

each type of activity)

Type of change/investment 1 = yes 2 = no 99 = don’t know

1. Cultivated additional subsistence crops

2. Cultivated additional cash crops

3. Hired more workers

4. Sold in new markets

5. Sold in additional markets

6. Improved crop quality

7. Reduced costs with cheaper source of credit

8. Bought inputs in greater volume at wholesale prices

9. Purchased small tools

10. Purchased major equipment and machinery

11. Purchased transport facilities

12. Invested (major) in farm site (farm building, storage

room, etc.)

13. Invested (minor) in farm site (tools, etc.)

14. Other: specify

48. During the last agricultural season, did you pawn part of your standing crops?

1 = yes 2 = no 99 = don’t know ❐

if yes: What did you receive?___________________________________________________________

What was the amount/value? __________________________________________________________

What was the amount/value you pawned?________________________________________________

49. If you own your farm land, during the past agricultural season, did you sell part of

your real estate?

1 = yes 2 = no 99 = don’t know ❐

if yes: How many ha? ________________________________________________________________

What was the amount/value you received? _______________________________________________

50. If you own your farm land, during the past agricultural season, did you let or lease

part of your real estate?

1 = yes 2 = no 99 = don’t know ❐

if yes: How many ha? ________________________________________________________________

What was the amount/value you received? _______________________________________________

TOOL 1: Impact Survey/page 9


IMPLEMENTATION STEPS

51. Farm inventory of machinery and equipment (Probe for all farm equipment and take into

account the five most important pieces such as a tractor, plough, sprayer, etc.)

Additionally, you should make a list of important equipment and categorise them, if necessary. In your

codebook, you should indicate all equipment mentioned for analysis. After the interview, you can

aggregate the data for all enterprise activities.

Description: Condition: Age: Value: Amount Owned:

TOTAL:

52. Livestock breeding: What animals and how many of each species do you own?

Make a list of all species, count male/female, adult/young animals separately.

Animal/Species How many animals do Thereof: Total expenses

you/ did you own

[fodder, medical

[number]

treatment, herdsmen,

etc.]

Past

season

Currently number sold sales revenue

Cattle stock:

Bull, beef

(Dairy) Cow

Calf

Pig stock:

Sheep stock:

Goat stock:

Poultry:

Other (Camels, horses, donkeys, etc.)

53. Did you have additional income from secondary animal production?

1 = yes 2 = no 99 = don’t know ❐

54. If yes, what and how much was your revenue during the survey period?

Type Sales revenue (gross) Expenses (additional to above

indicated items)

Milk

Cheese

Eggs

Sale of skins

Other

TOOL 1: Impact Survey/page 10

Calculated

profit in survey

period

weekly monthly weekly monthly 12 months

103


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

HOUSEHOLD LEVEL WELFARE: ASSETS

55. Do you own any of these consumer assets?

Item How many of this

item do you own

(within the household)?

Radio, tape

player

Important

household

utensils,

specify:

Jewellery

Bicycle

Television

Stove

Refrigerator

Motorcycle

Car

Tractor

Other, specify

Other, specify

104

Was this item acquired

during the

last 12 months?

What value would

you ascribe (if you

were to sell today)?

For clients only:

Were you a member

of the MFI

when this item

was acquired?

1 = yes 2 = no 1 = yes 2 = no

56. During the last 12 months, were any major improvements, repairs or additions

made to your housing?

1 = yes 2 = no 99 = don’t know ❐

57. if yes, indicate which ones:

Type: Read and check: For clients only: Were you a

member of the MFI when this

took place?

1 = yes 2 = no 1 = yes 2 = no

1. Housing repairs or improvements

(such as fixed or improved roof,

walls, floors, etc.)

2. Housing expansion (such as built an

extra room, stock room, shed, etc.)

3. improved water system (tap water)

4. Improved sanitation system (such as

drainage/ sewage system, latrines,

showers, etc.)

5. Installed electricity

6. Installed telephone

7. Other, specify

HOUSEHOLD LEVEL: COPING WITH DIFFICULT TIMES

58. During the last 12 months, has your basic household diet

1 = improved 2 = stayed the same 3 = worsened 4 = don’t know ❐


IMPLEMENTATION STEPS

TOOL 1: Impact Survey/page 11

if improved:

59. How has it improved: (quality – quantity), describe: ❐

1 = the average quantity increased 2 = we had no problems during the hungry season

3 = able to buy more basic food stuffs 4 = able to buy more condiments, vegetables, etc.

5 = able to buy more dairy products, eggs, etc. 6 = able to buy more meat, fish, etc.

7 = able to buy more convenience food 99 = don’t know

if worsened:

60. During the last 12 months, was there a period when you and your family didn’t

have a proper diet, either because of a lack of food or a lack of money to buy food

stuffs?

1 = yes 2 = no 99 = don’t know ❐

if yes:

61. How long did this period last? Indicate in months ❐❐

62. What did you/your household do to overcome the difficult time(s)? ❐❐

(multiple answers are possible) ❐❐

1 = Migration of you/family member(s)

to seek employment 2 = You or family member(s) got local employment

3 = borrowed money at no cost 4 = borrowed money at cost

6 = borrowed food at no cost 7 = borrowed food at cost

8 = sold productive business equipment 9 = sold real estate

10 = other, specify:____________________ 99 = don’t know

63. During the last 12 months, was there ever a period when you couldn’t undertake

your business activities?

1 = yes 2 = no 99 = don’t know ❐

if yes:

64. How long did this period last? Indicate in months ❐❐

INDIVIDUAL LEVEL: LOAN USE STRATEGIES, CASH SAVINGS

65. During the last 12 months, did you borrow money?

1 = yes 2 = no 99 = don’t know ❐

If yes:

66. From which sources, how often and how much did you borrow (for clients: other

than from the MFI)? (multiple answers are possible)

Type/Institution Amount of

last loan

ROSCA (traditional)

Another MFI

Commercial bank

Individual (family

member, friend)

Total

amount to

repay (including

interest)

How much

is left to

repay

Total # of

loans received

from this

source

Cumulative

loan

amount

Cumulative

interest

amount

105


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 1: Impact Survey/page 12

Moneylender

Tradesmen

Company

Other:

Total:

67. How did you use the borrowed money? (Consumption, business activities, etc.)

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

68. Why did you use the money in this way? (The question explores coping mechanisms clients

apply to meet business or household financial needs.)

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

69. During the last 12 months, did your personal cash savings…

1 = increase remarkably 2 = increase 3 = stay the same 4 = decrease

99 = don’t know ❐❐

70. Do you currently have any personal cash savings (for clients: other than those

with the MFI)? (ask for the institution/person/place and the current amount)

Institution Current amount

1 = under the mattress

2 = with a confidant

3 = ROSCA (traditional)

4 = Bank

5 = other development programme

6 = other

TOTAL

ONLY FOR CLIENTS: LOAN USE STRATEGIES

71. How did you spend your last loan? (multiple answers possible, try to find out how much she

or he has spent for each purpose and calculate the percent of total loan amount)

Loan Use Strategies

Loan: Estimation in % of total loan

1 = Business activity (off-farm)

2 = New business activity

3 = Agricultural production (without livestock)

4 = Purchase of livestock

5 = Purchase of food

6 = Payment of school fees

7 = Health care expenses

8 = Purchase of clothes

9 = Purchase of household items

0 = no

1 = yes

amount

106


10 = Cash for emergencies

11 = Pocket money

12 = Repayment of another loan

13 = To assist somebody else

14 = Gave it to spouse

15 = Other, specify:

IMPLEMENTATION STEPS

TOOL 1: Impact Survey/page 13

72. Why did you use the loan in this way? (The question should give a clearer picture on constraints

to investing in the business and to meeting other consumptive expenses.)

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

73. Who decided to use the loan for this/these purpose(s)? (The question is important for

microfinance programmes that serve women and seek to help them gain control over their financial

resources.)

1 = you, yourself 2 = my spouse 3 = another family member

4 = other, specify:______________ 99 = don’t know ❐❐

74. Did you have difficulty repaying your loan?

1 = yes 2 = no 99 = don’t know ❐

If yes:

75. What were the causes for the difficulty in repayment? List up to three causes:

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

76. Indicate up to four things you like most about the MFI or programme:

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

77. Indicate up to four things you dislike about the MFI or programme:

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

78. What are your suggestions for improvement?

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

107


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 1: Impact Survey: EXCEL Interview summary worksheet

The following worksheet is an example of how to compile the interview results obtained with

tool 1. To compile the data, we use an EXCEL spreadsheet. This is a basic model of how to

tabulate the answers given or marked with a cross in the survey. We would advise you to

design a similar sheet for each main section of the survey. The columns indicate the variables

utilized. Each variable should have a code or abbreviation. The lines indicate the number

of respondents; i.e. for each respondent, one line. For data analysis, see step 5.

SHEET 1: SOCIO-DEMOGRAPHIC INFORMATION

Interview GEN- AGE CIVIL SCHOOL SICK HH- HH- HH- HH- SCHOOL- SCHOOL- GRADE- SCHOOL- GRAD …

DER

SIZE ADUL CHILD1 CHILD2 CHILD GIRL GIRL BOY EBOY

Date % …

1 11. Apr 99 2 27 1 0 2 13 5 5 3 75 0 1 100 1 …

2 23. Apr 99 1 43 1 5 2 7 2 2 3 25 0 1 50 2 …

3

4

23. Apr 99 2 33 1 0 2 6 2 2 2 100 100 1 100 1 …

5

6

7

8

9

10

NOTE: codes are indicated in the survey questionnaire

columns: questions indicated in the survey in red

lines: persons interviewed (number of interviews)

108


TOOL 2 CLIENT MONITORING SYSTEM

IMPLEMENTATION STEPS

TYPE Quantitative,

monitoring

standardized questionnaire, longitudinal impact

DESCRIPTION This computer-based client monitoring system is used to

develop a database that is integrated into the loan application

and review process. It is suitable for microfinance programmes

that screen clients on each loan application and seek a costeffective,

yet rigorous analytical instrument. The monitoring system

can be structured around a central database, which receives input

via a modem from loan officers around the country if the

programme has decentralised and equipped offices.

PURPOSE The tool provides information about the impact of programme

participation on client enterprises (enterprise profits, value of fixed

assets, and employment) and households (household income and

assets, expenditures on food). A few key impact indicators allow

the systematic monitoring of processes of change.

Strengths:

- Low-cost system for tracking key impact indicators over time

- Impact monitoring integrated into loan application system

- Storing information in computer database allows for a variety of

reports and comparisons between various input factors (loan

size, sector, business type, gender of client, length of

membership, etc.) and subsequent impact (changes in assets,

education, etc.).

TIMING Initial and repeated loan application

IM TEAM Loan officers, who utilize computers to directly input information

during the application process

TIME REQUIRED 30 minutes for each loan applicant

DATA ANALYSIS Worksheet, Reporting Formats

SOURCE Based on the Client Profiling System of the WORKERS BANK of

JAMAICA (initially developed by L. Blank and R. Webster/MIS).

The Bank operates through decentralized branches and offers

services that replicate a traditional ROSCA (rotating savings and

credit association). It has a full range of banking services and

targets low-income individuals with small, non-agricultural enterprises.

109


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 2: Client Monitoring: Loan Application Form/page 1

Number of loan: ________________

110

CLIENT MONITORING 50

Loan Officer: Office/Bank:

Date Client No.: Loan No.:

PERSONAL

Name: Nickname: ID Type:

Date of birth: Sex: M ❐ F ❐ ID #:

Address:

Ownership: How long: Tel. No.:

Name of Landlord/Mortgagor: Comment:

Name of Spouse/Other Contact: Occupation:

BUSINESS OPERATIONS

Name of Company:

Address: Tel. No.:

Business/Activity: Code: Years:

Number of Employees:

Who are your clients? Neighbours ❐ Retail Stores/Market ❐

Other ❐ Describe:

Working

Proprietor: M___ F___

Describe suppliers and supply mechanism:

Describe production and sales processes:

Paid

Employees: M___ F___

Unpaid

Workers: M___ F___

Are there any threats to the following?

Client Base: Yes ❐ No ❐ Good Supply: Yes ❐ No ❐

Production: Yes ❐ No ❐ Sales: Yes ❐ No ❐

Explain:

Who runs the business when you are sick or away?

BUSINESS INVENTORY OF MACHINERY AND EQUIPMENT

Description: Condition: Age: Value: Amount Owned:

TOTAL:

50

Based on the Workers Bank EARLY DRAW PARTNER PLAN application form found in Blank

(1998); slight modifications were made.


BALANCE SHEET

IMPLEMENTATION STEPS

TOOL 2: Client Monitoring: Loan Application Form/page 2

Assets Liabilities and Net Worth

Cash Short-term Payables

Bank Machinery

Accounts Receivable Long-term Payables

Raw Material Other Liabilities

Work in Progress Total Liabilities

Finished Goods

Business Machinery

Business Real Estate

Other Assets Capital/Equity

Total Assets Total

SALES TREND

USE THE FOLLOWING CODE TO INDICATE TREND H = high A = average L = low (per month)

Activity: F M A M J J A S O N D

A

B

C

STATEMENT OF REVENUE AND EXPENSES

Business Household (12 months)

High Average Low Calculated

12 months

Salary:

Self-employment

Activity A Rental Income

Activity B Remittances

Activity C Sale of Agriculture

Other

TOTAL: TOTAL:

Business Household

EXPENSES High Average Low Calculated EXPENSES Calculated

12 months

(12 months)

Purchases A Food

Schooling

Purchases B Health Care

Purchases C Leases

Rent Rent/Mortgage

Telephone Telephone

Light Light

Water Water

Transportation Transportation

Salary

Labour

and

Repairs

Other Construction

Other

Total Total

Net Income Net Income

111


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 2: Client Monitoring: Loan Application Form/page 3

112

CREDIT REFERENCES

Supplier Credit/Loan/Lease

Amount Date: Amount paid Outstanding amount

Credit 1:

Credit 2

Credit 3

Credit 4

Other

CASH RESOURCES

Institution Account Deposit Regularity Balance

Savings: Bank1:

(describe) Bank 2:

Partner:

Home:

NGO, etc.:

Other:

Checking:

HOUSEHOLD ASSETS

DO THE MEMBERS OF YOUR HOUSEHOLD OWN ANY OF THE FOLLOWING?

Yes No Last Year Yes No Last Year

How Many Yes No How Many Yes No

Sewing machine TV Set

Gas stove Video

Equipment

Electric stove Washing

machine

Refrigerator Bicycle

Freezer Motorbike

Air Conditioner Car

Fan Other Vehicle

Radio/Cassette Utensils

Stereo

Equipment

Jewellery

Other

SCHEDULE OF ASSETS SUBJECT TO LIEN

(Owned by Applicant)

Description Serial # Condition Age Value

PURPOSE OF DRAW

The above assets are fully owned by me and I hereby consent to pledging them as Security for my Workers Bank Early

Draw Plan.

I also hereby agree to participate in any evaluation study conducted by Workers Bank for the purpose of assessing performance

of the Programme.

Signature: _______________________ ________________________________

Applicant Bank/Loan Officer


Number of loan: ________________

DEBT CAPACITY

Loan:

Net Business Income

Net Household Income - Applicant

Net Income Co-Applicant

Total Capacity

Repayment Capacity %

IMPLEMENTATION STEPS

TOOL 2: Client Monitoring System: Worksheet

CLIENT MONITORING SYSTEM 51

Worksheet

Due to:

Lack of Confidence ❐ Threats ❐

Lack of Information ❐ Credit History ❐

Bank Policy ❐ Other ❐

Describe:____________________

Adjusted Monthly Repayment Capacity

GENERAL COMMENTS

Term:

Credit Committee Approval

High Average Low

High Average Low

Repayment:

51

Based on the Workers Bank EARLY DRAW PARTNER PLAN application form included in Blank

(1998); slight modifications were made.

113


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 2: Client Monitoring System: Reporting Format 1

Durable Goods (% Owned)

Sewing Machine

Gas Stove

Electric Stove

Refrigerator

Freezer

Air Conditioner

Fan

Radio/Cassette

Stereo Equipment

TV Set

Video Equipment

Washing Machine

Bicycle

Motorbike

Car

Other Vehicles

Utensils

Jewellery

Own None

Durable Goods (Nr. Owned)

Sewing Machine

Gas Stove

Electric Stove

Refrigerator

Freezer

Air Conditioner

Fan

Radio/Cassette

Stereo Equipment

TV Set

Video Equipment

Washing Machine

Bicycle

Motorbike

Car

Other Vehicles

Utensils

Jewellery

Own None

Durable Goods (Percent

Purchased Last Year)

Sewing Machine

Gas Stove

Electric Stove

Refrigerator

Freezer

Air Conditioner

Fan

Radio/Cassette

Stereo Equipment

TV Set

Video Equipment

Washing Machine

Bicycle

Motorbike

Car

Other Vehicles

Utensils

Jewellery

Own None

114

Workers Bank of Jamaica: Recommended Reporting Format

COHORT #1

Year 1 (2000)

At first At

App. Repeat %

App. Change

COHORT #2

Year 2 (2001)

At first At

App. Repeat %

App. Change

COHORT #3

Year 3 (2002)

At first At

App. Repeat %

App. Change


Household Income ($)

Household Expenditures on

Food ($)

Client Savings ($)

Value of Enterprise Sales ($)

Value of Enterprise Assets ($)

Employment in Enterprise

(Number)

Paid

Unpaid

Proprietor

IMPLEMENTATION STEPS

TOOL 2: Client Monitoring System: Reporting Format 2

Donor Strategic Objectives: Reporting Format 3

Workers Bank of Jamaica: Recommended Reporting Format

COHORT #1

Year 1 (2000)

At first At

App. Repeat %

App. Change

Male/

Female

COHORT #2

Year 2 (2001)

At first At

App. Repeat %

App. Change

COHORT #3

Year 3 (2002)

At first At

App. Repeat %

App. Change

M/F M/F M/F M/F M/F

REPORTING FORMAT FOR DONOR STRATEGIC OBJECTIVES

Year 1 Year 2 Year 3 Year 4

M/F M/F M/F M/F

Number of New Loans

Average Size of New Loans

Number of Repeated Loans

Number of Jobs Created

115


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 2: Client Monitoring System: Time Schedule for Monitoring Reports

REPORT

DATE

Month 2, Year 1

Month 7, Year 1

Month 1, Year 2

Month 2, Year 2

Month 7, Year 2

Month 1, Year 3

Month 7, Year 3

Month 1, Year 4

116

Time Schedule for Monitoring Reports

REPORT PERIOD OF COVERAGE COHORT STATUS OF

REPORT

Profile of new clients Month 1, Year 1

1

Interim

Profile of new clients

Profile of new clients

Analysis of change in

monitoring indicators

Profile of new clients

Profile of new clients

Analysis of change in

monitoring indicators

Profile of new clients

Analysis of change in

monitoring indicators

Profile of new clients

Analysis of change in

monitoring indicators

Month 1 – 6, Year 1

Month 1 – 12, Year 1

Month 1, Year 1 (new)/

Month 1, Year 2 (repeat)

Month 1 – 6, Year 2

Month 1 – 12, Year 2

Month 1 – 12, Year 1 (new)/

Month 1 – 12, Year 2 (repeat)

Month 1 – 6, Year 3

Month 1 – 6, Year 2 (new)/

Month 1 – 6, Year 3 (repeat)

Month 1 – 12, Year 3

Month 1 –12, Year 2 (new)/

Month 1 – 12, Year 3 (repeat)

1

1

1

2

2

1

3

2

3

2 or 3

Interim

Final

Interim

Interim

Final

Final

Interim

Interim

Final

Final


TOOL 3 CLIENT EXIT INTERVIEW

TYPE Quantitative, standardised questionnaire

IMPLEMENTATION STEPS

DESCRIPTION A standard interview that is given to (group loan) clients when they

leave the lending programme in order to give programme

management complementary information on how to improve

programme intervention.

PURPOSE Tool 3 focuses on client satisfaction and reasons for leaving the

programme. Questions focus on which clients leave the

IM TEAM

programme, when and why clients leave the programme, client

opinions about programme strengths and weaknesses, and client

perceptions about how programme services meet their needs.

Information gathered with this tool about the reasons why clients

leave programmes complements the data obtained in the impact

survey. Normally, only active clients are assessed in IM. Clients

who have experienced problems or negative programme effects

are often not monitored.

Loan officer

TIME REQUIRED 20 minutes

DATA ANALYSIS EXCEL spreadsheet

SOURCE The model spreadsheet is taken from Women’s Opportunity Fund

CETZAM in Zambia which is a member of the OPPORTUNITY

INTERNATIONAL Network. 52

52 Compiled by C. Garber for the USAID AIMS Project.

117


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 3: Client Exit Interview: Interview Format/page 1

118

CLIENT EXIT INTERVIEW

(Fill in before meeting the client)

Loan Officer:

Date of interview: [DATE INTERVIEW] Interview identification number: [Nbr.]

Name: Sex: M ❐ or F ❐ [CLIENT SEX] Client ID number:

[CLIENT ID]

Bank/Group name: [BANK ID] Address:

Entry Date: [Entry date] Exit Date: [Exit date]

Type of borrower: Individual loan: Group loan: Other:

[type loan]

Number of loans taken: [nbr loans] Number of loan cycles completed:

[loan cycles]

Size of last loan: [last loan size] Was final loan repaid?

YES ❐ NO ❐ [loan repaid]

If NO, amount in arrears or default: [if NO amount arrear]

Total amount of loans: [total loan amount]

Total amount of savings: [total savings amount]

1. Reason for exit according to programme MIS: [Q1]

A. ❐ Client voluntarily left the programme/group

B. ❐ loan group failed, so client left

C. ❐ Group/programme expelled the client

D. ❐ Other: _____________________________

2. Type of business financed by last loan: [Q2]

A. ❐ Retail E. ❐ Animal breeding

B. ❐ Service F. ❐ Fishing

C. ❐ Production/industry G. ❐ Other: ___________________

D. ❐ Agriculture

3. My loans helped me to: [Q3]

A. ❐ Start a business G. ❐ Meet school fees

B. ❐ Keep my business going H. ❐ Meet medical expenses/funeral

C. ❐ Change businesses I. ❐ Finance celebrations

D. ❐ Expand/improve my business J. ❐ Amass savings

E. ❐ Buy equipment/tools, etc.

F. ❐ Hire more workers

4. Through investing my loan, my income: [Q4]

A. ❐ Increased

B. ❐ Stayed the same

C. ❐ Decreased

5. My last loan was: [Q5]

A. ❐ Very big and difficult to repay

B. ❐ Within my capacity to repay and sufficient to meet the needs of my business

C. ❐ Very small and insufficient to meet the needs of my business


IMPLEMENTATION STEPS

TOOL 3: Client Exit Interview: Interview Format/page 2

6. Who made the primary decision to leave the programme? [Q6]

A. ❐ I made the decision (go to question 8)

B. ❐ Someone else in the family. Specify who: _____________________________

Why? ______________________________________________(go to question 8)

C. ❐ The group made the decision (go to question 7)

D. ❐ The programme made the decision (go to question 8)

7. In your opinion, what factors led the group to decide to exclude your continued

participation? (only if C in the previous question is marked!) [Q7]

A. ❐ Repayment problems

B. ❐ Attendance problems

C. ❐ Difficulties with other members of the group

D. ❐ Other reasons: _______________________________________________

8. What are the main reasons that you are leaving or left the programme? (Do not read

answers; multiple responses are possible)

a. Programme policies: [Q8a]

1. ❐ The loan amount is too small.

2. ❐ The loan length is too short.

3. ❐ The repayment schedule does not meet my

needs.

4. ❐ The loan became too expensive (interest, fees,

etc.).

5. ❐ The disbursement of the loan is not efficient.

6. ❐ I was unwilling to borrow because of other

conditions (obligatory savings, training, etc.).

7. ❐ I was unable or unwilling to attend all the group

meetings.

8. ❐ I had personal conflicts with group members.

9. ❐ I did not like the group pressure/rules imposed

by the group.

10. ❐ I did not like the treatment by the staff or had

personal conflicts with staff.

11. ❐ I found a programme with better terms.

Which one?

_________________________________________

_________________________________________

Why is it better?

_________________________________________

_________________________________________

c. Client business reasons: [Q8c]

1. ❐ I have enough working capital now to keep my

business going.

2. ❐ My business is seasonal; I will borrow again

when I need to do so.

3. ❐ I am unable to repay the loan because my business

is not doing well.

4. ❐ I decided to close the business and do

something else.

5. ❐ Other: ___________________________

_______________________________________

b. Personal reasons: [Q8b]

1. ❐I cannot continue because I spent the money on

a crisis (illness, death, etc.) .

2. ❐ I cannot continue because I spent the money or

a celebration (marriage, etc.) in the family/for

friends.

3. ❐ I am sick/pregnant/must take care of another

person and do not find the time or have the ability

to continue the business at the same pace.

4. ❐ I am moving out of the area.

5. ❐ A family member told me to stop participating in

the programme.

6. ❐ I borrowed the money for somebody else and

now this person has difficulty repaying.

7. ❐ Others: ____________________________

___________________________________

d. Community and economic disasters: [Q8d]

1. ❐ My business was ruined by a disaster (robbery,

flood, drought, fire, etc.)

2. ❐ Poor economic conditions have left my

customers with less money with which to buy my

goods/services.

3. ❐ The competition is too strong

4. ❐ Other:______________________________

F ______________________________________

________________________________________

e. Other reasons: [Q8e]

1. ❐ Other: _____________________________________________________________________

2. ❐ Don’t know

119


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 3: Client Exit Interview: Interview Format/page 3

9. My Bank has helped me in the following ways: [Q9]

A. ❐ Provided support when I needed help

B. ❐ Given me business ideas

C. ❐ Encouraged new friendships

D. ❐ Allowed me to develop my leadership skills

E. ❐ Other ________________________________

F. ❐ No help received

10. In general, my experience of participating in the Bank has been: [Q10]

A. ❐ Very positive

B. ❐ Positive

C. ❐ Fine

D. ❐ Negative

11. Other observations by the Loan Officer: [Q11]

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

120


IMPLEMENTATION STEPS

TOOL 3: Client Exit Interview: Interview Summary Worksheet

The following worksheet is an example of how to summarize the interview results obtained

from tool 5 based on a EXCEL spreadsheet. It provides a basic model of how to tabulate the

answers given or marked with a cross in the questionnaire. To analyse the data, you need to

make descriptive statistical calculations (mean values, percentage, etc.) of the answers

given (per column).

Name of of of Loan Officer: BS

last loan if NO total total

Bank Interview Interview Interview CLIENT CLIENT Entry Exit type nbr loan loan repaid amount loan savings

ID Date Date Date ID SEX date date loan loans cycle size arrears amount amount Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8a Q8b Q8c Q8d Q8e Q9

1 5 11. Apr 00 13 F 25. Jun 96 15. Feb 00 I 5 7 350 Y 1.050 450,00 A B C A C B 7 1+5 2A+C

2 3 23. Apr 00 219 M 1. Dez 98 3. Apr 00 I 1 1 150 N 125 150 0,00 C E I B A D 3+10 2 2 3 2 F

3 5 23. Apr 00 56 F 31. Jul 96 23. Apr 00 I 3 7 85 Y 355 23,43 B A B A B A 3 2 3 1 A+C

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

NOTE: codes are indicated in the questionnaire

columns: questions indicated in the

questionnaire [in red]

lines: persons interviewed (number of

interviews)

121


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 4

TYPE

122

CASE STUDY 1:

CLIENT HISTORY/LOAN USE & SAVINGS

STRATEGIES

Qualitative in-depth interview based on a mini survey

DESCRIPTION The client loan use & savings strategies tool provides qualitative indepth

information on hypotheses tested in tool 1.

There are two methods of implementation:

A: Client self-portrait: Ask the client how they have changed over

the past year with probing questions

B: Open-ended interview: ask categorical questions

PURPOSE Case studies should generate information and insight at all levels of

analysis, with particular focus on changes at the individual level.

The case study should explore loan use and savings strategies,

business evolution, details on activities in the household economic

portfolio, including options, choices, and outcomes during recent

crises, and other key events in the household. Client satisfaction

with the project should also be explored. The information gathered

is used to analyse the processes of change (patterns of behaviour,

directions and perceptions).

IM TEAM Staff members (see tool 1), taking into account that women should

be interviewed by female staff in some societies

SURVEY UNIT Five to ten clients from each subcategory (tool 1)

TIME REQUIRED 45 minutes

FREQUENCY Each case study should involve two or three in-depth interviews

with the selected client: one after the first round of the impact survey,

another after round two of the survey, and a third between the

two rounds of the survey. The third interview is advisable because it

allows staff to monitor changes during the two-year interval between

survey rounds and to capture and highlight seasonal variations

in a given year.

DATA ANALYSIS Interview summary sheet: aggregated diary matrix

SOURCE Nancy Horn of OPPORTUNITY INTERNATIONAL designed the

initial version of this tool. Adaptations were then made on the basis

of impact assessments undertaken by two GTZ microfinance programmes.

53

53 Compare Schaefer (2000).


IMPLEMENTATION STEPS

TOOL 4: CASE STUDY 1: CLIENT HISTORY/LOAN USE & SAVINGS STRATEGIES/

Checklist of Questions/page 1

This checklist condenses a matrix to fit on one page. An actual working matrix should cover several

pages, with half to one page per question.

CHECKLIST OF QUESTIONS

CLIENT HISTORY/LOAN USE & SAVINGS STRATEGIES

Name: _______________________________________

Location: _______________________________________

Date: _______________________________________

Socio-demographic information: age, marital status, educational level, health, type and # of

business(es), employees

Household Structure: # of family members, occupation of spouse, # of children and their age, children

enrolled in school/type of school, decision-making processes, responsibilities

Background: professional and other experiences, business history, special conditions related to local setting

Coping Strategies (Responses to Crises): recent crises, problems incurred, coping mechanisms,

consequences

Use of Financial Service: purpose(s) of the loan(s) and use(s) of savings

123


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 4: CASE STUDY 1: CLIENT HISTORY/LOAN USE & SAVINGS STRATEGIES/

Checklist of Questions/page 2

Why did you use your loan and/or savings in these ways? This question should give

a clearer picture of constraints to investing in the business and to meeting other consumptive expenses.

Who decided to use the loan or savings in this way? This question is important for

microfinance programmes that serve women and seek to help them gain control over financial resources.

What would you have done if you had not taken a loan? (This question explores coping

mechanisms that clients apply to meet business or household financial needs and seeks to understand

whether the client has any other financial resources.)

Interaction with other FIs: name and type of FI(s), financial products used

Satisfaction/dissatisfaction with programme: institutional structure, financial products,

loan repayment terms, savings facilities

Other remarks and comments:

124


IMPLEMENTATION STEPS

TOOL 4: CASE STUDY 1: CLIENT HISTORY/LOAN USE & SAVINGS STRATEGIES/Diary

Matrix/Aggregated Diary Matrix

Results obtained in case study interviews are summarized and aggregated per sub-group of

clients (gender aspects and years of participation) then compared with the results of other

sub-groups in order to develop conclusions for management purposes.

INTENT OF

QUESTIONS

Socio-demographic

information:

• Type of client

• Household structures

• Social background

• Coping strategies (how the

household responds to

crisis)

Professional history:

• Professional background

• Business history

• Business vision

Use of financial services:

• Type of financial services

• Type of financial

institutions

• Business strategies

• Reasons for the use of

complementary financial

sources

Satisfaction/dissatisfaction

with the programme’s

services:

• Financial services

• Human capacity building

• Institutional capacity

building

Supplementary remarks and

comments:

Findings (per subgroup)

Comparison with

other sub-groups

Conclusions

125


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 5 CASE STUDY 2: CLIENT EMPOWERMENT

TYPE Qualitative in-depth interview

DESCRIPTION The client empowerment tool tests hypotheses of increased control

of resources and increased self-esteem on the part of women

clients. There are three implementation methods (compare to A and

B tool 4):

A: Client self-portrait: Ask the client how they have changed over the

past year using probing questions.

B: Conduct an open-ended interview, asking the client categorical

questions.

C: Hold a focus group with women clients, asking categorical questions.

PURPOSE Gather information and identify the ways that the client has been

empowered personally, in her family, in her community, and in her

business:

• Verify what changes have occurred by referring to the impact

variables selected from the survey questionnaire.

• Examine how these changes have occurred by exploring the

interaction between different factors and the pathways of

change.

• Explore why these changes have occurred by uncovering

causal linkages and testing counterfactual hypotheses.

IM TEAM Project staff, preferably female officers

SURVEY UNIT

126

Women clients who have participated for more than one year in the

programme; selection is based on a sample of clients who

completed tool 1; can be used for individuals as well as groups that

are randomly selected

TIME REQUIRED 60 (individuals) to 120 (groups) minutes

DATA

ANALYSIS

(Portrait) Summary Sheet (Past – Present)

SOURCE Nancy Horn of OPPORTUNITY INTERNATIONAL designed the

initial version of this tool. Members of the AIMS/SEEP tools team

subsequently revised it. 54

54 The SEEP Network (2000); compare also Mayouox (1997).


IMPLEMENTATION STEPS

TOOL 5: Case Study 2: Client Empowerment/Menu of Categorical Questions

This menu is a list of suggested questions to ask during the interview. You should adapt it according to

your organization’s needs. To identify a pattern of change, ask the selected questions twice: once for

the past and again for the present. The questions below address past client actions. Be sure to change

the verb tense for the present!

Menu of Categorical Questions

Individual:

• What kind of person did you used to be?

• If I had been with you before you joined the programme, what would I have seen you doing?

• How did you feel about yourself before joining the programme?

• What kind of dreams/goals did you have for your life?

• What types of actions did you take/NOT take to fulfil these dreams/goals?

• When you left your home, how did you look at the world? (as a set of problems, a set of opportunities,

a set of challenges that you could overcome?)

• What did people say about you?

Business:

• Did you have a business before joining the programme? If so, please describe.

• What kind of business-person did you used to be?

• How did you manage your business?

• How did you feel about yourself as a business operator/manager? How was your business doing?

Why?

• What kind of dreams/goals did you have for your business?

• What kind of decisions did you make/NOT make about your business? What kind of decisions did

you refer to other people? To whom?

• What kind of obstacles or constraints did you have in regards to operating your business successfully?

• What did your customers say about you?

Family/Household:

• What kind of family member were you in your household?

• If I had been with you before joining the programme, what would I have seen you doing in your

family/household?

• How did you feel about yourself as a family member in your household? How was your family/household

doing?

• What kind of dreams/goals did you have for your family/household?

• What kind of problems did you have in your family/household?

• What kind of decisions did you make/NOT make about your family/household? What types of

decisions did you refer to other members of the household?

• What did your family/household members say about you?

Community:

• What kind of person were you in your community?

• If I had been with you before joining the programme, what would I have seen you doing in your

community?

• What type of relationship did you have in your community? How was the community doing?

• Before joining the programme, what roles did you have and/or activities did you participate in with

your community?

• What kind of problems/constraints did you observe in your community?

• How did you try to resolve these problems/constraints?

127


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 5: Case Study 2: Client Empowerment/Intent of Categorical Questions

Each question from the Menu of Categorical Questions has a particular intent. This chart explains the

intent of selected questions from that menu.

INTENT OF CATEGORICAL QUESTIONS

Past tense

Questions Intent

INDIVIDUAL

If I had been with you before you joined the programme,

what would I have seen you doing?

How did you feel about yourself before joining the programme?

128

To obtain a sense of how the client led her life – if she

was joyful or sorrowful, active or passive, fully engaged

or withdrawn, opportunistic or lethargic

To understand the client’s level of self-esteem and selfconfidence

What kind of dreams/goals did you have for your life? To determine if the client had any personal dreams

about self-improvement or desires for the future

BUSINESS

Did you have a business before joining the programme?

If so, please describe.

What kind of dreams/goals did you have for your business?

What kind of decisions did you make about your business?

What kind of decisions did you refer to other people?

To whom?

How did you manage your business before joining the

programme?

To determine if the client had one or several business(es)

and the type of business(es)

Assuming the answer is positive, this question seeks to

determine if the client had a desire to grow her business(es)

in a particular way

To determine if the client had the self-confidence or

freedom to make her own decisions

To determine how entrepreneurial the client was, and

how much risk she felt confident taking

Were there any obstacles that you faced in operating To determine what strategies the client used to over-

your business? What were they? Did you overcome come hardship (any obstacle to operating her busi-

them? How?

ness(es) smoothly)

FAMILY/HOUSEHOLD

How did you feel about yourself as a family member in

your household?

What type of dreams or goals did you have for your

family or household?

What kind of decisions did you make about your family/

household? What type of decisions did you refer to others

in the household?

COMMUNITY

What type of relationships did you have in your community?

Who were the most significant people you related

to?

Before joining the programme, what roles did you have

or activities did you participate in with your community?

What kind of problems/constraints did you observe in

your community?

In what ways did you try to resolve these problems and

constraints?

To identify whether females were subordinate in the

family and determine whether a client felt respected as a

contributing member of the family

To determine whether the client had desires such as

owning a dream house or her own land, sending her

children to school or university?

To determine the type of family decisions the client

made on her own and the type she made with others.

To determine how a client related to her neighbours and

other community members and who some of the most

important people were in her interactions

To determine how active a client was in the community

and whether she participated in community events, political

parties or social groups

To determine how perceptive to community concerns a

client was and whether she identified those concerns

with her own

To determine whether a client joined a group to solve a

problem and whether she took any actions to improve

the community


IMPLEMENTATION STEPS

TOOL 5: Case Study 2: Client Empowerment/Portrait Summary

This summary matrix is simply constructed. For each time period, categorical questions are noted with

room to write client responses next to the questions, and answer any probing follow-up questions. This

example shows parts of the matrix condensed to fit on one page. An actual working matrix will occupy

several pages, with an allocation of one page per question.

COMPONENTS OF EMPOWERMENT PORTRAIT SUMMARY

Past Present

Categorical Questions

INDIVIDUAL

Q.#

Q.#

Q.#

Q.#

Q.#

Q.#

Q.#

Q.#

Q.#

Q.#

BUSINESS

Q.#

Q.#

FAMILY/HOUSEHOLD

Q.#

Q.#

COMMUNITY

Q.#

Q.#

129


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 6 ROUNDTABLE: CLIENT SATISFACTION

TYPE Client observations

Qualitative, semi-structured interviews

DESCRIPTION Checklist of guiding questions about client satisfaction: how clients

value the programme in terms of appropriateness (satisfaction of their

needs), and if/how it serves to build up human and institutional capital

Part 1: A checklist of guiding questions for programme staff based on

results from programme reports on performance indicators and on client

observations by field staff; the questions should be answered before

entering into discussion with client focus groups

Part 2: Checklist of questions for in-depth interviews with client groups

PURPOSE Stakeholders (key persons from different client groups, programme

staff and others) exchange opinions and experiences about the types

and directions of impact and develop constructive conclusions and

recommendations to improve programme intervention (services provided,

types of products, institutional structure). A plan of action should

then be developed on the basis of the information obtained.

IM TEAM Moderator, facilitator, note taker

Programme staff, fieldworker

TIME REQUIRED 30 to 60 minutes for each part

DATA ANALYSIS Interactive roundtable summary matrix

Part 3: Exchange of information between participants (programme

staff and client groups) using the principles of a roundtable

SOURCE Schaefer (2000)

130


IMPLEMENTATION STEPS

TOOL 6: Roundtable: Client Satisfaction (Part 1):

Checklist of Questions for Programme Staff/page 1

Project staff should answer each question before starting focus group interviews with representatives

of client groups. Results from programme records as well as observations from

field project staff should serve as the information base.

CLIENT SATISFACTION - PART 1

Checklist of questions (programme staff)

Who are the clients? (targeting screens, nature of products and services, participation requirements, evolution

continuum)

How are clients treated? (provision of reliable services at reasonable prices, mutual accountability and trust)

What are the incentives and sanction mechanisms of membership? (contractual relationships,

upgrading of clients)

What are the selection criteria/mechanisms to become a member of a group? (selfscreening,

risk of exclusion of people)

What are the characteristics of the client groups? (set of attributes: socio-economic aspects,

gender, size, etc.)

131


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 6: Roundtable: Client Satisfaction (Part 1):

Checklist of Questions for Programme Staff/page 2

What are the characteristics of the capacity-building process? (rules, collective socialization)

What administrative procedures are followed? (administrative procedures, bookkeeping, internal

and external control)

How do clients demonstrate commitment to the programme? (performance indicators from

programme records, increased sense of ownership, interest in contract enforcement, system accountability)

What are the characteristics of drop-outs (clients who left the programme)? Do we

know anything about why they left the MFI? (group problems, dissatisfaction, difficulties running their

business(es), difficulties repaying their loan, etc.)

132


IMPLEMENTATION STEPS

TOOL 6: Roundtable: Client Satisfaction (Part 2):

Checklist of Questions for Client Focus Groups/page 1

Project staff discusses aspects of building human resources and organizational capacity with

representatives of client groups.

CLIENT SATISFACTION - PART 2

Checklist of questions (client focus groups)

What are the characteristics of your groups? (local leader respected by the group, social hierarchies,

literacy level of members, gender aspects, self-selection of members, group size, group enforcement

including incentives and sanctions

What are the procedures to become a member of the group? (ask them to describe the conditions

and the selection process)

If people have difficulty becoming a member, please explain why? Why are some people

excluded? (severe poverty, no cash-based income-generating activities, etc.)

Do you have any other requirements for group/programme membership? (regular meetings,

peer guarantees, group responsibility to collect loan repayments)

Are the established rules and administrative procedures clear to you? (rules in terms of

agendas, roles and responsibilities; regular member attendance at meetings, administrative and bookkeeping systems,

etc.)

133


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 6: Roundtable: Client Satisfaction (Part 2):

Checklist of Questions for Client Focus Groups/page 2

If you have problems understanding or abiding by the rules, please explain why. (lack of

simplicity and transparency, systems prone to misuse and corruption)

Have you thought of solutions to resolve these problems? If yes, please describe

them.

Do the financial products and services offered by the programme meet your needs? If

they are not useful to you, please explain why. (nature of products and services, provision of reliable

services at reasonable prices, linkage of future, larger loans to timely repayment - upgrading of clients)

If the financial products and services do not meet your needs, please suggest how to

improve them.

Do you know why other people left the programme? (lack of resources, internal group problems,

inappropriate programme conditions, individual problems, etc.)

134


IMPLEMENTATION STEPS

TOOL 6: Roundtable: Client Satisfaction – Roundtable Summary Matrix/page 1

Roundtable Summary Matrix

Results from Part 1 and Part 2 are summarized and discussed by participants in order to

develop incentives to ameliorate programme intervention and increase impact.

Intent of Questions Findings

Programme

Staff

Outreach: Who are the clients?

• targeting screens (max. land holdings,

employment status, geographic

focus)

• nature of products and services (very

small amounts, rapid turnover loans)

• participation requirements (regular

meetings, peer guarantees)

• evolution continuum: % of clientele by

gender

• beneficiaries: Recipients of free

services (passive)

• clients: Participants in Contractual

relationships with mutual obligations

investors/Shareholders: Owners of

assets in a financial institution

• managers: Strategic and operational

decision-makers

Market orientation: How are clients

treated?

• provision of reliable services at reasonable

prices, which creates real

value for clients; mutual accountability

and trust attracts and maintains a

large client base

Client responsibility:

• establishment of clear contractual

relationships with incentives and

sanctions for specific client behaviour

(peer guarantees, group responsibility

to collect loan repayments)

• linkage of future, larger loans to

prompt repayment: upgrading of

clients

Self-selection:

• self-screening feature to share collective

responsibility

• group dynamics are an impetus for

successful group lending, bears the

risk of exclusion of people, either socially

or economically (severe poverty,

no economic activities)

Findings

Clients

Conclusions

135


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

TOOL 6: Roundtable Summary Matrix/page 2

ROUNDTABLE SUMMARY MATRIX (CONT.)

136

Intent of Questions Findings

Programme

Staff

Group Formation and Capacity

Building:

• set of attributes: support/participation

by the powerful

(local leaders respected by the

group)

• at least a few literate members

• single-gender groups

• self-selection of members (socioeconomically

similar)

• group enforcement with incentives

and sanctions

• group size: assures mutual accountability

and cost efficiency

Discipline and routine: process of

capacity building

• rules in terms of understood

agendas, roles, and responsibilities

• collective socialization (members

attend meetings regularly, client

exits, etc.)

Simplicity and transparency:

• administrative procedures and

bookkeeping system

• transactions conducted in public

to minimize corruption and

suspicion

Demonstration and commitment to

the programme:

• guarantees against outstanding

loans

• financial assets that can be mobilized

in emergencies

• direct or indirect financial assets

for the MFI

increased sense of ownership and

interest in contract enforcement

by group members

• system accountability to the market

and group members

Findings

Clients

Conclusions


OBJECTIVES

STEP 5: DATA ANALYSIS

IMPLEMENTATION STEPS

Present the main principles of data analysis and assess the

quantitative and qualitative information that guide you in

developing your own assessment instruments

Impact assessment should tell us:

Whether identified changes can be linked to participation in

the programme;

How length of programme participation is associated with

impact;

How loan size and terms are associated with impact;

Triangulation of information; and,

What are the drivers for dysfunction.

ACTIVITIES Quantitative and qualitative data are prepared, transformed,

checked and entered into the computer.

Quantitative data are assessed on the basis of the most

common types of analysis.

Examples for documentation and presentation of the

quantifiable results are demonstrated.

Qualitative data are aggregated and analysed on the basis of

the most common types of analysis.

Examples for documentation and presentation of qualitative

information are demonstrated.

RESPONSIBILITY Evaluation Team, experienced in (statistical) data analysis

TIMING

Clear documentation of information and straight aggregation

of data brings you half way to plausibly explaining

patterns of change with a high level of confidence in the

results.

After data collection is finished

METHODS Computer-based analysis: descriptive analysis with EXCEL;

inferential analysis with SPSS

Qualitative Summary Matrixes

137


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

AGGREGATION OF DATA SETS

Based on the documentation techniques presented in step 4, this section presents

the main principles of data analysis and the assessment of quantitative and qualitative

information. We will then guide you to develop your own assessment instru-

ments. 55

Data Logging

Data preparation involves checking data for accuracy, transforming data, entering

data into the computer, and developing and documenting a database structure that

integrates the various measurements.

In all but the simplest studies, you need to set up a procedure for logging and keeping

track of the information until you are ready to do a comprehensive data analysis.

There are different ways to keep track of incoming data. In most cases, you will want

to set up a database that enables you to assess what data you already have and

what is outstanding at any given time. You can accomplish this with a standard computerized

database programme, though this requires familiarity with the programme.

You can also accomplish the task by using a standard statistical programme (e.g.,

EXCEL or SPSS) and running simple descriptive analyses to report on data status.

Transforming the Data

For all IM and IA processes, you should generate a printed codebook that describes

the data and indicates where and how it can be accessed. A codebook is an

indispensable tool for the analysis team. Together with the database, it should provide

comprehensive documentation that would enable other researchers who may

subsequently wish to analyse the data to do so without additional information. The

codebook should include at least the following items for each variable:

138

• variable name

• variable description

• variable format (number, data, text)

instrument/method of collection

55 Most of the following sections are taken from Trochim (2000), and SEEP (2000).


• data collected

• respondent or group

variable location (in database)

notes

IMPLEMENTATION STEPS

Once the data has been entered, it is almost always necessary to transform the raw

data into variables that can be used in analysis. A wide variety of transformations is

possible. Some of the more common ones are described in the following sections.

Data Coding

Categorizing responses so that they can be analysed quantitatively requires that

they be transformed, or translated, into numeric codes. Coding is particularly important

for any survey that will be analysed statistically. A survey supervisor should be

responsible for assigning (new) codes to ensure consistent logic and sequencing

numbers. Each survey should be completely coded before data entry begins. For all

questions that were answered ”other,” a code-book should be prepared.

Numeric Codes

1. Date code

Example: Date joined programme:______________ (day/month/year)

To be entered as 10/09/1999

2. Interval code (counting numbers)

Example: Size of your family: ❐❐ (specify number of members)

With interval codes, an average or mean can be calculated.

3. Categorical number

Example: Marital status: Are you...?

1. Married/ free union 3. Widowed

2. Separated/ divorced 4. Single

Each code number corresponds to a specific and discrete response, rather than to

an amount. Frequencies and prevalence ( in %) can be calculated.

Example: Over the last twelve months, your income has ..?

1 2 3 0

1. Decreased 2. stayed the 3. increased 0. don’t

same know

139


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Responses 1-3 are organized in a scale and can be presented either as frequencies

or percentages. A comparison of the mean values for different sample groups (exclusion

of all 0 responses) is possible.

Multi-response Questions

This type of question allows a respondent to choose several different factors that

they consider as relevant.

Open-code Responses

140

Example: When you apply for a loan, what factors do you consider

important?

1. the interest rate must be low ❐

2. the distance to the point of service should not ❐

be longer than a two hour walk

3. the repayment schedule should be flexible ❐

4. there is a possibility to upgrade ❐

5. access to complementary services ❐

6. other: specify____________________________ ❐

7. don’t know ❐

The sixth answer in the example above is open for responses that are not reflected

in the pre-coded list. The answer should be noted and the supervisor or the survey

should review the ”other” responses later and then define discrete and distinct code

numbers that will be subsequently added to the code book.

Another example of an open response follows:

Example: What are your suggestions to ameliorate the financial products

and services so that they meet your needs?

________________________________________________

________________________________________________

Responses that have a similar meaning should be grouped together under a common

numeric code in order to simplify and clarify the main answers.

Pre-specified Codes

Pre-specified codes refer to answers that have the same codes throughout the

survey questionnaire. These codes are fixed before interviews begin to simplify data

entry.

Example: Yes = 1 and No = 2 Don’t know = 0


Open-code Responses

IMPLEMENTATION STEPS

The sixth answer in the example above is open for responses that are not reflected

in the pre-coded list. The answer should be noted and the supervisor or the survey

should review the ”other” responses later and then define discrete and distinct code

numbers that will be subsequently added to the code book.

Another example of an open response follows:

Example: What are your suggestions to ameliorate the financial products

and services so that they meet your needs?

________________________________________________

________________________________________________

Responses that have a similar meaning should be grouped together under a common

numeric code in order to simplify and clarify the main answers.

Pre-specified Codes

Pre-specified codes refer to answers that have the same codes throughout the

survey questionnaire. These codes are fixed before interviews begin to simplify data

entry.

Example: Yes = 1 and No = 2 Don’t know = 0

Coding Enterprise Returns Information:

A rather challenging task is to code the detailed information an interviewer collects

on enterprise costs, revenue and profit, or on household expenses. Such

information can be collected per week, per two-weeks, per month, or per season. It

is necessary to calculate and code the data according to a specific time period.

Normally the time period corresponds to the frequency of the impact survey. It is

recommended to calculate such information monthly, taking into account that

amounts are subject to fluctuation (high – low seasons, sickness of clients, crises,

etc.) and to gather such information for the length of the survey period, normally 12

months.

Checking Data for Accuracy

As soon as data is received, you should screen it for accuracy. In some circumstances,

doing this right away will allow you to go back to the sample to clarify any

141


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

problems or errors. There are several questions you should ask as part of this initial

data screening:

142

• Are the responses legible/readable?

• Are all important questions answered?

• Are the responses complete?

• Is all relevant contextual information included (e.g., date, time, place, researcher)?

Missing Values

Many analysis programmes automatically treat blank values as missing. In others,

you need to designate specific values to represent missing values. For instance, you

might use a value of -99 to indicate that an item is missing. You need to check the

specific programme you are using to determine how to handle missing values.

Entering Data in the Computer

There is a wide variety of ways to enter data into a computer for analysis. The easiest

is probably to type the data in directly. In order to assure a high level of data

accuracy, you should set up an accuracy check. For example, you might spot check

records on a random basis. Once the data have been entered, you can use various

programmes to summarize the data that check whether all data are within acceptable

limits and boundaries. Such summaries enable you to easily spot a discrepancy,

such as when a person’s age is entered as 601. We recommend that you use

Windows EXCEL, because it is already installed on most computers and is easy to

use. Data entry accuracy is critical for data analysis. One data entry operator should

be responsible for the whole process.

Data Analysis and Presentation of the Results: Quantitative Data

Most Common Types of Analysis

By the time you get to data analysis, most of the difficult work has been done. It is

much more difficult to define core issues, develop and implement a sampling plan;

conceptualise, operationalise and test your measurements and develop a design

structure. If you have done the other work well, data analysis is usually a fairly

straightforward affair. Various methods of data analysis are described below.


Descriptive Analysis

IMPLEMENTATION STEPS

For descriptive analysis, statistics are used to describe basic data features. Descriptive

statistics provide summaries about the sample and measurements. Together

with simple graphic analysis, they form the basis of virtually every quantitative data

analysis. Descriptive statistics simply describe what the data shows. They are used

to present quantitative descriptions in a manageable form. Descriptive statistics provide

a powerful summary that allows comparisons to be made across people or

other units. The information is used to describe programme experiences, however,

rather than to evaluate impact and assign causality. Still, interesting trends can be

seen in descriptive results. To provide an example of how to document and illustrate

data, we have summarised fictive results of IM and IA for a microfinance programme

in the following tables.

Example 1: Programme Portfolio History for One-Year and Two-Year Clients

(in €); Village Bank x in an African Country

Variables 1-Year Clients

n = 54

2-Year Clients

n = 45

Average Number of Months in the Programme 16 Months 24 Months

Average Number of Programme Loans 2,6 5,1

Average Length of Programme Loans (Types: 4month

loans, 6-month loans, 12-month loans)

6,2 Months 4,7 Months

Average Amount of First Loan 31 € 23 €

Accumulated Average Amount of Loans 34 € 31 €

Average Amount of Current Loan: 35 € 38 €

Large Village (Sub-urban) 51 € 59 €

Small Village (Rural) 28 € 29 €

Average Amount of Current Savings Deposit 12 € 21 €

Number of Clients Reporting Difficulties in

Repaying their Last Programme Loan

5 6

Example 2: Client Individual and Household Demographic Information; Village

Bank x in an African Country

Variables 1-Year

Clients

n = 54

2-Year

Clients

n = 45

Percent married - monogamous 45 42 56

Percent married - polygamous 41 46 28

Percent other civil status 14 12 16

Mean Age in years 35 39 35

New Clients

n = 25

Mean Years in School 3,6 3,5 4,1

143


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Percent who never attended school 81,3% 82,5% 78,4%

Mean number in household 9,8 10,1 9,7

Mean number of wives 1,6 1,9 1,5

Mean Dependency Ratio 1,7 1,9 1,9

Percent female-headed households 2,3% 2,2% 2,5%

Percent of households with income

from off-farm employment/activities

Univariate Analysis

144

84% 81% 89%

Univariate analysis involves the examination across cases of one variable at a time.

In general, we look at three major characteristics for each variable. In most situations,

we would describe all three of these characteristics for each of the variables in

our IA.

• The central tendency: mean, median, mode

• The distribution: frequency

• The dispersion: range, standard deviation, variance

Central Tendency 56

The central tendency of a distribution is an estimate of the centre of a distribution of

values. There are three major types of estimates of central tendency described below.

Mean

The Mean or average is probably the most commonly used method of describing

central tendency. To compute the mean, you add together all values and then divide

by the number of values. For example, consider the test score values:

15, 20, 21, 20, 36, 15, 25, 15

The sum of these 8 values is 167. The mean is therefore 167/8 = 20.875.

Median

The Median is the score found at the exact middle of the set of values. If there are a

number of extreme cases in a data set, median results could be preferable because

they provide a truer comparison of each sample group’s results (for instance enterprise

return, monthly enterprise sales and profits). One way to compute the median

56 Taken from Trochim (2000)


IMPLEMENTATION STEPS

is to list all scores in numerical order, and then locate the score in the middle of the

sample. If we order the 8 scores shown above, we would get:

15,15,15,20,20,21,25,36

There are 8 scores and score #4 and #5 represent the halfway point. Since both of

these scores are 20, the median is 20. If the two middle scores had different values,

you would have to interpolate to determine the median. Since the median value

is the mid-point with 50% of all cases falling below and 50% above, it is not as prone

to extreme values as the mean value.

Mode

The Mode is the most frequently-occurring value in a set of values. In our example,

the value 15 occurs three times and is the mode. In some distributions, there is

more than one modal value. Bimodal distributions have two values that occur most

frequently.

Notice that for the same set of 8 scores above we got three different values --

20.875, 20, and 15. For a truly normal distribution, the mean, median and mode

are equal to one another.

Distribution

The distribution is a summary of the frequency of individual values or ranges of values

for a variable. The simplest distribution would list every value of a variable and

the number of persons who had each value. For instance, a typical way to describe

the distribution of MFI members by year in a programme would be to list the number

or percent of clients each year. Gender could be described by listing the number or

percent of males and females. In these cases, the variables had few enough values

that we could list each one and summarize how many sample cases had the value.

But what do we do for a variable like income or aggregated value of credit? With

such variables, there can be a large number of possible values, with relatively few

people having each one. In such cases, the raw scores should be grouped into

categories according to ranges of values. For instance, we could group income or

credit into four or five ranges of values.

One of the most common ways to describe a single variable is with a frequency

distribution. Depending on the particular variable, all data values could be repre-

145


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

sented, or they could be grouped into categories first (e.g., for age, years of

membership, gender distribution, income categories, etc.). It is usually not sensible

to determine the frequency for each value. Rather, values are normally grouped into

ranges and then frequencies are determined. Distributions may also be displayed

using percentages. For example, you could use percentages to describe people in

different income levels or age ranges.

Categorical Variables

Many questions in the impact survey are categorical variables representing discrete

choices rather than amounts.

For illustration, let us look at the impact survey question on reported changes in

income during the past 12 months (1 = increased, 2 = stayed the same, 3 = decreased)

with one-year clients n = 54, two-year clients n = 45, and new clients - the

comparison group - n = 25. The table below presents the aggregated data and -

frequency distributions.

Example 3: During the last 12 months, how has your income changed?

Change in Personal Income over the last 12 Months

N = 54 N = 45 N = 25

one-year clients two-year clients new clients

Increased 55% 75% 58%

stayed the same 35% 12% 35%

Decreased 10% 13% 7%

The frequency must first be calculated for each category. In EXCEL, use the function

”frequency.” If a respondent has not answered a question in the sample, we

treat the question as ”not applicable” (Code 99), which is similar to ”missing value.”

Excluding such a case results in the number of sub-samples decreasing respectively.

Furthermore; we look for differences between non-clients (using new clients

as the comparison group) and two client groups: one-year clients and two-year clients

(0 = non-clients, 1 = one-year clients, 2 = two-year clients). This requires a twostep

process. First, we create a cross-table presenting responses by survey sample

group in EXCEL. The cross-table reports the number and percentage for each category

and sample group. For categorical data, the statistical test of differences between

sample groups is normally the Chi-Square Test, which is also included in EX-

146


IMPLEMENTATION STEPS

CEL. The problem arises that because we have three sample groups, we are not

able to determine which two groups were significantly different, even if a significant

difference was calculated. For this reason, we exclude one group (let’s say the oneyear

clients) and run the test on the basis of a cross-table, which thus indicates the

results for two sample groups. However, although a difference in percent is indicated

in the table, the statistical test might indicate that while more two-year clients

reported an increase in income, the difference in frequency might not be statistically

significant.

Interval Variables

Some of the variables in the impact survey are interval variables, such as number of

years operating a business. For illustration, please see the example below.

Example 4: For how many years have you operated your business?

Years in

Business

One-year % Two-year % New % Cumulative

clients

clients

clients

Frequency Frequency Frequency Frequency %

0 0 0,00 0 0,00 5 20,00 5 4,03

1 4 7,41 0 0,00 6 24,00 10 8,06

2 7 12,96 7 15,56 7 28,00 21 16,94

3 7 12,96 8 17,78 4 16,00 19 15,32

4 10 18,52 13 28,89 1 4,00 24 19,35

5 12 22,22 8 17,78 1 4,00 21 16,94

6 14 25,93 9 20,00 1 4,00 24 19,35

N = 54 100,00 N = 45 100,00 N = 25 100,00 124,00 100,00

To present the data, we first calculated the mean value of years in business and

generated the frequency table above. From these results, we can see that for the 54

one-year clients, the average or mean number of years in business is 4.13; for the

45 two-year clients, it is 4.10 and for the 25 new clients, it is 1.88. We see that the

proportion of business start-ups and one-year businesses within the group of new

clients exceeds the proportion of start-ups and one-year businesses within the sample

of one-year clients.

147


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Dispersion

Dispersion refers to the spread of values around the central tendency. There are two

common measures of dispersion: the range and the standard deviation. The range

is the highest value minus the lowest value. Using the same set of values as before,

the high value is 36 and the low value is 15, so the range is 36 - 15 = 21.

The Standard Deviation is a more accurate and detailed estimate of dispersion

because an outlier can greatly exaggerate the range (as was true in this example

where the single outlier value of 36 stands apart from the rest of the values. The

standard deviation shows the relation of the set of values to their mean. Using the

same values again, we first calculate the difference between each value and the

mean (20.875). Values below the mean have negative discrepancies and values

above it have positive ones. Next, we square each discrepancy. Then, we take

these "squares" and sum them to get the Sum of Squares (SS) value (350.875).

Next, we divide this sum by the number of scores minus one. (350.875 / 7 = 50.125)

This value is known as the variance. To arrive at the standard deviation, we take

the square root of the variance (SQRT(50.125) = 7.079901129253). Every statistics

programme is capable of making these calculations. When we put the eight scores

into EXCEL, the following table was produced, which confirms the calculations done

by hand.

148

N 8

Mean 20.8750

Median 20.000

Mode 15.00

Std. Deviation 7.0799

Variance 50.1250

Range 21.00

The standard deviation allows us to reach some conclusions about specific scores in

our distribution. For instance, since the mean in our example is 20.875 and the

standard deviation is 7.0799, we can estimate that approximately 95% of the scores

fall in the range of 20.875-(2*7.0799) to 20.875+(2*7.0799) or between 6.7152 and

35.0348. This kind of information is critical to analysing the relationship between

variables, even when the variables are measured on entirely different scales.


Inferential Analysis

IMPLEMENTATION STEPS

Inferential statistics investigate questions, models and hypotheses. In many cases,

the conclusions drawn from inferential statistics have implications beyond the

immediate data. For instance, inferential statistics can be used to try to infer what a

population thinks from sample data. Inferential statistics can also be used to judge

the probability of whether an observed difference between groups is dependable or

happened by chance. Thus, inferential statistics are used to make inferences from

data about general conditions whereas descriptive statistics describe what is

happening in the data.

One of the most important analyses in programme impact evaluations involves comparing

the impact variables of programme groups with those of non-programme

groups. How we do this depends on the research design we use. Research designs

are divided into two major types: experimental and quasi-experimental. Impact assessment

uses the second type, quasi experimental design. 57 If you decide to utilize

quasi-experimental statistics in your Impact Study (tool 1), the analysis should be

carried out by somebody familiar with this kind of statistical analysis who has

appropriate software (for example SPSS).

We will concentrate on inferential statistics that are useful in impact evaluation. One

of the simplest inferential tests is used to compare the average performance of two

groups on a single measure to see if there is a difference. There are three ways to

estimate the treatment effect for a post-test only randomised experiment. The first is

an independent t-test. The second is a one-way Analysis of Variance (ANOVA) between

two independent groups. The third is regression analysis, which regresses

the post-test values.

57 Experimental Analysis. The simple two-group post-test randomised experiment is usually

analysed with the t-test or one-way ANOVA (p-test). Factorial experimental designs are usually

analysed with the Analysis of Variance (ANOVA) Model. Randomised Block Designs use a special

form of the ANOVA blocking model that uses dummy-coded variables to represent blocks. The

Analysis of Covariance Experimental Design uses the Analysis of Covariance statistical model.

Quasi-Experimental Analysis. Quasi-experimental designs differ from experimental ones in that

they don't use random assignment to assign units (e.g., people) to programme groups, so that the

problem of selection bias is avoided. The lack of random assignment in these designs tends to

complicate their analysis considerably. Internal validity is not assured due to selectivity bias. For

example, to analyse Non-equivalent Group Design (NEGD) we must adjust the pre-test scores for

measurement error in what is often called a Reliability-Corrected Analysis of Covariance model.

Compare Trochim (2000).

149


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Whenever you wish to compare the average performance between two groups, you

should consider using the t-test. 58 The t-test assesses whether the means of two

groups are statistically different. This analysis is appropriate whenever you want to

compare the means of two groups, and is especially appropriate for normally-distributed

data.

The formula for the t-test is a ratio. The top part of the ratio is the difference between

the two means or averages. The bottom part is the measure of the variability or

dispersion of the values.

Effect =

150

Difference between group means

Variability of groups

= t Value

We can now examine how to estimate differences between groups, often called the

effect size. In this context, we would calculate what is known as the standard error

of the difference between the means. The standard error incorporates information

about the standard deviation (variability) of the two groups. The ratio that we compute

is called the t-value and describes the difference between the groups relative to

the variability of the values of the groups.

It is useful to understand what is meant by the term "difference" when we ask "Is

there a difference between the groups?" In the figure below, the mean values for

each group are indicated with dashed lines. The difference between the means is

the horizontal difference between where the means of the programme group and the

means of the reference group hit the horizontal axis.

58

Although t-tests are normally used for randomised sample groups, they can also be used for quasiexperimental

designs if the data are normally distributed.


IMPLEMENTATION STEPS

Example 5: Is there a meaningful difference between changes in income over

the last 12 months for non-clients versus clients?

Green curve: Programme group mean

Blue curve: Reference group mean

Source: Trochim (2000): t-test.

There are three possible outcomes: medium, high and low variability. The differences

between the means in all three situations is exactly the same. The only

difference is the variability or spread of the scores around the means. In the case of

low variability, the groups differ the most because the bell-shaped curves for the two

groups overlap the least. In the high variability case, quite a few values from the

programme group and the reference group overlap. This is important because you

need to determine whether the two groups are different. It is not sufficient to subtract

one mean from the other -- you must take the variability around the means into account!

A small difference between means will be hard to detect if variability is high. A

large difference between means will be easy to detect if variability is low.

Another way to statistically test whether the results of client sample groups are

significantly different is the ANOVA test. Like the t-test, the ANOVA test can only be

used for data that is normally distributed. Homogeneity of variance can be tested

with the Bartlett’s test (p-value less than 0.05).

151


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Documentation and Illustration

Good and clear documentation and presentation of results is very important for

transparency. In the following section, only a few examples of charts are illustrated.

However, the same approaches could be applied to other questions included in the

impact survey.

For example, frequency distributions can be depicted in two ways, as tables or

graphs. Example 6 on reported change in personal income over the last year is often

referred to as a histogram or bar chart and depicts both a table and a graph in

a combined presentation.

Example 6: Reported change in personal income over the last 12 months

152

Distribution

100%

80%

60%

40%

20%

0%

Reported Change in Personal Income

Over the Last 12 Months

increased

stayed the

same

one-year client 55% 35% 10%

two-year client 75% 12% 13%

new client 58% 35% 7%

Income Change

decreased

one-year client

two-year client

new client

Another example of illustration is presented in example 7 on distress periods at the

household level. First, the aggregated results are listed in a table. Because the results

refer to programme results, a statistical significance test can be conducted to

assess whether the responses of the sample groups are significantly different and

affect programme impact.


IMPLEMENTATION STEPS

Example 7: Distress periods at the household level over the last 12 Months

Type of Distress: N = 54 N = 45 N = 25

one-year clients two-year clients new clients

Percent experiencing food insecurity

14 9 21

Length of acute food insecurity (in

months)

0.82 0.40 1.75

Length of business disruption (in

months)

0.95 0.50 3.25

The tabulated results of example 7 on distress periods experienced by three groups

of clients can be illustrated by different types of charts, as demonstrated below. Your

selection of which chart to use depends on what type of information you want to

depict.

The percent of clients experiencing food insecurity can be illustrated by a pie chart.

Percent experiencing food insecurity

new clients

N = 25

48%

one-year

clients

N = 54

32%

two-year

clients

N = 45

20%

The length of acute food insecurity experienced by households can be depicted by a

web chart that indicates the aggregate length of food insecurity in months.

Length of acute food insecurity (in months)

N = 25 new clients

N = 54 one-year clients

2

1

0

N = 45 two-year clients

PIE CHART

WEB CHART

153


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

A bar chart can be used to illustrate the length of business disruption in months.

154

Length of business disruption (in months)

3,5

3

2,5

2

1,5

1

0,5

0

0,95

one-year

clients

two-year

clients

0,5

3,25

new clients

N = 54 N = 45 N = 25

For further examples of how data can be tabulated and presented, please see examples

one through four in this chapter.

Data Analysis and Presentation of the Results: Qualitative Data

Most Common Types of Analysis

For each tool presented in step 4, we also presented corresponding information

summary matrixes. In the following section, we describe how to assess the -

information obtained using various tools and steps.

TOOL 5: Case Study 2: Client Empowerment 59

For both interview methods described, we would use the same methods to analyse

and present the data.

Step 1: Write up your field notes; within each section, develop paragraphs

for each of the categorical questions asked.

Step 2: Summarize key information on a client empowerment summary

matrix by extracting key points for each question and time period for

each client and then entering the information on a matrix to explore

patterns of change.

59 Compare SEEP (2000), Chapters 8-14.

BAR CHART


IMPLEMENTATION STEPS

Example 1: Client Empowerment Summary Matrix (retrospective)

Question Past Present

Feelings about yourself, business,

family/household and community

Dreams and goals about yourself,

business, family/household and

community

Example 2: Client Empowerment Summary Matrix (evolution)

Questions

Changed feelings about yourself,

business, family/household and

community

Realized dreams and goals about

yourself, business, family/ household

and community

Year 1 Year 2 Year 3

Step 3: Develop Tables to summarize the data from all matrixes and identify

patterns of behaviour. Develop a table that counts how many clients

have experienced similar changes.

Example 3: Decision-making shifts from ”other” to ”self” (n=15)

Change in: Past Present

Decisions about yourself 3 8

Decisions about your business 4 10

Decisions about your family/household 2 12

Decisions about community participation 0 5

The table above indicates that in the past, few clients felt empowered to make decisions.

Over time, however, a positive change has occurred.

Step 4: Search for meaning by aggregating data in a systematic manner,

which allows patterns of behaviour to emerge. Ask the following questions

as you review the information:

• How are the responses to one question related to those of another?

• Have changes in self-perception resulted in changes in behaviour?

• Have changes in behaviour resulted in greater well-being? How?

155


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

156

• What types of decisions do clients now make on their own, compared

with the past?

• In what kind of groups or community activities does the client now

participate that she did not before?

Step 5: Analyse data from all tables and matrixes then write an analysis by

clustering answers from each respondent together by question or

time period.

The following table gives an example of how to summarize client dissatisfaction with

a programme. Because the question was designed in an open form and multiple

answers were possible, categories of answers should first be made from responses.

The descriptive information should then be summarised by similar frequencies with

or without statistical comparisons between groups.

Example 4: Client Dissatisfaction (Impact Survey: Question 77)

Up to four aspects clients disliked

about the MFI

One-year clients

n = 54

Two-year clients

n = 45

High interest rate 10 8

Size of loan 7 12

Length of loan cycle 13 14

Unsatisfactory savings facilities 6 3

Meetings too frequent, too long, etc. 9 10

Problems with group dynamic 12 7

Nothing or don’t know 55 61

Other: 6 4

Participatory Appraisal Methods

Clear documentation of results is important in participatory appraisal methods and

stores the information for future project planning. It is more difficult to systematically

aggregate and transparently illustrate qualitative data than quantitative data. Summary

sheets for participatory appraisal methods serve as operational tools for further

discussion and for elaborating a plan of action with clients. Matrixes allow

comparisons of information and data from different client subgroups.


IMPLEMENTATION STEPS

The next several pages present an example of a documentation sheet that can be

used to gather information about client groups and sub-groups. An example of a

summary matrix then follows.

157


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Documentation Sheet 60

1. Village:

2. Day/hour:

3. Note taker:

4. Facilitator:

5. Tool Applied:

6. Type of subgroup: (gender, ethnic)

7. Number/age of participants:

• a. number of men: at the beginning:

at the end:

• a. age group(s):

---------------------------------------------------------------------------------------------------

• b. number of women at the beginning:

at the end:

• b. age group(s):

8. Other important information about the participants: (social status, how

participants rise to speak, etc.):

9. Duration of session/tool:

10. Specific conditions: (harvest, holidays, period of stress, etc.)

11. Location where instrument was applied:

12. Materials utilized:

13. Comments:

60 Translated version of ”Fiche de Documentation” in Schaefer (1997), and CFSDR (1996).

158


Village: Date:

Note taker:

Facilitator:

Results:

14. Key questions and interview guide:

15. Which primary responses were given?

IMPLEMENTATION STEPS

159


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Village: Date:

Note taker:

Facilitator:

Results:

16. What supplementary information was obtained?

17. What are your conclusions? How do you assess the situation?

18. Which parts of the session were easy to conduct?

19. Which parts of the session were difficult to conduct?

20. What questions/information must be further investigated/focused on?

160


Village: Date:

Note taker:

Facilitator:

Results:

21. Copy of the results of the subgroup: (map, diagram, etc.)

IMPLEMENTATION STEPS

161


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Summary Matrix:

Example 4: Seasonal calendar on income-generating activities, financial demand

and financial sources

162

sub-group

tool

• opportunities

• main

problem(s)

• other

problems

• constraints/

difficulties

(earning

money, resourceallocation,

etc.)

• type of income-generating

activities

(season)

• type of financing/

use

• additional

information

• Women

(new clients)

• seasonal

calendar

• Women

(repeat

borrowers)

• seasonal

calendar

RESULTS

• Men (new

clients)

• seasonal

calendar

• Men (repeatborrowers)

• seasonal

calendar

Conclusions


ANNEX 1: SOURCES AND FURTHER READING

ANNEX 1 – SOURCES AND FURTHER READING

FOCUSING ON ASPECTS OF MFI IMPACT MONITORING AND ASSESSMENT

Barnes, Carolyn. 1996. Assets and the Impact of Microenterprise Finance Programmes.

Washington, D.C.: Management Systems International.

Barnes, Carolyn, Morris, Gayle, and Gaile, Gary. 1998. An Assessment of the Impact

of Microfinance Services in Uganda: Baseline Findings. Volume I and II.

AIMS Project. Washington, D.C.: Management Systems International.

Barnes, Carolyn, and Sebstad, Jennefer. 2000. Guidelines for Microfinance Impact

Assessments: Discussion Paper for the CGAP 3 Virtual Meeting (October 18-

19, 1999). AIMS Paper. Washington, D.C.: Management Systems International.

Berger, M. 1989. "Giving Women Credit: The Strengths and Limitations of Credit as

a Tool for Alleviating Poverty." In: World Development 17, pp 1017-1032.

Blank, Lorraine. 1998. Client Monitoring Systems for Microfinance Institutions. AIMS

Assistance to the Workers Bank in Jamaica. Washington, D.C.: Management

Systems International.

BMZ. 1998. Mikofinanzierung im Rahmen einer Strategie zur Finanzsystementwicklung.

07.05.98. Bonn: BMZ. Fachgesprächsprotokolle.

Chen, Martha Alter and Elizabeth Dunn. 1996. Household Economic Portfolios.

AIMS Project. Washington, D.C.: Management Systems International.

Chen, Martha Alter. 1997. A Guide for Assessing the Impact of Microenterprise Services

at the Individual Level. AIMS Project. Washington, D.C.: Management

Systems International.

Cheston, Susy and Larry Reed. 1999. Measuring Transformation: Assessing and

Improving the Impact of Microcredit. Plenary Paper presented at the Microcredit

Summit Meeting of Councils, June 24-26, Abidjan, Côte d’Ivoire.

Coffey, Elizabeth. 1998. Agricultural Finance: Getting the Policies Right. (2). Agricultural

Finance Revisited. Rome/Eschborn: FAO/ GTZ.

Cohen, Monique. 1999. Opening up the Impact Assessment Agenda. Paper Prepared

for the Latin American Microfinance Meeting. October 25th – 28th, 1999,

Lima, Peru.

Craig, D. and Porter, D. 1997. ”Framing Participation: Development Projects,

Professionals, and Organisations.” Development in Practice, Vol. 7(3), pp 229-

236.

Daniels, Lisa. 1999. ”Alternatives for Measuring Profits and Net Worth of Microenterprises.”

AIMS Project. Washington, D.C.: Management Systems International.

Diagne, A. 1998. Impact of Access to Credit on Income and Food Security in Malawi.

(UnPub). Washington D.C.: IFPRI.

Diagne, A., M. Zeller, and M. Sharma. 1999. Determinants of Household Access to

and Participation in Formal and Informal Credit Markets in Malawi and Bangladesh.

Washington D.C.: IFPRI.

163


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Dunn, Elizabeth, Nicholas Kalitzandonakes, and Corinne Valdivia. 1996. Risk and

the Impacts of Microenterprise Services. AIMS Project. Washington, D.C.,

Management Systems International.

Dunn, Elizabeth. 1996. Household, Microenterprises, and Debts. AIMS Project.

Washington, D.C.: Management Systems International.

Dunn, Elizabeth. 1997. Diversification of the Household Economic Portfolio. AIMS

Project. Washington, D.C.: Management Systems International.

Edgcomb, Elaine L. 1996. PVO/NGO Tools for Household and Business Impact

Assessment: Report of a Planning Meeting. AIMS Project. Washington, D.C.:

Management Systems International.

Edgcomb, Elaine and Laura Barton. 1998. Social Intermediation and Microfinance

Programs. A Literature Review. (Microenterprise Best Practices Project). Bethesda:

Development Alternatives, Inc.

Evans, G.E., Adams, A.M., Mohammed, R., Norris, A.H. 1999. ”Demystifying Nonparticipation

in Microcredit: A Population-Based Analysis. In: World Development,

Vol. 27(2), pp 419-430.

FAO and GTZ. 1998. Agricultural Finance Revisited: Why? (1). Agriculture Finance

Revisited. Rome/ Eschborn: FAO/GTZ.

Gaile, Garry R. and Jennifer Foster 1996. Review of Methodological Approaches to

the Study of the Impact of Microenterprise Credit Programmes. AIMS Project.

Washington, D.C.: Management Systems International.

Goldstein, Gilles (IRAM) and Issa Barro (DCEG). 1999. Etude sur le Rôle et l’Impact

des Services et Produits d’Epargne du Secteur Informel et des Institutions de

Microfinance en Afrique de l’Ouest (Bénin, Burkina Faso, Ghana, Guinée, Mali

et Togo). Paris/Dakar: PNUD/FENU:.

Greeley, M. 1997. ”Poverty and Well-Being: Problems for Poverty Reduction and the

Role of Credit.” In: ”Who needs Credit? Poverty and Finance in Bangladesh.”

Geoffrey D. Wood and Iffath Sharif (eds.). Dhaka: The University Press.

Gulli, Hege. 1998. Microfinance and Poverty. Questioning the Conventional Wisdom.

Washington, D.C.: IADB.

Hashemi, S.M., S.R. Schuler, and A.P. Riley. 1996. "Rural Credit Programmes and

Women's Empowerment in Bangladesh." In: World Development 24, pp 635-

653.

Hatch, John K. and Laura Frederick. 1998. Poverty Assessment by Microfinance

Institutions: A Review of Current Practice. Microenterprise Best Practices Project.

Bethesda: Development Alternatives, Inc.

Heidhues, Franz and Gertrud Schrieder. 1999. Rural Financial Market Development.

Discussion Paper 99/01. Stuttgart: University of Hohenheim, Institute of Agricultural

Economics and Social Sciences in the Tropics and Subtropics.

Hulme, David. 2000. ”Impact Assessment Methodologies for Microfinance: Theory,

Experience and Better Practice.” In: World Development 28 (1), pp. 79-98.

IDRC. 1997. Planning Monitoring and Evaluation of Programme Performance – A

Resource Book. Draft Version. Ottawa: IDRC.

164


ANNEX 1 – SOURCES AND FURTHER READING

Inserra, Anne. 1996. A Review of Approaches for Measurement of Microenterprise

and Household Income. AIMS Project. Washington, D.C.: Management Systems

International.

IRAM. Impact des Systèmes Financiers Décentralisés. 1997. Etude Comparée dans

Trois Pays D'Afrique de l'Ouest, Bénin, Burkina Faso, Guinée. Paris: Ministère

de la Coopération.

Johnson, Susan and Ben Rogaly. 1997. Microfinance and Poverty Reduction. Oxford:

OXFAM.

Johnson, Susan. 1998. ”Policy Arena. Microfinance North and South: Contrasting

Current Debates”. In: Journal of International Development, Vol. 10(6), pp 799-

809.

Little, Peter D. 1997. Income and Assets as Impact Indicators. AIMS Project.

Washington, D.C., Management Systems International.

Mainhart, Andrew. 1999. Management Information Systems for Microfinance. An

Evaluation Framework (Microenterprise Best Practises Project). Bethesda: Development

Alternatives, Inc.

Mayouox, Linda. 1997. ”Impact Assessment and Women's Empowerment in Microfinance

Programmes: Issues for a Participatory Action and Learning Approach”.

Background Paper for CGAP Meeting, April 1997.

MkNelly, Barbara and Lippold, Karen. 1998. Evaluation D'Impact Menée par le

Practicien: A Test au Mali. AIMS Project. Washington, D.C.: Management Systems

International.

Mosley, Paul. 1998. The Use of Control Groups in Impact Assessments for Microfinance.

Working Papers. Enterprise and Cooperative Development Department

(19). ILO: Geneve.

Muazzam Husain, A.M., (ed.). 1998. ”Poverty Alleviation and Empowerment. The

Second Impact Assessment Study of BRAC’s Rural Development Programme,

Bangladesh.” Dhaka: BRAC.

Noponen, Helzi. 1997. ”Participatory Monitoring and Evaluation - A Prototype Internal

Learning System for Livelihood and Micro-Credit Programmes.” In: Community

Development Journal 32(1), pp 30-48.

Noponen, Helzi. 1998. ”Learning Friend” Diary for Members of Self Help Savings

and Credit Groups. Developed in Partnership with: Kerala Ganghi Smarak Nidhi

and Friends of Women’s World Banking (India).

Peace, G. and David Hulme. 1994. "Microenterprise and Children - What are the

Intra-Household Impacts of Income-Generating Programmes." In: Small Enterprise

Development, Vol. 5, pp 21-29.

Pitt, Mark M. and Shahidur R. Khandker. 1996. Household and Intrahousehold Impacts

of the Grameen Bank and Similar Targeted Credit Programmes in Bangladesh.

World Bank Discussion Papers (320). Washington, D.C.: World Bank.

Putnam, R.D. 1993. The Prosperous Community: Social Capital and Public Life. In:

The American Prospect Vol. 13, pp 35-42.

Schaefer, Birgit. 2000. Wirkungsanalyse 1997 - 1999 der Associations Féminines

d'Epargne et de Crédit im PADER-NORD (Programme d'Appui au Développement

Rural de la Région Norde, Département Crédit-Epargne)/Côte d'Ivoire,

165


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Endbericht für die GTZ. Universität Hohenheim, Institut für Agrar- und Sozialökonomie

in den Tropen und Subtropen: Stuttgart. (forthcoming)

Schaefer, Birgit. 2000. Wirkungsanalyse 1997 – 1999 PMR – R.F.A.. (Promotion

des Mutuelles Rurales)/ Niger. Endbericht für die GTZ. Universität Hohenheim,

Institut für Agrar- und Sozialökonomie in den Tropen und Subtropen: Stuttgart.

(forthcoming)

Schrieder, Gertrud. 1996. The Role of Rural Finance for Food Security of the Poor.

Frankfurt/M.: Peter Lang.

Schrieder, Gertrud and Manohar Sharma. 1999. ”Impact of Finance on Poverty Reduction

and Social Capital Formation – A Review and Synthesis of Empirical

Evidence.” In: Savings and Development, Vol. 231, pp 67-93.

Sebstad, Jennefer, Catherine Neill, Carolyn Barnes, and Gregory Chen. 1995. Assessing

the Impact of Microenterprise Intervention: A Framework for Analysis.

AIMS Project. Washington, D.C., Management Systems International.

Sebstad, Jennefer and Gregory Chen. 1996. Overview of Studies on the Impact of

Microenterprise Credit. AIMS Project. Washington, D.C.: Management Systems

International.

Sebstad, Jennefer. 1998. Towards Guidelines for Lower-Costs Impact Assessment

Methodologies For Microenterprise Programmes. Discussion Paper for the

Second Virtual Meeting of the CGAP Working Group on Impact Assessment

Methodologies. Washington, D.C.: Management Systems International.

SEEP Network, The. 2000. Learning from Clients: Assessment Tools for Microfinance

Practitioners, Draft Manual, AIMS Project. Washington D.C.: Management

Systems International.

Shams Mustafa, Ishrat Ara, Dilruba Banu, Altaf Hossain, Azmal Kabir, Mohammad

Mohsin, Abu Yusuf, and Sarwar Jahan. 1996. ”Beacon of Hope. An Impact Assessment

Study of BRAC’s Rural Development Programme.”

Simanowitz, Anton, Nkuna, Ben (The Small Enterprise Foundation, South Africa);

Kasim, Sukor (Universiti Sains Malaysia), and Robert Gailey. 1999. Overcoming

the Obstacles of Identifying the Poorest Families: Using Participatory

Wealth Ranking, The CASHPOR House Index, and Other Measurements to

Identify and Encourage the Participation of the Poorest Families, Especially the

Women of those Families. Paper presented at the Microcredit Summit Meeting

of Councils in Abidjan, Côte d’Ivoire, 24-26 June 1999.

Snodgrass, Donald. 1996. The Economic, Policy, and Regulatory Environment.

AIMS Project. Washington, D.C.: Management Systems International.

Snodgrass, Donald. 1997. Assessing the Effects of Programme Characteristics and

Programme Context on the Impact of Microenterprise Services: AIMS Project. A

Guide for Practitioners. Washington, D.C.: Management Systems International.

Trape, Philippe et al. 1998. Pérennité et Institutionnalisation des Systèmes Financiers

Décentralisés. Volume I: Actes du Séminaire. Paris, 1997. Volume II: Annexes.

Paris: Ministère des Affaires Etrangères – Coopération et Francophonie.

Zaman, Hassan. 1998. ”Who Benefits and to What Extent?" Evaluating the Targeting,

Poverty Alleviation and Empowerment Objectives of BRAC’s Micro-Credit

Programme. DPhil Thesis, Sussex: Department of Economics, University of

Sussex.

166


ANNEX 1 – SOURCES AND FURTHER READING

Zaman, Hassan. 1999. Assessing the Poverty and Vulnerability Impact of Micro-

Credit in Bangladesh: A Case Study of BRAC. Workpapers. Washington D.C.:

The World Bank.

Zeller, Manfred, Gertrud Schrieder, Joachim von Braun, and Franz Heidhues. 1997.

Rural Finance for Food Security for the Poor. Implications for Research and

Policy. Food Policy Review 4. Washington D.C.: IFPRI.

Zeller, Manfred and Manohar Sharma. 1998. Rural Finance and Poverty Alleviation.

Food Policy Report. Washington, D.C.: IFPRI.

Zeller, Manfred, Gertrud Schrieder, Joachim von Braun, and Franz Heidhues. 1999.

Credit for the Rural Poor in Sub-Saharan Africa. Washington, D.C.: IFPRI.

FOCUSING ON MFI PERFORMANCE AND OUTREACH

CGAP Secretariat. 1998(1). Format for Appraisal of Microfinance Institutions.

Washington D.C.: CGAP. World Bank.

CGAP. 1998(2). How Microfinance Providers Target the Poor. A Compendium of

Strategies. New York: PACT Publications.

BMZ. 1997. Micro-Finance: The Strategic Approach of Germany’s Bilateral Development

Co-operation. Discussion Paper. Bonn: BMZ.

Evans, Anna. 1997. PEARLS: A Tool for Financial Stabilization, Monitoring, and

Evaluation. NEXUS Newsletter 38. New York: SEEP Network.

Kantor, Paula, and Christine Robinson. 1998. Financial Sustainability for Credit Programmes

– A Travel Survival Guide (Manual). Washington D.C.: The World

Bank.

Ledgerwood, Joanna. 1996. Financial Management Training for Microenterprise

Organizations: Finance Study Guide. New York: PACT Publications.

Ledgerwood, Joanna. 1999. Microfinance Handbook: An Institutional and Financial

Perspective (Sustainable Banking with the Poor Project). Washington D.C.:

IBRD/The World Bank.

Paxton, Julia, and Cecile Fruman. 1998. Outreach and Sustainability of Savings-

First vs. Credit-First Financial Institutions: A Comparative Analysis of Eight Microfinance

Institutions in Africa. (Sustainable Banking with the Poor Project).

Washington D.C.: The World Bank.

PLANETFINANCE. 1999. Rating Instrument G.I.R.A.F.E. Paris: PLANETFINANCE.

Saltzman, Sonia B., and Darcy Salinger. 1998. The ACCION CAMEL Technical

Note. (Microenterprise Best Practices Project). Bethesda: Development Alternatives,

Inc.

SEEP Network and Calmeadow. 1995. Financial Ratio Analysis of Microfinance

Institutions. New York: PACT Publications.

Waterfield, Charles, and Nick Ramsing. 1998: Management Information Systems for

Microfinance Institutions: A Handbook. Prepared for CGAP by Deloitte Touche

Tohmatsu International, in Association with MEDA Trade & Consulting and

Shorebank Advisory Services, Washington D.C.

Waterfield, Charles, and Tony Sheldon. 1997. Assessing and Selecting Loan

Tracking Software Systems. New York: Women’s World Banking.

167


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Yaron, Jacob. 1997. Assessing Development Finance Institutions. A Public Interest

Analysis. World Bank Discussion Papers 174. Washington, D.C.: The World

Bank.

Yaron, Jacob, McDonald P. Benjamin Jr., and Gerda L. Piprek. 1998. Rural Finance:

Issues, Design, and Best Practices. Environmentally and Socially Sustainable

Development Studies and Monographs Series 14. Washington D.C.: World

Bank.

GENERAL ANALYTICAL APPROACHES AND EVALUATION:

Balzer, Geert. 1999. Sektorprojekt: Qualitätssicherung in der beruflichen Aus- und

Weiterbildung. Arbeitsfeld Wirkungsmonitoring. Auftaktworkshop Orlando 15.-

16.12.1999 – Ergebnisbericht. Eschborn: GTZ. mimeo.

Carvahalho, Soniya and Howard White. 1997(1). What Have We Learned? Monitoring

Projects for the Poor. Washington D.C.: The World Bank.

Carvahalho, Soniya and Howard White. 1997(2). Combining the Quantitative and

Qualitative Approaches to Poverty Measurement and Analysis. The Practise

and the Potential. Washington D.C.: The World Bank.

CDE/GTZ/KIT. 1998. Sustainable Land Management. Guidelines for Impact Monitoring

Workbook & Toolkit. Bern: CDE.

CFSDR (Centre de Formation Supérieure pour le Développement Rurale, Berlin).

1996. Atelier de Formation Sur la MARP du 31 Juillet au 2 Août 1996, San Pédro,

Côte d’Ivoire, Documentation, mimeo.

Doss, Cheryl R. 1996. ”Testing Models of Intrahousehold Resource Allocation.” In:

World Development, Vol. 2410, pp 1597-1609.

Germann, Dorsi, Pohl, Eberhard and Burkhard Schwartz. 1996. Participatory Impact

Monitoring. Booklets 1 – 4. GATE/GTZ. Wiesbaden: Vieweg.

Gueye, Bara and Schoonmaker-Freudenberger, Karen. 1991. Introduction à la

MARP, RRA, Quelques Notes pour Appuyer une Formation Pratique, London:

IIED.

GTZ. 1996. Process Monitoring. Work Document for Project Staff. Pilot Project

NARMS. Eschborn: GTZ (Department 402).

GTZ. 1998(1). Poverty Monitoring in Rural Development Projects: the Wealth Ranking

Method and its Application in the PRODILO project. Draft Version. Eschborn:

GTZ.

GTZ. 1998(2). Project Monitoring. An Orientation for Technical Corporation Projects.

Eschborn: GTZ (Unit 04: Strategic Corporate Development).

GTZ. 1999. The Quality Management of GTZ. Eschborn: GTZ (Unit 04: Strategic

Corporate Development).

Heckman, James. 1997. ”Instrumental Variables. A Study of Implicit Behavioral Assumptions

Used in Making Program Evaluations.” In: The Journal of Human

Resources, Vol. 32(3), pp 441-462.

IIED. 1993. Bulletin de Liaison et d’Echanges d’Informations sur la MARP, Outils,

Technique et Attitudes: Le Partage de Quelques Expériences. No. 01. London:

IIED.

168


ANNEX 1 – SOURCES AND FURTHER READING

IIED/OXFAM. 1993. OXFAM Initie ses Partenaires à la MARP. Rapport de l’Atélier à

Ségou, Mali du 14 au 23 Juin 1993. London: IIED.

INNONET. 2000. Interview Guidelines. ULR: http://www.innonet.org/.

Kothari, Uma. 2000. Developing Guidelines for Assessing Achievement in the Eight

Focal Areas of Social Development Work and for Assessing Outcomes – Assessment

Procedures. Manchester: University of Manchester, Institute for Development

Policy and Management.

Kuby, Thomas. 2000. Turning Attention Towards Results. How GTZ is Building Up

its Impact Evaluation Capacity. Internal Evaluation Team. Eschborn: GTZ.

Lobb-Rabe, Alison. 1998. Premise Paper on Project Monitoring and Poverty Assessment.

Pilotprojekt Armutsbekämpfung. Eschborn: GTZ.

McArthur, Harold J. Jr. 1997. ”Participatory Monitoring and Evaluation: Passing Fad

or the Logical Next Step in Research and Development Methodologies?” In:

”Self-Assessment. Participatory Dimensions of Monitoring and Evaluation.” pp

13-24. UPWARD (ed.), Los Banos/Laguna: UPWARD.

Moffitt, R. 1991. "Programme Evaluation with Nonexperimental Data." In: Evaluation

Review Vol. 15, pp 291-314.

Mohr, Lawrence B. 1995. Impact Analysis for Programme Evaluation. Thousands

Oaks et al. Newbury Park, CA: SAGE Publications.

Neubert, Dieter. 2000. Poverty Alleviation in Complex and Dynamic Social Fields.

Discussion Paper 00/01. Stuttgart: University of Hohenheim, Institute of Agricultural

Economics and Social Sciences in the Tropics and Subtropics.

Neubert, Susanne. 1998. SWAP – ein neues System zur Wirkungsanalyse armutsorientierter

Projekte in der Entwicklungszusammenarbeit. Berlin: DIE.

Ojha, Durga P. 1998. Impact Monitoring – Approaches and Indicators. Experiences

of GTZ-Supported Multi-Sectoral Rural Development Projects in Asia.

Eschborn: GTZ (Division 45).

Patton, Michael. 1990. Qualitative Evaluation and Research Methods. 2nd Edition.

Newbury Park, CA: SAGE Publications.

Pearce, John. 1996. Measuring Social Wealth. A Study of Social Audit Practices for

Community and Co-operative Enterprises. London: New Economics Foundation.

Peters, Dai. 1997. ”Participatory Impact Assessment: Framework for Success, Recipe

for Disaster”, In: ”Self-Assessment. Participatory Dimensions of Monitoring

and Evaluation.” pp 37-56. UPWARD (ed.). Los Banos/Laguna: UPWARD.

Preuss Hans-Joachim and Volker Steigerwald. 1996. Wirkungsbeobachtung in der

GTZ – von der Projektfortschrittskontrolle zu Qualitätsmanagement. Eschborn:

GTZ (Unit 04: Strategic Corporate Development).

Proehl. 1996. Anwendung systemanalytischer Methoden in der Selbsevaluierung

komplexer Entwicklungsprojekte. Planberatung/Regionalentwicklung in Mayo

Kebbi, Tschad. Eschborn: GTZ (Abteilung 403).

Ravallion, Martin. 1994. Poverty Comparison. Chur et al.: Howard Academic Publisher.

Ravallion, Martin. 1999. The Mystery of Vanishing Benefits: Ms Speedy Analyst's

Introduction to Evaluation. Workpapers. Washington, D.C: The World Bank.

169


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Reardon, Thomas. 1997. ”Using Evidence of Household Income Diversification to

Inform Study of the Rural Nonfarm Labor Market in Africa.” In: World Development,

Vol. 25(5), pp 735-747.

Saint Paul Foundation, The. 1993. How to Evaluate Foundation Programmes. St.

Paul, MN: The Saint Paul Foundation.

Saith, Ruhi. 1998. Gender Sensivity of Well-Being Indicators. Discussion Paper 95.

Geneva: UNRISD.

Schaefer, Birgit. 1997. Enquête de base/Analyse des effets. Atelier de Formation

sur la MARP du 11 au 14 Août 1997 à Korhogo. Programme d’Appui au Développement

dans la Région Norde (PADER-NORD), Département Crédit –

Epargne, GTZ – ANADER, Korhogo – Côte d’Ivoire, Documentation, mimeo.

Schönhuth, Michael and Kievelitz, Uwe. 1994. Les Méthodes Participatives de

Recherche et de Planification Dans la Coopération au Développement: Diagnostic

Rapide en Milieu Rural. Diagnostic participatif. Eschborn: GTZ.

Steigerwald, Volker. 1999. Recent Changes in the BMZ/GTZ Evaluation System and

Approaches to Impact Monitoring. Presentation and Open Forum. PIMU, Colombo,

Sri Lanka. 15 December 1999. Team Internal Evaluation, Strategic

Corporate Development Unit. GTZ: Eschborn.

Trochim, William M. 1999. The Research Methods Knowledge Base, 2nd Edition.

Ithaca, New York: Cornell Custom Publishing.

Trochim, William M. 2000. The Research Methods Knowledge Base, 2nd Edition.

Internet WWW page, at URL: .

Current Version as of 02. August 2000.

UNDP. 1997. Result-oriented Monitoring and Evaluation. A Handbook for Programme

Managers. New York: OESP (ULR: http://www.undp.org/eo/maetoc.htm).

UNITED WAY. 1999. ULR: http://national.unitedway.org/outcomes/

UNSO/PNGT/IIED 1993: L’utilisation de la MARP Dans le Cadre de la Gestion des

Terroirs, Rapport de l’Atelier de Formation sur la MARP, Koudougou du 30/11

au 09/12/1993, Burkina Faso, London: IIED.

Vahlhaus, Martina. 1999. Orientierungsrahmen für das Wirkungsmonitoring in Projekten

der Wirtschafts- und Beschäftigungsförderung. Eschborn: GTZ.

Vahlhaus, M. and Kuby T., Guidelines for Impact Monitoring in Economic and Employment

Promotion Projects with Special Reference to Poverty Reduction Impacts

(Part I), Eschborn: GTZ 2001

Vahlhaus, M., Guidelines for Impact Monitoring in Economic and Employment Promotion

Projects with Special Reference to Poverty Reduction Impacts (Part II),

Eschborn: GTZ, 2001

World Bank. 1996. Participation Sourcebook. Washington D.C.: IBRD.

170


ANNEX 2 – RAPID APPRAISAL METHODS: EXAMPLES

ANNEX 2: RAPID APPRAISAL METHODS: EXAMPLES

Rapid appraisal methods are quick, low cost ways to gather data, especially questions

about contextual factors and performance, to support the information needs of

managers.

Advantages: Limitations:

• In-depth understanding of complex socioeconomic

processes or systems (operationalisation

of social and economic phenomena)

• Flexibility: explore relevant new ideas

and issues that may not have been anticipated

in the planning stage

• Quick results from different client segments

and other stakeholders

• Provision of information from knowledgeable

people

• Encourage interactions among participants

• Inherent checks and balances minimize

false or extreme views

• Limited reliability and validity concerning

specific local conditions

• Lack of quantitative data from which generalizations

can be made for a whole

population

• Validity and reliability can be undermined

by the flexible format, which is susceptible

to facilitator bias

• Discussions can de dominated or manipulated

by a small number of individuals

Tools: Individuals/groups:

• Key informant interviews

• Focus group interviews

• Community interviews

• Direct observation

• Mini-surveys

• programme officials

• client focus groups

• village head, traditional community representatives

• local government officials

• retailers, craftsmen

• peasants

• agricultural extension personnel

• academics

Technical notes on Focus Group Interviews and Key Informant Interviews follow

below. These are the main tools used in rapid appraisal and can serve as examples

for the other qualitative rapid appraisal tools listed above.

171


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

FOCUS GROUP INTERVIEWS

A focus group interview is an inexpensive rapid appraisal technique that provides

quick qualitative information to aid in the process of formulating impact hypotheses

and identifying and selecting indicator sets. A facilitator guides 5 to 12 people in a

discussion about their experiences, feelings, and preferences about having access

to financial and social intermediation services. Sessions typically last one to two

hours.

Steps:

1. Select the team: should be small, with at least one facilitator who speaks

the local language

2. Select the participants: consult key informants to select strategic individuals

3. Decide on timing and location

4. Prepare the discussion guide: outline issues and topics to be discussed

5. Conduct interview: establish rapport, phrase questions carefully, use probing

techniques (repetition of question, adoption of ”sophisticated naïveté”

posture, pause for answers, repeat replies, ask when, what, where, which,

and how questions, use neutral comments, control the discussion, minimize

group pressure)

6. Record the discussion

7. Analyse results: read summaries together, read transcripts, analyse each

question separately (words, framework, internal agreement, precision responses,

the big picture, report purpose)

KEY INFORMANT INTERVIEWS

Key informant interviews are qualitative, in-depth interviews of individuals selected

for their first-hand knowledge about a topic of interest. The number of key informants

depends on the number of topics to be investigated. If possible, it is advisable

to select several key informants on the same topic in order to avoid a subjective

information base. The interviews are loosely structured, relying on a list of issues

to be discussed. Key informant interviews resemble a conversation among

acquaintances, allowing a free flow of ideas and information. Interviewers frame

questions spontaneously, probe for information and take notes.

Steps:

1. Formulate study questions

2. Prepare a short interview guide

3. Select key informants

4. Conduct interviews: establish rapport, sequence questions, phrase questions

carefully, use probing techniques, maintain a neutral attitude, minimize translation

difficulties

5. Take adequate notes

6. Analyse interview data: interview summary sheets, descriptive codes, storage

and retrieval, presentation of data

7. Check for reliability and validity: representative key informants, reliable key

informants, possible bias of interviewer or investigator, negative evidence, informant

feedback

172


ANNEX 3: INTERVIEW GUIDE 61

BASIC GUIDELINES FOR INTERVIEWERS

ANNEX 3 – INTERVIEW GUIDE

1. Provide a comfortable interview atmosphere that will help build the trust of the

respondents. This will enable you to have an honest discussion.

Emphasize the confidentiality of the interview (exceptions to this confidentiality must be

agreed to by respondents).

Be sensitive to the attitudes and beliefs of the respondents and try not to offend them.

Show sincere interest in the person or people being interviewed beyond just getting the

information.

Explain the purpose of the interview, how the respondents were selected, how their responses

will be used, and whether they will have the opportunity to see the results of the

interview or evaluation. Allow them to ask questions about the interview.

Try not to interrupt the respondents.

Prepare an interview guide that contains all of the questions you will ask in the interview.

Begin with the questions that are the least threatening or those that are the easiest to answer.

2. Communicate the questions clearly and consistently.

Give the same explanations and directions to each respondent.

Try to read the question in the same way for each respondent.

Make sure every question is asked.

Do not show your own feelings about the question or expected response.

3. Record responses carefully and thoroughly. This involves skills in active listening,

observation, and unbiased recording.

Record responses as inconspicuously as possible. NOTE: If responses are to be recorded,

the respondents should be told and give their permission.

Have a consistent format for recording responses that allows for recording both verbal and

non-verbal responses.

Listen carefully and record responses, remarks and comments just as given, using respondents'

exact words.

Ask respondents to clarify vague or ambiguous answers; get them to be more specific

when necessary.

Observe and note non-verbal responses such as facial expressions, other body language,

and marked silences, etc.

Keep eye contact with the respondents as much as possible.

Do not offer your own comments and views about their responses or demonstrate a value

judgment on what they have said.

Do not read into what respondents say; ask for clarification if the answer is unclear.

Remain "tuned in" to the respondents' responses even while taking notes.

Make sure records are readable.

4. Keep the interview on track.

Keep the respondents' attention on the question being asked. Gently bring them back if

they get off track.

Probe for a response and further information if necessary.

Review the interview guide to make sure all questions were answered before ending the

interview.

Be prepared for unexpected problems by being flexible.

When ending the interview, thank the respondents for their time and effort!

61 Source after: ULR: http://www.innonet.org/

173


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

MORE INFORMATION ON INTERVIEWS

BEST TYPE OF INFO TO GATHER

Exploratory and probing

Verbal, spontaneous

Good sources for impressions and concrete information

Useful for verifying other information collected through different means

BEST TYPE OF RESPONDENTS

Anyone willing to discuss personal matters, reactions, attitudes, etc. with an appropriate

interviewer

BENEFITS

Allows for gathering rich, in-depth information fairly quickly

Natural form for respondents to share information; answers are generally more reliable

when interviews are conducted face-to-face

Allows for spontaneity and further probing, which may lead to important, but unexpected,

issues

Respondents do not have to be literate

POTENTIAL DRAWBACKS (AND HOW TO AVOID THEM): Interviews tend to be more

costly and time-consuming, and thus reach fewer people than surveys.

If appropriate, small group interviews can be done instead of individual interviews.

Use a facilitator who already has a good rapport with the respondents and train him/her

in basic interviewing skills. Facilitator biases and lack of skills in interviewing can jeopardize

the entire interview.

Provide training to facilitators in at least basic interviewing skills, covering basic guidelines

for interviewers. Less structured interviews may not provide the information

needed.

Prepare an interview guide with an introduction for facilitators to follow during interviews.

The guide should pose questions in the exact format and order in which they will

be asked.

Writing down responses during interviews can be difficult. Prepare a format for recording

responses and include it with the interview guide.

If appropriate, emphasize to the respondents that their answers are confidential.

Try to have two people facilitating interviews: one to conduct the interview and the

other to record responses.

If recording responses seems to slow down the interview, let respondents know that

their answers are being noted thoroughly because what they have to say is important.

Long, detailed responses are more difficult to analyse, especially if the interview has

gotten off track or if responses are vague.

When appropriate, include a data analysis plan with the interview guide.

Probe further for needed information.

Keep the respondent focused on the question being asked.

174


ANNEX 4: SAMPLING METHODS 62

ANNEX 4 – SAMPLING METHODS

This annex provides additional useful hints about how to choose a sample for impact

monitoring and assessment that ensures a high level of data accuracy.

THE SAMPLE POPULATION

A sample population is the total number of people who belong to a specific group

from which we need information. For example, the clients of an MFI could be a sample

population. All sampling processes are based on the following steps.

1. THE THEORETICAL POPULATION Who do we want to look at generally?

2. THE STUDY POPULATION What population can we access?

3. THE SAMPLING FRAME How can we access them?

4. THE SAMPLE Who is in our study?

After determining the study population, we need to determine the sample size. The

sample size is the number of people in a population from whom we should collect

information in order to be able to generalize about the entire population with some

degree of accuracy. If resources are very limited, we may need to settle for a

smaller sample size and thus a lesser degree of accuracy.

A SIMPLE TABLE FOR DETERMINING SAMPLE SIZE

To determine how many people are needed to interview or survey, first determine

the total population. See the table below. 63 By questioning the number of people in

the sample, you can accurately predict how the entire population will respond. If you

question the number of people in the column labelled "Less Accuracy" there is a

90% chance that if you asked everyone in the population the same question, they

would respond in a similar way. If you question the number of people in the column

labelled "Greater Accuracy" there is a 95% chance that if you had asked everyone in

the population the same question, that they would respond in the same way.

62

The descriptions used are taken from a comprehensive web-based textbook, Research Methods

Knowledge Base (© William M.K.Trochin 1998-2000) and were complemented with examples from

an impact assessment case. Source: URL http://trochim.human.cornell.edu/kb/sampling.htm

63

This table was adapted from two tables taken from The Saint Paul Foundation (1993), How to

Evaluate Foundation Programmes, The Saint Paul Foundation, St. Paul, MN.

175


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Population

Sample Size

Less Accuracy* Greater Accuracy**

10 8 9

50 28 44

100 40 79

150 46 108

300 55 168

500 59 217

1000 63 277

1500 64 305

1800 65 316

NON-PROBABILITY AND PROBABILITY SAMPLING

The difference between non-probability and probability sampling is that non-probability

sampling does not involve random selection and probability sampling does.

With a probability sample, we know the odds or probability that we have represented

the population well. We are able to estimate confidence intervals for the statistics.

With non-probability samples, it is difficult to know whether we have represented the

population well. It is thus important to choose the best sampling method and state

the limitations and potential of the one selected.

In science, probability or random sampling methods are preferred over nonprobability

methods because they are considered to be more accurate and rigorous.

However, in applied social research, which includes impact monitoring and assessment,

there may be circumstances when it is not feasible, practical, affordable or

theoretically sensible to do random sampling. Below, we consider a wide range of

non-probability alternatives.

Non-probability sampling

We can divide non-probability sampling methods into two broad categories:

accidental and purposive. In many research contexts, we compose a sample by

asking for volunteers either in an accidental or haphazard way or by convenience

sampling. The problem with these types of samples is that we have no evidence

that they are representative of the general population we are interested in -

questioning, and, in many cases, we clearly suspect that they are not.

176

* "Lesser" refers to a degree of

accuracy = + 0.10; Proportion

of Sample Size = 0.50;

Confidence Level = 90%

**"Greater" refers to a degree

of accuracy = + 0.05;

Proportion of Sample Size =

0.50; Confidence Level = 95% .


Purposive Sampling

ANNEX 4 – SAMPLING METHODS

In purposive sampling, there is usually one or more specific predefined groups

under study. This is the case in microfinance impact assessment. Purposive sampling

can be very useful when you need to conduct a targeted sample quickly and

when sampling for proportionality is not the primary concern. Purposive samples are

likely to ascertain the opinions of the target population, but are also likely to overweight

subgroups in the population that are more readily accessible.

Each of the following methods is a subcategory of purposive sampling methods.

• Modal Instance Sampling

As explained earlier, the mode is the most frequently occurring value in a

distribution. In a modal instance sample, the most frequent, or typical case is

sampled. Modal instance sampling is only sensible in informal sampling contexts,

because one cannot be sure that the variables chosen (such as income,

age, etc.) are the only or the most relevant ones for classifying the

typical client.

• Expert Sampling

Expert sampling assembles a sample of persons with known or demonstrable

experience or expertise in some area. Such a sample is often carried out

under the auspices of a panel of experts. Key informant interviews fall into

this category. There are two reasons to do expert sampling. It is the best way

to elicit the views of people with specific expertise. In such cases, expert

sampling is a subset of purposive sampling. The other reason to use expert

sampling is to provide evidence for the validity of another sampling approach.

An expert panel consisting of people with acknowledged experience

and insight into a field or topic can be asked to examine the modal definitions

you have developed and comment on their appropriateness and validity. The

advantage in such a case is that the researcher does not need to defend

their decisions alone. Rather, they have known experts backing them. The

disadvantage is that the experts can be, and often are, wrong and/or biased.

• Quota Sampling

In quota sampling, you select people non-randomly according to a fixed

quota. There are two types of quota sampling: proportional and non proportional.

In proportional quota sampling the major characteristics of the population

are represented by sampling a proportional amount of each. For in-

177


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

178

stance, if you know the population (MFI clients) is 40% women and 60%

men, and you want a total sample size of 100, you continue sampling until

you arrive at those percentages and then you stop. Thus, if you have already

sampled 40 women, but not sixty men, you continue to sample only men

even if legitimate women respondents come along, because you have already

met your quota. The challenge here (as with much purposive sampling)

is that you must decide upon which characteristics you will base the

quota.

• Non-proportional quota sampling is a bit less restrictive. With this method,

you specify the minimum number of sampled units you desire in each category.

However, you are not concerned with having numbers that match the

proportions in the population. Rather, you ensure that you sample enough to

be able to talk about even small groups in the population. This method is the

non-probability analogue of stratified random sampling because it is typically

used to assure that smaller groups are adequately represented in your sample.

• Heterogeneity Sampling

We use heterogeneity sampling when we want to include all opinions or

views and we are not concerned about representing views proportionately.

Another term for this is sampling for diversity. In many brainstorming or

nominal group processes (including RRA and PLA methods), we use some

form of heterogeneity sampling because our primary interest is getting a

broad spectrum of ideas, rather than identifying the average or modal instance

ideas. In effect, we would like to be sampling ideas rather than people.

We imagine that there is a universe of all possible ideas relevant to a

topic and that we want to sample this population rather than the population

that has the ideas. In order to understand all of the ideas held, especially the

unusual ones, we need to include a broad and diverse range of participants.

• Snowball Sampling

In snowball sampling, you begin by identifying someone who meets the criteria

for inclusion in your study, then you ask them to recommend others who

they may know who also meet the criteria. With this method, you capitalize

on informal social networks to identify specific respondents who are otherwise

hard to locate. Although this method hardly leads to representative

samples, there are times when it may be the best method available. Snow-


ANNEX 4 – SAMPLING METHODS

ball sampling is especially useful when you are trying to reach populations

that are inaccessible or hard to find.

Probability Sampling

A probability sampling method is any method that utilizes some form of random

selection. In random selection, you must ensure that the different units of your

population have equal probabilities of being chosen. People have long practiced

various forms of random selection, such as picking a name out of a hat, or drawing

straws. These days, we tend to use computers as the mechanism for generating

random numbers as the basis for random selection (for example, EXCEL has such a

programme).

Before explaining various probability methods, we define some basic terms in the

table below.

N = the number of cases in the sampling frame

n = the number of cases in the sample

NCn = the number of combinations (subsets) of n from N

f = n/N = the sampling fraction

Some basic methods of random samplings are explained following.

Simple Random Sampling

The simplest form of random sampling is called simple random sampling. The

objective is to select n units from N, ensuring that each NCn has an equal chance of

being selected.

Procedure: Use a table of random numbers, a computer random number generator,

or a mechanical device to select the sample (using the names in a hat or ball machine

procedure).

For microfinance, the following steps explain how to select a simple random sample.

First, organize the sampling frame. To accomplish this, go through agency records

to identify clients. We are lucky if the agency has accurate computerized records

that can quickly produce such a list. Then, decide on the number of clients to have

in the final sample and select the actual sample. If we want to select 30 MFI clients

179


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

to survey and there are 300 clients in total, then the sampling fractions are f = n/N =

30/300 = .10 or 10%. With the use of computers, there is a much easier way to do

this, especially if the names of the clients are already in a computer. Many computer

programmes can generate a series of random numbers. Let's assume that you can

copy and paste the list of client names into a column of an EXCEL spreadsheet.

Then, in the column right next to it, paste the function =RAND() which is EXCEL's

way of putting a random number between 0 and 1 in the cells. Then, sort both columns

-- the list of names and the random numbers -- by the random numbers. This

rearranges the list in random order from the lowest to the highest random number.

Then, all you have to do is take the first thirty names in the sorted list. You could

probably accomplish the whole thing in under a minute.

Simple random sampling is simple to accomplish and is easy to explain to others.

Because simple random sampling is a fair way to select a sample, it is reasonable to

generalize the results from the sample back to the population. Simple random sampling

is not the most statistically efficient method of sampling, however, and you may

not get good representation of subgroups in a population because of the luck of the

draw. To avoid these problems, we must use other sampling methods.

Systematic Random Sampling

For this type of sampling to work, the units in the population must be randomly ordered,

at least with respect to the characteristics that you are measuring.

Procedure: Below are the steps to follow:

180

• number the units in the population from 1 to N

• decide on the n (sample size) that you want or need

• k = N/n = the interval size

• randomly select an integer between 1 and k

• then take every k th unit

This will be clearer with an example. Assume that we have a population of N=100

people and you want to take a sample of n=20. To use systematic sampling, the

population must be listed in random order. The sampling fraction would be f =

20/100 = 20% and the interval size, k, is equal to N/n = 100/20 = 5. Next, select a

random integer from 1 to 5; let us say that you chose 4. Then, to select the sample,


ANNEX 4 – SAMPLING METHODS

start with the 4th unit in the list and take every k-th unit (every 5th, because k=5) for

the sample. You would thus sample units 4, 9, 14, 19, and so on, and would end up

with 20 units in your sample.

Stratified Random Sampling

Stratified random sampling, also sometimes called proportional or quota random

sampling, involves dividing your population into homogeneous subgroups and then

taking a simple random sample in each subgroup.

Procedure: Divide the population into non-overlapping groups (i.e., strata) N1, N2,

N3, ... Ni, such that N1 + N2 + N3 + ... + Ni = N. Then do a simple random sample of f

= n/N in each strata.

There are several reasons why you might prefer stratified sampling over simple random

sampling. First, it assures that you will be able to represent not only

programme participants, but also key subgroups such as new MFI clients, clients

with memberships from one to two years, and clients with more than three years of

membership. Another possibility is stratifying the sample into women and men or

rural and urban clients. This may be the only way to effectively assure that you will

be able to talk about specific subgroups. If the subgroup is extremely small, you can

use different sampling fractions (f) within the different strata to randomly over-sample

the small group. If you do so, however, you will then need to weight the smallgroup

estimates using the sampling fraction whenever you want overall population

estimates. When we use the same sampling fraction within strata, we are conducting

proportionate stratified random sampling. When we use different sampling fractions

in the strata, we call this disproportionate stratified random sampling. Stratified

random sampling is generally more statistically precise than simple random sampling.

This is only true if the strata or groups are homogeneous. If they are, we expect

that the variability within groups is lower than the variability for the population

as a whole. Stratified sampling capitalizes on this fact. 64

64 For a very clear and vivid presentation of examples, see previous footnotes.

181


MONITORING & ASSESSMENT IN MICROFINANCE PROGRAMMES

Cluster (Area) Random Sampling

One problem with random sampling methods is that if we need to sample a population

that is disbursed across a wide geographic region, we will need to cover a lot of

ground geographically in order to reach the units sampled.

Cluster sampling eases this problem by following the steps below.

divide the population into clusters (usually along geographic boundaries,

such as intervention area)

randomly sample clusters

measure all units within sampled clusters

For instance, let’s assume that we want to conduct an impact assessment for a programme

that consists of 50 village banks and the intervention area covers five counties

of country x. If we do a simple random sample of clients from the entire intervention

area, we will need to cover the entire geographic region. Instead, we decide

to do a cluster sample of ten village banks in two counties. Once these are selected,

we go to every village bank in the two areas. This strategy helps us cut down on

mileage. Cluster or area sampling is useful in such situations and is done primarily

for administrative efficiency.

182

More magazines by this user
Similar magazines