The GBSC Program Group Discussion Key Vocabulary / Terms

The GBSC Program
The GBSC Program
Group Discussion
Key Vocabulary / Terms
1. Level of Randomization: the level of observation (ex. individual, household, school, village) at
which treatment and comparison groups are randomly assigned.
2. Sampling Frame: The complete list of the population of interest in the study. This is not
necessarily the complete population of the country or area being studied, but is restricted to the eligible
population, eg. families with children under five, or female-headed households.
3. Ways to Randomize:
Lottery: Appropriate when there are limited resources or implementation capacity and demand for a
program or service exceeds supply.
Lottery at the margin: Random assignment when not able/willing to randomize among
eligible people but for people who are at borderline of eligibility.
Rotation: Randomization where groups receive treatment in turns. Concern is managing expectations
and measuring long-run impacts.
Phase-in: Staggered randomization that follows natural expansion of a program. Concern is
managing expectations.
Encouragement: When access to a program cannot be restricted for ethical/practical reasons,
randomly assign an incentive to the treatment group to increase probability of being treated.
4. Threats to Validity
Attrition: Either the drop out of participants from the treatment group during the intervention, or
failure to collect data from a unit in subsequent rounds of a panel data survey. Either form of attrition
can result in biased impact estimates.
Partial/Imperfect Compliance: This occurs when a fraction of the individuals who are offered the
treatment take it up or when members of the comparison group may receive the treatment. The
potential for compliance issues should be anticipated at the design stage; in order to be valid analysis
must use groups assigned from initial randomization.
Spillover: When the intervention has an impact (either positive or negative) on units not in the
treatment group. Ignoring spillover effects will result in a biased impact estimate. If there are spillover
effects then the group of beneficiaries is larger than the group of participants. When the spillover
affects members of the control group, this is a special case of contagion.
i
Background:
A dynamic rural bank in Ethiopia, the Greater Bahir Dar Savings Cooperative (GBSC), is about to
launch a large scale up of its operations in the Amhara region. GBSC has worked for over 20 years in
this region where 90% of the population, and virtually everyone outside the regional capital, lacks
access to formal financial services. Over the last decade, GBSC has developed a very cost-effective
1

The GBSC Program
protocol for delivery of financial services in rural areas using point of sale technology. The model
works as follows: GBSC vets and enrolls individuals who live in rural areas to become agents of the
cooperative. These individuals then enroll and carry out transactions for GBSC clients using a
biometric point of sale technology. Accounts can be used to send money or pay bills, but GBSC also
hopes that clients can maintain a positive balance on their accounts and thus accumulate savings.
Agents often operate out of a central, existing place of work, such as a general store, though
sometimes they travel to client homes in surrounding villages. GBSC coordinators visit the agents
on a weekly basis to pick up and deliver cash, though they can come immediately if there is an
urgent liquidity issue. Starting in 2011, after a promising pilot, the management began an ambitious
marketing campaign, planning to open 15,000 new savings accounts in four provinces by the end of
the year.
While GBSC’s new product appears to have no technological glitches, and in initial focus group
discussions customers have been keen to open an account, take-up of the product levels remain
very low. Instead of the targeted 15,000 accounts, GBSC only managed to open 3,700 accounts.
Among those who have opened an account, many do not seem to use it for savings, continuing
instead to save by storing money under their mattress or through the purchase of jewelry or
livestock. At the end of December, GBSC conducted a short representative survey of about 3,000
adults living in the villages where their 200 GBSC agents operate and held a series of focus groups
examining attitudes towards saving and consumption. Some of these villages have a population of
80, while others are as large as thirty thousand. The survey showed that about 800 of the surveyed
individuals only a small percentage have an GBSC account. Responses from the focus groups
revealed low levels of information about the benefits of savings, ignorance of details regarding how
the account worked, and a preference for the tangible / visible reminders of their savings that
livestock and jewelry provided.
GBSC has reached out to IPA researchers to conduct a study testing the impacts of alternative
strategies to spur uptake among their new clients. They decided to provide information to
establish trust among clients, explain the benefits of the account, and explain how the account
works. They have considered the following range of interventions:
- workshops and community meetings
- individual visits to potential clients
- advertisements (using posters in community centers, radio or television, leaflets)
- text message based promotions
- promotions such as a raffle among those who enroll
GBSC would like to know whether providing information increases the likelihood that potential
clients will sign up for accounts. They would also like to know whether accountholders who
received information end up saving more.
2

The GBSC Program
Group Discussion
How would you evaluate the effectiveness of the proposed marketing campaign Choose a couple of
the alternatives that GBSC proposes design a study that keeps logistical, financial, and political
constraints under consideration.
1) First, specify the level of randomization: Of the interventions that GBSC is considering,
which are more appropriate to individual level randomization Which would require group
level randomization Discuss the advantages and disadvantages of each, including potential
threats to validity.
2) How would you suggest that GBSC actually conduct the intervention(s) in order to make a
randomized trial possible
3) In studies that look at takeup as an important outcome variable, you need to have a sample
frame. Examples include: list of farmers belonging to a particular association to see what
percent uses fertilizer; census of adults in a village to see who takes up a micro-loan;
recipients of marketing letters or emails with different messages. What is your sample
frame in this study How will you select the control group
4) What if GBSC were to try more than one intervention They are curious to know if one
marketing method is more cost-effective than another, and if there is a multiplier effect
when two interventions are combined. How would you design the study to answer these
questions
i Some definitions paraphrased from 3ie (2009) 3ie impact evaluation glossary. International Initiative for Impact
Evaluation: New Delhi, India
3