Citizen Report Cards to Improve Service Delivery - South Africa ...
Citizen Report Cards to Improve Service Delivery - South Africa ...
Citizen Report Cards to Improve Service Delivery - South Africa ...
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>South</strong> <strong>Africa</strong><br />
<strong>Citizen</strong> <strong>Report</strong> <strong>Cards</strong> <strong>to</strong> <strong>Improve</strong> <strong>Service</strong> <strong>Delivery</strong><br />
A Preliminary Outline<br />
Moni<strong>to</strong>ring and Impact Evaluation Conference<br />
The Presidency and The World Bank<br />
June 19-23, 2006<br />
Background<br />
Too often, services fail poor people—in access, in quantity, in quality. But the fact that there<br />
are strong examples where services do work means governments and citizens can do better.<br />
How? By putting poor people at the center of service provision: by enabling them <strong>to</strong> moni<strong>to</strong>r<br />
and discipline service providers, by amplifying their voice in policymaking, and by<br />
strengthening the incentives for providers <strong>to</strong> serve their clients.<br />
The public sec<strong>to</strong>r has generally taken on responsibility for the delivery of public<br />
services and frequently used civil service bureaucracies as the instrument. This approach has<br />
had dramatic successes and many failures across the world. Particularly for poor people,<br />
there are widespread challenges in providing affordable access, fixing dysfunctional facilities,<br />
improving technical quality, increasing client responsiveness, and raising productivity.<br />
Neither economic growth, nor simply increasing public spending, nor coming up with<br />
technocratic solutions is enough <strong>to</strong> meet this challenge.<br />
The failures in service provision have not gone unnoticed. Indeed, there is a whole<br />
host of proposed institutional solutions: civil service reform, privatization, democratization,<br />
decentralization, contracting out, provision through NGOs, empowerment, participa<strong>to</strong>ry<br />
methods, social funds, community driven development, user associations. With each of these<br />
solutions comes a variety of techniques and instruments: demand-side transfers, participa<strong>to</strong>ry<br />
rural appraisals, facility surveys, participa<strong>to</strong>ry budgets. None is a panacea.<br />
To analyze service delivery, it is helpful <strong>to</strong> have a simple conceptual framework. Let<br />
us start with the specific, such as a child in a classroom, a pregnant woman at a clinic,<br />
someone turning a tap for water. Each is seeking a service, and the proximate determinants of<br />
success are clear. For any individual service transaction <strong>to</strong> be successful, there needs <strong>to</strong> be a<br />
frontline provider who is capable, who has access <strong>to</strong> adequate resources and inputs, and who<br />
is motivated <strong>to</strong> pursue an achievable goal. The general question: what institutional<br />
conditions support the emergence of capable, motivated frontline providers with clear<br />
objectives and adequate resources? The answer: successful services emerge from institutional<br />
relationships in which the ac<strong>to</strong>rs are accountable <strong>to</strong> each other.<br />
The Relationships of Accountability<br />
For the conceptual framework, we distinguish four relationships of accountably in service<br />
delivery: those between citizen/clients, politicians, provider organizations, and frontline<br />
service providers.<br />
1
Of politicians <strong>to</strong> citizens. The term voice is used <strong>to</strong> express the complex relationships<br />
of accountability between citizens and politicians. Voice is about politics, but it covers much<br />
more. The voice relationship includes formal political mechanisms (political parties and<br />
elections) and informal ones (advocacy groups and public information campaigns).<br />
Delegation and finance between citizen and state are the decisions about pursuing collective<br />
objectives and mobilizing of public resources <strong>to</strong> meet those objectives. <strong>Citizen</strong>s need<br />
information about how actions of the state have promoted their well-being. They also need<br />
some mechanism for enforceability, <strong>to</strong> make sure that politicians and policymakers are<br />
rewarded for good actions and penalized for bad ones. If politicians have abused their<br />
position, or even just not pursued objectives aggressively and effectively, citizens need a<br />
variety of mechanisms—not just periodic elections—<strong>to</strong> make politicians and policymakers<br />
accountable.<br />
Of the organizational provider <strong>to</strong> the state. The relationships between policymakers<br />
and service providers can be thought of as compacts. The compact is not always as specific<br />
and legally enforceable as a contract, though a contract can be one form of a compact.<br />
Instead, it is a broad agreement about a long-term relationship. The policymaker provides<br />
resources and delegates powers and responsibility for collective objectives <strong>to</strong> the service<br />
providers. The policymaker generates information about the performance of organizations.<br />
Enforceability comes in<strong>to</strong> play when the compact also specifies the rewards (and possibly the<br />
penalties) that depend on the service provider’s actions and outputs. The line between “the<br />
state” and “public sec<strong>to</strong>r organizational provider” is not always easy <strong>to</strong> draw.<br />
Of the frontline professionals <strong>to</strong> the organizational provider and its management. In<br />
every organization formal and informal <strong>to</strong>ols of management provide frontline workers with<br />
assignments and delineated areas of responsibility, equipping them with the resources <strong>to</strong> act.<br />
In public agencies this management function is at times blurred because providers are<br />
employees of “the government.” But all the standard management issues of selecting,<br />
training, and motivating workers in an organization apply <strong>to</strong> all organizations—private, NGO,<br />
government, whatever. All service provision organizations—whether a government ministry,<br />
a religious body, a nonprofit NGO, or a for-profit firm—have <strong>to</strong> create a relationship of<br />
accountability with their frontline providers.<br />
Of the provider <strong>to</strong> the citizen-client: client power. Because the policymaker cannot<br />
specify all actions of providers in the compact, citizens must reveal <strong>to</strong> providers their demand<br />
for services and moni<strong>to</strong>r the providers’ provision of services. Clients and organizational<br />
providers interact through the individuals who provide services—teachers, doc<strong>to</strong>rs, engineers,<br />
repairmen—the frontline professionals and frontline workers.<br />
How is accountability achieved in the public sec<strong>to</strong>r? To begin <strong>to</strong> answer this question,<br />
compare how accountability is achieved in many market contexts. In the market, dissatisfied<br />
consumers can successfully use the exit option; i.e., if the price is <strong>to</strong>o high or the quality <strong>to</strong>o<br />
low, the consumer can choose not <strong>to</strong> buy the good or buy from another producer. If many<br />
consumers act in the same way, this will influence the producer’s profitability and, in the end,<br />
its survival in the market. The exit mechanism, however, may not work well in the public<br />
sec<strong>to</strong>r. First of all, in some cases there may be no easily available alternative <strong>to</strong> the local<br />
public provider. Second, the link between the public provider’s performance and its financial<br />
position (or its staff’s remuneration) is typically weak or non-existent.<br />
2
Governments have tried <strong>to</strong> compensate the lack of a well-functioning exit mechanism<br />
by increasing control. That is, create or strengthen legal and financial management methods<br />
<strong>to</strong> moni<strong>to</strong>r public sec<strong>to</strong>r performance. This is a <strong>to</strong>p down approach where some government<br />
agencies are assigned <strong>to</strong> control and moni<strong>to</strong>r others. Evidence from a wide spectrum of<br />
countries, however, shows that these systems can, at best, provide only a limited form of<br />
moni<strong>to</strong>ring.<br />
A complementary approach emphasizes citizens’ voice and “client power” as<br />
important elements of making services work for poor people. When the exit option has no<br />
bearing or is weak, strengthening the voice mechanism may be the only available option. In<br />
recent years, civil society organizations and others have experimented with various<br />
interventions this general objective in mind. A common thread across these interventions is<br />
that they take the users of public services as a starting point. Thus, rather than focusing on<br />
service providers' accountability <strong>to</strong> policymakers alone, the idea is <strong>to</strong> engage citizens at the<br />
bot<strong>to</strong>m of the public service delivery chain by providing easy access <strong>to</strong> information on the<br />
workings of public programs intended for their benefit. This empowers citizens <strong>to</strong> demand<br />
certain standards, <strong>to</strong> moni<strong>to</strong>r service quality, and <strong>to</strong> challenge abuses by officials with whom<br />
they interact.<br />
The <strong>to</strong>pic of this project proposal, the <strong>Citizen</strong> <strong>Report</strong> Card, is one of the new <strong>to</strong>ols<br />
aimed at increasing client power over frontline providers and hence at improving services.<br />
<strong>Citizen</strong> <strong>Report</strong> <strong>Cards</strong>: What Are They and Why Use Them?<br />
Some information strategies look directly at public service outputs (quality and quantity of<br />
services provided by government) rather than inputs (prices paid, budgets committed and<br />
disbursed). The best known are the citizen report cards developed by the Public Affairs<br />
Centre in Bangalore, India. 1 <strong>Citizen</strong>s are asked <strong>to</strong> rate service access and quality and <strong>to</strong> report<br />
on concerns about public services, general grievances, and corruption. <strong>Citizen</strong> report cards<br />
have spread beyond Bangalore <strong>to</strong> cities in Kenya, Mozambique, the Philippines, Ukraine, and<br />
Vietnam. They have been scaled up in India <strong>to</strong> cover urban and rural services in 24 states. In<br />
Uganda, they have been applied <strong>to</strong> health care at the local level in rural areas, and are<br />
currently being evaluated rigorously for impact. Overall, the citizen report cards have<br />
stimulated considerable media, bureaucratic, and political attention and there is general<br />
acknowledgment of their positive contribution <strong>to</strong> service improvements.<br />
Because citizen report cards focus on service outcomes, they do not necessarily<br />
provide citizens/clients with information about specific decisions that policymakers have<br />
taken—or not taken. Nor do they give them information (at least in their first round) about<br />
service benchmarks, except <strong>to</strong> the extent that the agencies themselves have established service<br />
standards (but repeat report cards do provide implicit benchmarks from the previous report<br />
card). So it can be hard for citizens/clients <strong>to</strong> assess, on the basis of one report card, whether<br />
the results justify voting against the incumbents at the next election.<br />
<strong>Citizen</strong> <strong>Report</strong> <strong>Cards</strong> in Bangalore seem <strong>to</strong> have had a more direct influence on the<br />
heads and senior managers of the municipal and utility agencies responsible for services. The<br />
high visibility of report cards in the press and civic forums turned them in<strong>to</strong> league tables of<br />
1 Samuel Paul, 2002, Holding the State <strong>to</strong> Account: <strong>Citizen</strong> Moni<strong>to</strong>ring in Action. Bangalore: Books for Change.<br />
3
the efficacy of municipal agencies. The reputational competition arising from the report cards<br />
is enhanced by joint agency meetings attended by prominent social and political leaders and<br />
citizens.<br />
A <strong>Report</strong> Card is a <strong>to</strong>ol <strong>to</strong> collect feedback from the users (and potential users) of<br />
public services and disseminate this information back <strong>to</strong> the citizens/users so they have<br />
reliable information about how their neighborhood/community at large views the quality and<br />
efficacy of service delivery, and so they can compare service delivery in their neighborhood<br />
vis-à-vis other neighborhoods in their city, larger metropolitan area, or across cities or<br />
municipalities in the country at large. The basic idea is that reliable and representative<br />
information about the users’ experiences and entitlements is critical for citizens’ ability <strong>to</strong><br />
moni<strong>to</strong>r service providers. It also improves users’ ability <strong>to</strong> challenge abuses of the system,<br />
since reliable quantitative information is more difficult for service providers <strong>to</strong> brush aside as<br />
anecdotal, partial, or simply irrelevant.<br />
Practically, a <strong>Report</strong> Card has three components: (a) collecting quantitative<br />
information from users (citizens) and service providers, using micro survey techniques; (b)<br />
assembling this information in “easy access/comprehensible report cards”; (c) disseminating<br />
the report cards <strong>to</strong> users and providers and providing them with practical information on how<br />
best <strong>to</strong> use this information; and (d) implementing repeat user and provider surveys <strong>to</strong> assess<br />
impact on service delivery outcomes.<br />
Proposal for Consultative <strong>Report</strong> Card in Municipal <strong>Service</strong>s in <strong>South</strong> <strong>Africa</strong><br />
Currently, a Consultative <strong>Report</strong> Card initiative is being trialed in <strong>South</strong> <strong>Africa</strong> in the metro<br />
Tshwane area (questionnaire attached), in collaboration with the National Treasury and the<br />
Human Sciences Research Council (HSRC). Following the pilot, the plan is <strong>to</strong> extend the<br />
<strong>Report</strong> <strong>Cards</strong> <strong>to</strong> another 40 or so municipalities across <strong>South</strong> <strong>Africa</strong>, rural and urban. In all<br />
instances, the <strong>Report</strong> <strong>Cards</strong> will gather citizens’ input on the core services which<br />
municipalities are mandated <strong>to</strong> deliver, such as water, sanitation, waste disposal, etc. A<br />
critical element of the <strong>Report</strong> Card process will be <strong>to</strong> ensure there is effective dissemination<br />
of the data and information gathered from the <strong>Report</strong> Card <strong>to</strong> citizens of that municipality –<br />
they, after all, will be at the crux of demanding and pushing for better service delivery from<br />
the municipality. To assess the efficacy of the <strong>Report</strong> Card in stimulating citizens’ voice and<br />
pressure for change, another 40 municipalities will be selected as control group. This<br />
approach will allow a rigorous impact assessment <strong>to</strong> be undertaken <strong>to</strong> determine whether<br />
citizens’ access <strong>to</strong> information is, indeed, an important fac<strong>to</strong>r responsible for positive changes<br />
recorded in service delivery. Ideally, the <strong>Report</strong> Card activity will result in citizens in those<br />
municipalities having solid and reliable information about the coverage, quality, and<br />
efficiency of services in their own neighborhood/municipality and the information necessary<br />
<strong>to</strong> assess service delivery in their locality relative <strong>to</strong> the other municipalities and <strong>to</strong> the<br />
national average. Advocacy based on such information can then lead <strong>to</strong> real improvements on<br />
the ground.<br />
The Tshwane pilot <strong>Report</strong> Card pilot is expected <strong>to</strong> be completed by the end of<br />
Oc<strong>to</strong>ber 2006. The expansion <strong>to</strong> other municipalities will begin as of November 2006,<br />
including surveys of users and service providers. Dissemination activities will take place in<br />
mid-2007 (and will possibly be repeated after a few months). The repeat surveys of users and<br />
providers <strong>to</strong> assess the impact of this intervention will be implemented in June-July 2008.<br />
4
Impact Evaluation of the <strong>Report</strong> <strong>Cards</strong><br />
Identifying and implementing incentives that give rise <strong>to</strong> a strong relationship of<br />
accountability between service providers and beneficiaries is viewed by many as critical for<br />
improving service delivery. How <strong>to</strong> achieve this, however, is less unders<strong>to</strong>od. Systematic<br />
evaluation of service delivery innovations <strong>to</strong> increase accountability can show what works,<br />
what doesn’t and why, a first step <strong>to</strong> scaling up successes.<br />
There are many different ways <strong>to</strong> evaluate a program/project, each subject <strong>to</strong> more or<br />
less serious constraints. An impact evaluation attempts <strong>to</strong> answer the counterfactual question:<br />
how would individuals or municipalities that participated in a program, for instance, the<br />
<strong>Report</strong> <strong>Cards</strong>, have fared in the absence of the program? The difficulty with this question is<br />
immediate: at a given point in time, an individual or municipality is observed <strong>to</strong> either be<br />
exposed or not exposed <strong>to</strong> the program. It is therefore not possible <strong>to</strong> obtain an estimate of the<br />
impact of the program on each individual or municipality. However, it may be possible <strong>to</strong><br />
obtain the average impact of the program by comparing participating municipalities (the<br />
treatment group) with a similar group of municipalities who are not exposed <strong>to</strong> the program<br />
(the control group). One implication of this is that an evaluation based on statistical methods<br />
requires are “large” sample. This is one reason why the <strong>Report</strong> Card project will take its<br />
starting point the municipal level.<br />
The problem facing an evalua<strong>to</strong>r is how <strong>to</strong> design the evaluation study so as <strong>to</strong><br />
establish a credible comparison group -- a group of municipalities who in the absence of the<br />
program would have had outcomes similar <strong>to</strong> those who were exposed <strong>to</strong> the program. This<br />
group should give us an idea of what would have happened <strong>to</strong> the members of the program<br />
group, if they had not been exposed <strong>to</strong> the <strong>Report</strong> Card, and thus allow us <strong>to</strong> obtain an<br />
estimate of the average impact on the group in question.<br />
To solve this problem, program evaluations typically need <strong>to</strong> be carefully planned in<br />
advance in order <strong>to</strong> determine which group is a likely control group. One credible way <strong>to</strong><br />
ensure that the treatment and comparison groups are similar is <strong>to</strong> select randomly from a<br />
potential population of participants (such as municipalities). If the sample is large enough, one<br />
can then be assured that, on average, those who are exposed <strong>to</strong> the program are no different<br />
than those who are not, and thus that a statistically significant difference between the groups<br />
in the outcomes the program was planning <strong>to</strong> affect can be confidently attributed <strong>to</strong> the<br />
program.<br />
To evaluate a program, one also needs <strong>to</strong> have a quantifiable outcome measure(s).<br />
This, in turn, requires knowledge of how the program possibly could influence outcomes. In<br />
this particular case, we require knowledge about how households with access <strong>to</strong> information<br />
on how the local providers, and/or local government, perform (and what citizens are entitled<br />
<strong>to</strong>) can possibly influence the services they receive. A necessary condition for this <strong>to</strong> be the<br />
case is that the local providers and/or local governments must be able <strong>to</strong> influence the<br />
outcomes that are measured and that one of the reasons the initial outcomes are suboptimal is<br />
related <strong>to</strong> choices made by the provider or local government (for example, budgets are spent<br />
on items not benefiting the intended beneficiary, or low effort is exerted in executing local<br />
programs).<br />
5
The background study on the supply side of service delivery, with a focus on the local<br />
government level, was commissioned <strong>to</strong> assist in identifying what type of services possibly<br />
could be included in the <strong>Report</strong> Card project. 2 Similarly, focus groups of users, frontline<br />
service providers, and local politicians are being carried out in May-June 2006 in eight<br />
randomly selected municipalities <strong>to</strong> get a better sense of issues faced on the ground by both<br />
users and providers.<br />
Extension of Impact Evaluation <strong>to</strong> the “Compact”<br />
More accurate information on performance and entitlements may not be enough <strong>to</strong> strengthen<br />
accountability in the public sec<strong>to</strong>r. In fact, transparency can just produce cynicism if those<br />
misbehaving operate with impunity. Thus, information provision will not work unless there<br />
are members of the public willing <strong>to</strong> make use of the information and if citizens can either<br />
directly or indirectly sanction the officials or their superiors.<br />
One way <strong>to</strong> test this hypothesis would be <strong>to</strong> not only introduce variation in what type<br />
of information is disseminated back <strong>to</strong> households, but also experiment with increased<br />
moni<strong>to</strong>ring from upper-level government authorities. In this case, one must identify services<br />
that are provided by the municipality (or local providers) but where provincial or national<br />
agencies also have formal role in moni<strong>to</strong>ring (and possible sanction) the local providers.<br />
RReinikka/JSvensson/CWinter<br />
18/06/2006<br />
2 Doreen Atkinson, 2006, “The Status of <strong>Service</strong> <strong>Delivery</strong> and Impacts in <strong>South</strong> <strong>Africa</strong>:<br />
A Preliminary Overview” World Bank. Processed.<br />
6