Using Data to Influence Government Decisions

Using Data to InfluenceGovernment DecisionsOpportunities and Challenges forUser-Centered Design to ImproveGovernance Data ImpactVERSION FOR DISSEMINATIONPrepared for the Open Society FoundationsJuly 2015

In late 2014, the Open Society Foundations (OSF) commissioned Rebootto conduct a scoping study on the opportunities and challenges in leveraginguser-centered design to enhance the impact of governance datainitiatives. Focusing on a specific set of governance data communities,the study first sought to understand current program and product designpractices used by these communities. Then, it assessed the potentialimpact that broader design collaborations may have on their ability toinfluence government decision-making.The scoping study resulted in a final report for OSF to inform furtherwork on these topics. This document has been adapted from that reportto share project lessons with the broader transparency and accountabilitycommunity. This version aims to be useful for practitioners working onrelated initiatives to those studied, and to donors seeking to supportthem. For respondent privacy, it does not include details on specificinitiatives.Contents3 Introduction4 Toward a More NuancedApproach to Governance Ecosystems9 Recommendations For Donors14 Looking Forward1

DefinitionsAcknowledgmentsReboot is grateful to Janet Haven from the Open Society Foundations’Information Program for her support and thought partnership throughoutthis work, and to Laura Bacon from Omidyar Network for her early inputs.We would also like to thank our respondents—both independent practitionersand representatives from the organizations listed—who volunteeredtheir time to share their valuable insights with us: American Assembly,Development Initiatives, Fair Play Alliance, the Government of Mexico’sOffice of the President National Digital Strategy, International BudgetPartnership, LittleSis, Open Contracting Partnership, Open Corporates,Open North, Poderopedia, Practical Participation, Results for Development,and the World Bank Group.In the context of this report, the following terms have beenused in specific ways:Data Consumers (or Users): Actors who use information to influence governmentdecision-making (e.g. by integrating data into advocacy efforts).Data consumers can include groups or individuals who also generate newinformation about government practices and processes.Data Producers: Actors who generate new information or analysis ongovernment practices and processes that may be used to influence governmentdecision-making.Data Standards: Agreements on representations, formats, and definitionsof common information to help improve their utility, reusability, andactionability.Governance Data: Information on the structures and allocations of political,economic, and social authority and resources that impact governanceprocesses or outcomes. Such information may relate to topics such as ruleof law, legislative processes, public spending, and corporate influence inpolitics.Impact: Having observable influence on increasing the social accountabilityof government decisions, particularly as they relate to public spendingand resource allocation. This influence may be direct (e.g. by changingthe design or implementation of a policy or program) or indirect (e.g. byshifting institutional norms or culture that, in turn, influence individualdecisions).Multi-stakeholder Initiative (MSI): A governance structure that convenesdiverse actors for dialogue, decision-making, and action around commoninterests or objectives.User-Centered Design (UCD): A multi-stage problem-solving process thatoptimizes solutions based on users’ needs, behaviors, constraints, and operatingcontexts. Solutions are repeatedly tested and refined throughoutthe design and development process before implementation.2

INTRODUCTIONThe potential of new applications of data to improve governmentperformance and accountability is widely recognized. Data, presentedin innovative and widely accessible formats, has the ability toinfluence how public resources are allocated and accounted for.As a result, recent years have seen a proliferation of data advocacyefforts and data products that seek to influence governanceoutcomes. There are now several communities of practice employingdata in pursuit of greater government accountability; theirefforts span a range of issues including public spending, corporateinfluence in politics, and natural resources management. Further,multi-stakeholder initiatives (MSIs) are convening actors to developtechnical standards for data production and management. In doingso, they seek to make governance data more interoperable and dataproducts more usable.Yet, despite the significant investments in governance data effortsto date, evidence of their impact on governance outcomes remainsthin and far from systematic. Before defining technical norms, itis important to first understand why the data products currentlyproduced have had limited impact. Otherwise, there is a risk ofossifying ineffective practices into widely adopted norms. Technicalquality, while important, is secondary to real-world utility. Datastandards should be developed based on concrete cases, not theoreticalnotions, of pathways to impact.Through consultations with governance and open data experts, andobservations of open government fora, the Open Society Foundations’(OSF) Information Program and Reboot began exploring whythe impact of governance data has not yet met expectations. 1One explanation frequently cited by governancedata actors is that they lack clarity onwho their users are and why they might usegovernance data (or not). As a result, theirwork is largely based on assumptions of howdata could be used to influence governanceoutcomes. 2 To help address this issue, OSFand Reboot became interested in helping‘data producers’ better understand and serve‘data consumers.’ 3 We hypothesized thatgovernance data will be more impactfulif producers could identify their priorityaudiences; articulate how these audiencesmay incorporate governance data into theirwork; and optimize products and servicesbased on these audiences’ needs, capacities,and constraints.To test this hypothesis, OSF commissioneda scoping study to gauge the governancedata community’s interest and ability toengage in an experimental design exercise,and to determine the potential of such2 These sentiments were surfaced and reiterated by manyrespondents and was a key issue driving the function of theGovernance Data Alliance. The Alliance has established a UserFeedback Working Group, of which Reboot is a member,to help address this issue.3 For the purpose of this study, the term ‘consumer’ isused interchangeably with ‘user.’a demonstration project in influencingbroader open government impact and discourse.The study was conducted by Rebootfrom December 2014 to March 2015.Semi-structured, in-depth interviews wereconducted with 21 respondents; of these,eight were in-person and the rest werevia video or teleconference. Respondentsincluded staff and stakeholders of MSIsfrom diverse functional roles and positions(i.e. leadership, programmatic staff, boardmembers); experts in governance data; andpresumed data users.Beyond surfacing opportunities for furtherwork, this research generated observationson attitudes and ways of working in thegovernance data community that may belimiting its impact.This report includes overarching researchinsights, and presents implications andrecommendations for donors who seek tosupport governance data actors andcommunities.1 In this study, “impact” is defined as having observable influence on increasing the social accountabilityof government decisions, particularly as they relate to spending and resource allocation. This influencemay be direct (e.g. by changing the design or implementation of a policy or program) or indirect (e.g.|by shifting institutional norms or culture that, in turn, influence individual decisions).3

TOWARD A MORENUANCED APPROACH TOGOVERNANCE ECOSYSTEMSGovernance is ultimately about the structures and allocationsof authority, power, and resources. Governance outcomes aredetermined by complex ecosystems of actors engaged in ongoingcontests around the distribution of authority, power, and resources;influencing these outcomes therefore necessitates engaging inthese political contests. This research suggests, however, that todo so, the governance data community may benefit from newapproaches to conceptualizing and executing its work.This scoping study elicited two overarching challenges thatare limiting the potential of governance data actors:1. Popular (or commonly used) conceptions of governanceecosystems tend to be overly reductive; as a result, theyprevent the strategic targeting of actors who can usegovernance data to advance desired outcomes.2. Governance ecosystems are complicated by factors notusually acknowledged in strategic planning and programdesign processes. Such factors include government incentivestructures and political volatility.This section discusses these findings broadly in the context of thegovernance data community before offering recommendations fordonors seeking to act on them.4

Challenge One: Popular conceptions of governanceecosystems tend to be overly reductive and can hinderthe strategic targeting of efforts.Governance outcomes are determined by the relationships andinteractions between actors of complex, dynamic ecosystems.Within governance data communities, the commonly used categoriesof ‘data producer’ and ‘data consumer’ represent a convenient buttheoretical division of those who may engage with governance data.In practice, the relationship between such actors is not so neatlydefined. Use of these labels prevents an accurate understanding ofgovernance ecosystems and the chain of events that influencedifferent decisions.Current discourse also tends to assume that ‘users’ are to be‘designed for,’ rather than recognizing that there is a range of actorswho might be recruited, trained, lobbied, serviced, supported, orotherwise engaged to influence governance outcomes.The labels of ‘producer’ and ‘consumer’ are false binariesand should be discarded.Rather, it is necessary to map the range of actors who haveinfluence —formal and informal— over a specific governance issue,and to understand how their respective histories, interests, andrelationships intersect and collectively shape outcomes. This processhelps identify actors with significant influence on a particular issue,whether due to individual positions or to accumulated authorityfrom being at the nexus of networks or processes. Such actorsshould be targeted as priority data users.Blurring the Lines Between Data Providers and ConsumersStudy respondents, for the most part, rejected the labels of ‘data production’ or ‘data consumption’when describing their work around governance data. A few described starting their journeysin governance data as ‘data users’ that sought out and used data for a specific purpose. Overtime, they began to package and share their interpretations of the data they’d collected and, inthe process, became ‘data producers.’One member of a successful government watchdog group explained how, over a decade, thegroup grew from being a gatherer of governance data to a leading national producer of originalanalysis to inform diverse advocacy efforts.The organization was founded by a group of journalists that wanted to share information in supportof their investigative reporting. To assist with its own work, the group began capturing informationit received from the government in response to its filed Freedom of Information requests.Its own capacity, however, only went so far in identifying suspicious patterns in received data;consequently, the journalists sought support to expand their audience. The group opened accessto its data, with a focus on corporate influence and procurement data. Over time, it developed anopen data portfolio which includes tools for data mapping and analysis.Today, the organization positions itself as an advocacy group that provides access to data, andthat uses data to inform its own government engagement. As one staff member explained, “Ouropen data projects seek to not only create our own internal cases for fighting corruption, but toalso generally provide data to others [to achieve the same goals].”Governance data communities tend to refer to ‘users’ in broad categories,such as “government,” “private sector,” “civil society,” “media,” and others.Recognizing the heterogeneity within each category, and identifying priorityactors within them, will be critical to designing data products that advancethe desired impact.While segmenting by sector helps describe the potential universe of governancedata users, further specificity is needed to determine which actors aremore influential and hence priority targets.5

Challenge Two: Governance ecosystems are complicated by factors notusually acknowledged in strategic planning and program design processes.Such factors include government incentive structures and political volatility.Respondents surfaced many specifictypes of users that fall under the broad,sector-based, classifications. This can helpdata providers better understand who theyare designing for, how to appeal to theirincentives, and how to help them integratedata into existing activities and processes.Such an exercise will also lead to greaterdefinition of target users beyond thesector-based categorizations that areData UsersCertain governance data users aremore influential than others.Identifying them requires addinggranularity to commonly used,sector-based categorization.Some examples surfaced byrespondents include:GOVERNMENT• Electedofficials• Civil servants• Reformers• LawenforcementInfluential ActorsRather than breaking the governance dataecosystem along traditional ‘supply’ and ‘demand’divides, it is more useful to categorize actorsbased on their degree of influence over targetoutcomes. Influential governance data actors tendto come from specialized or niche groups, andvary across a number of dimensions.currently in common use. A more granularunderstanding of actor categories willallow for more effective engagement. Thequestion then is not, “How do we reach‘users’?” but rather, “Of all the possible actorsthat may use governance data, who has themost influence over decisions on this issueand how are they exercising that influence?How can we source, package, and share datato encourage them to fit governance datainto their existing work?”CIVIL SOCIETY MEDIA PRIVATE SECTOR• Advocacy• Research• Data &technology• Investigativejournalists• Data journalists• Corporations• SMEs• Forensicauditing firmsMSI respondents generally spokeconfidently about the theoretical andtechnical dimensions of governance dataand with greater hesitation on operationalpaths toward impact. (The politicalimplications of data production and usagewere touched on briefly.) With few exceptions,respondents had less substantiveengagements with governments—the actorsthey sought to influence (i.e. as opposed towith civil society)—which contributed toa limited understanding of their interestsand capacities. This suggests a disconnectbetween how governance ecosystems andprocesses are structured and operate, andhow many governance data actors understandthem.Although the ‘governance data community’is loosely defined, members fromgovernment tend to represent a minority.Of those from government, anecdotalevidence suggests that many are relativelynew to the public sector, reform-oriented,and, as a result, often on the peripheries ofpower and decision-making. Civil societyactors, while open to the idea of engagingwith government, lack systematic processesto test their assumptions and to developempirically grounded, nuanced understandingsof how government works.Peixoto (2013) notes that any accountabilitymechanism built on disclosure principlesrequires a minimal chain of events:1. Governmental information is disclosed;2. The disclosed information reaches itsintended public;3. Members of the public are able toprocess the disclosed information andreact to it; and4. Public officials respond to the public’sreaction or are sanctioned by the publicthrough institutional means. 4Though greatly simplified, this basic modelof accountability highlights the limits ofcurrent approaches to governance data,which emphasize the first two steps in thechain and largely ignore those after. Aspreviously noted, the governance datacommunity struggles to identify actors withthe interest and capacity to use discloseddata to effect change (Step 3). While thecommunity is actively grappling withthis question, current efforts do notadequately account for the complexity ofgovernance processes—and specifically,of how governments operate. As a result,they apply linear strategies to dynamicenvironments and are unable to elicitthe desired responses from government(Step 4). Challenges in identifying and targetingactors that have influence withingovernment frustrates the success of4 Peixoto, Tiago. “The Uncertain Relationship Between Open Dataand Accountability: A Response to Yu and Robinson’sThe New Ambiguity of “Open Government”. University of LosAngeles Law Review 2013.6

Motivation to influence government decisionsGovernance Actors & Their InfluenceTo what degree are actors motivated and/or incentivizedto increase the social accountability of public health policy?HIGH MOTIVATIONThe influence of different actors on governmentdecisions will invariably change based onthe specific issue and context; as such, thereare various ways of understanding relativeinfluence.Motivation and capacity are two criticaldimensions in determining influence. Thosewho are highly interested in affecting specificgovernment decisions (x-axis) and who haveextensive resources and know-how in doing so(y-axis) are more influential.For governance data to be impactful, thisframework suggests targeting actors thatare highly motivated and highly capable. Italso suggests that an actor’s influence can beincreased by growing its interest in influencinga particular issue (moving it rightward on thex-axis), and/or by enhancing its ability to do so(moving it upward on the y-axis).To illustrate this point, consider the followingscenario: The diagram on the following pageis a hypothetical map of different actors’ influenceon public health policy. In this example, acertain think tank is influential on public healthissues. It lacks, however, a policy agenda foruniversal health coverage; in turn, it does notwant to, and is unable to, offer a cohesive orcompelling position on the topic.Under the country’s new open data decree, theOffice of the President has asked the Ministryof Health to release its data. To derive valuefrom open data, the government benefits fromunderstanding what datasets and analysisexternal experts need.For example, this think tank lacked data onhistorical national spending on health insuranceand outcomes. Now that it has this data,it compares national results with World Bankdata from other contexts, including those withuniversal health coverage. This leads the thinktank to formulate a policy position and motivatesit to lobby government on this issue.Once motivated, the think tank uses the evidenceto effectively advocate for its position.To increase its capability in doing so, it needsto understand which actors inside governmenthave the mandate and/or authority to influencepolicy on this issue, which elements of itsposition will resonate with these actors, andhow best to convince them of its argument.This understanding will enable the think tankto design and execute persuasive lobbyingefforts.While this example is theoretical and highlysimplified, it helps to illustrate the value ofmapping actors’ influence. It shows that optimal‘data users’ differ by issue and context,and that there are multiple pathways to achievingimpact in each scenario. It also highlightsopportunities for building the motivations andcapabilities of influential data users as ways toaffecting change in government, and proposesusing deep research to identify mechanisms todoing so.ADACResearch InstitutionsMotivations: To assess public sector programs and offer policyMotivations: To assess public sector programs and offer policyrecommendations for reform.recommendations for reform.Capabilities: Considered reputable, and are sought out forCapabilities: Considered reputable, and are sought out forpolicy guidance. May not have in-house expertise to advocatepolicy guidance. May not have in-house expertise to advocatefor research beyond its creation.for research beyond its creation.BGovernment ReformersMotivations: To improve public sector effectiveness andMotivations: To improve public sector effectiveness andefficiency.efficiency.Capabilities: Have energy and enthusiasm to push forCapabilities: Have energy and enthusiasm to push forchange from within. Bureaucratic procedures and policieschange from within. Bureaucratic procedures and policiesimpede their advances.impede their advances.CDBHIGH CAPACITYInvestigative JournalistsCapacity to influencegovernment decisionsTo what extent do actors have thecapability and resources to increasethe social accountability of publichealth policy?Motivations: To surface information that impacts the public goodMotivations: To surface information that impacts the public goodCapabilities: Are experienced in the process of creating changeCapabilities: Are experienced in the process of creating changethrough information transparency. Have limited human andthrough information transparency. Have limited human andfinancial resources.financial resources.CorporationsMotivations: To lobby for public policies that benefit industry.Motivations: To lobby for public policies that benefit industry.Capabilities: Have access to a high level of resources. HaveCapabilities: Have access to high level of resources. Haveestablished government relationships and extensive experienceestablished government relationships and extensive experiencein lobbying.in lobbying.7

accountability mechanisms that use governancedata. Current methods of user identificationare ad-hoc in nature and limited inscope. Some organizations search for mediaattributions manually or by using a webcrawler, usually at irregular intervals. Somerely on word-of-mouth. Others collect userdata through mandatory online registrationprocesses; to date, no significant analysisof such data has been conducted. Beyondjournalists—whose usage of governancedata is sometimes publicly manifest intheir work—no organization interviewedmonitors any other user group in asystematic way; some acknowledged this asa constraint to understanding their users.The governance data community maybenefit form more sophisticated approachesto understanding users. There is a growingrecognition that approaches aren’t working.Some CSOs are becoming more proactivein doing so. Methods used, however, remainlargely limited in scope and depth, and tendto target known users. But there are moreusers then those currently identified.The lack of user understanding has ledsome to the notion that both consumingand producing data is the most effectivepath to impact. Reflecting on a rich careerin budget monitoring, one governance dataexpert contended that successes largelyresult from marrying data productionwith consumption. Actors that are mostimpactful, he said, play roles on both sidesof the data coin: “There should be overlapbetween producers and consumers. Theidea of useful data is to create a chain ofdata consumers who become producers,and producers who become consumers.”Indeed, a data producer that consumesinformation is more likely to put out usefuldata and data products because it has agrounded, firsthand understanding of userneeds and constraints—its own. Using datathemselves helps actors to select, package,and present data to be most useful for peerusers. But one need not have to be a userto understand users—design research cansurface the same information, while mitigatinga producer’s bias toward ‘users likeme.’ Any one actor’s influence on governmentdecision-making will be limited; assuch, the outputs of self-referential designprocesses will also be limited in utility.The governance data community canbenefit from new approaches to strategicplanning and program design. Reflectingon an eight-country review of opengovernment initiatives published in 2013,Khagram, de Renzio, and Fung discuss theanalytical process required to determinehow and why impact occurs: “The conditionsthat are likely to be associated withmore open government [...] can initially beclassified into three broad categories: political,economic, and cultural or historical. Inaddition, the interactions among these conditionsneed to be analyzed in relation toCase Study: User Research on Procurement DataLeading governance data actors recognize the need to better understand their users andsome are experimenting with different methods of user research. An anti-corruption CSO inEastern Europe is one example.To better understand who is using its tools, this CSO ran a short survey on its website, andconducted follow-up interviews with a subset of respondents. The research focused on threekey open data tools: i) an open contracts platform where municipal contract data scraped fromgovernment websites is released in user-friendly and searchable formats; ii) is a combination ofa public procurement site and company register that acts a search engine for corporate tycoons;and iii) is a database of structured government datasets.Through research, the CSO found that journalists were using its tools much more than expected—typically on a daily basis as a part of a standard routine. More experienced investigative journalists,however, were not because they tended to be older and were not skilled technology users.The study also surfaced users that the CSO did not previously know about, including forensicauditors and law enforcement, who it believes are driving significant change efforts. For example,forensic auditors are believed to study data on European Union funding and present evidence incourt; this impacts how these funds are allocated at a national level.Based on this research, the CSO has begun strategizing how to reach older influential journalists—and other offline users—and help them use its products. To help integrate its tool into an offlineworkflow, the CSO is considering the development of a short set of instructions that can be physicallyposted on desks to direct these users to its tool, and to guide them through its functionality.The CSO is in the process of further analyzing research results to inform other process and tooladjustments.The CSO’s experience demonstrates how even targeted, small investments in user research cansurface useful insights to inform strategy and product development.country contexts, taking into account theactors involved, their potentially conflictinginterests, power, and capabilities, theinstitutions that shape their behavior, andthe incentives that such institutions create.This, in turn, will help to explain how andwhy specific outcomes occurred (or not).” 5To achieve impact, a similarly analyticaldesign process is needed, one that isgrounded in politics and accounts for thecomplexity of governance processes andecosystems.5 Khagram, Sanjeev; Fung, Archon; De Renzio, Paolo. OpenBudgets: The Political Economy of Transparency, Participation,and Accountability. Washington DC: Brookings Institution Press.April 5, 20138

Donors play a central role in ensuring that efforts seeking to improvegovernment accountability are impactful. This study surfaced severalways in which donors may be able to support governance dataactors in.• Developing products that are more responsive to user needsand political opportunities; and, as a result, are more likely toachieve greater adoption and impact.RECOMMENDATIONSFOR DONORSThis research study surfaced several ways in which donorscan support governance data actors toward realizinggreater impact; many of the recommendations may alsoapply to broader transparency and accountability initiatives.• Encouraging interactions and coalitions across traditionalstakeholder divides to build the understanding that is critical forconstructive collaboration.• Improving learning practices across the community to betteridentify and share what works and doesn’t, and to help integratethem into operational practice.Summary of Recommendations1. Discourage a ‘build it and they will come’ mentality2. Encourage systematic, scalable approaches to learning aboutusers3. Reduce the risk of “I am the user” syndrome4. Develop strong social design capacities5. Facilitate user peer exchange to grow the agency and influenceof the data user community6. Encourage cross-sector empathy building to strengthen learningprocesses and outcomes7. Encourage grantees to develop learning agendas based on howtarget communities of practice actually operate8. Set specific indicators and targets for tracking a grantee’s contributionsto broader community learning and process change9. Steer learning efforts toward community-wide process change(beyond organizational learning and peer exchange)9

Develop products that are moreresponsive to user needs and politicalopportunities1. Discourage a ‘build it and they willcome’ mentalityCivic technologists are becoming increasinglyskilled at assessing user needs andcreating tools to address these needs.However, many continue to assume oranticipate the needs of broadly defineduser groups and lack formal processes totest their assumptions. As a result, manytools produced struggle to achieve broaderusage and adoption. Resources are investedin their creation and then on incrementaltechnical improvements, but with littleevidence of how they influence governmentdecision-making. To discourage the ‘build itand they will come’ approach, donors can:Tie grantee support to workplansthat prioritize users early and often.Donors can motivate grantees to allocateresources to foundational research anddesign through contracting mechanismsthat require it periodically. A commonbehavior of grantees who build technologicaltools is to recognize the importance ofuser research, but prioritize other technicaltasks. It is critical for rigorous user researchto precede and inform tool design, anddonors can stage-gate support to promptthese activities.Encourage grantees working on earlyprototypes or nascent ideas to defineand test their use-cases before toolsare fully built. Risk aversion often leadsdonors toward grantees who are comfortablein demonstrating the functionality oftheir tool. While these potential granteesmay make the most convincing pitch, theymay also be too far into development tointegrate new lessons about users or specificuse-cases. Early designs and open questionsdon’t necessarily mean high risk. Instead,donors can see them as opportunities toreduce medium- and long-term risk byusing design research to determine preciseuse cases.2. Encourage systematic, scalableapproaches to learning about usersGovernance data initiatives often identifyand learn about users through passive,ad-hoc methods. Some limit their scopeof inquiry to a subset of presumed users.Many rely on feedback mechanisms thatare informal and therefore difficult tostrengthen or scale. In this way, initiativeswill continue to understand data usethrough a narrow and potentiallymisleading lens. To support broaderthinking and new practices, donors can:Encourage grantees to conductfoundational research and development(R&D). The social sector oftenexpects foundational R&D to be completedby the time a group is seeking funding. Thefunding models for many social organizationsthen makes it difficult for them toinvest in strong R&D; as such, proposalsare often filled with untested assumptions.Once funded, grantees are then expectedto execute against their proposal and oftenlack the incentives, resources, and processesto look beyond or pursue alternativesto their early thinking. (By contrast, theprivate sector tends to make significantinvestments in R&D at the start of aproduct’s development process, and thencontinuously over its lifecycle.) Donorscan fund early and deep R&D work toensure initiatives are grounded in (andexecuting against) an empiricallyderived understanding of users.Help grantees select and sequenceresearch, design, and testingapproaches, and ensure they arebeing deployed under a cohesivestrategy. Qualitative, macro-view research(e.g. focused on the landscape in which theintervention will operate) is most useful atthe start of a project to assess opportunitiesand set direction. It can also be usedto refresh or redirect strategy. Conversely,quantitative, micro-view research (e.g.focused on the mechanisms of and10

interactions with the intervention) is moresuitable at a later stage to select executionoptions or refine approaches. Userfeedback, whether qualitative or quantitative,is best when continuously collectedand integrated into ongoing implementation.Donors can help grantees determinewhich approaches are most appropriateat which stage of their process, as well aslearn how to both build successful feedbackmechanisms and integrate inputs toongoing program adjustments3. Reduce the risk of “I am the user”syndromeThe governance data community recognizesthe value of a user-oriented approach tostrategy and design. As a result, some hirepresumed data users into their teams. Thismodel, however, risks supplanting formative(and often expansive) design research withdirection from a single user group. The“I am the user” syndrome may bias worktoward needs derived from a limited set ofexperiences. To reduce this risk,donors can:Encourage diversity in grantee staffingof and interaction with data users. Themore user perspectives represented in anorganization, the more likely it is to applya user experience lens to its work. Donors’wide networks put them in strong positionsto identify appropriate user candidates forhire. While hired users should contributeto designing interactions with data and dataproducts, they should not overly influencedirection-setting around priority users anduse cases. Donors can help identify andintroduce a wider array of potential datausers to expand grantees’ fields of vision.Set targets for grantees related toidentifying a broad and precise setof user needs. Donors can incentivizework through goal setting; ‘depth of designresearch’ can be one useful category fortarget-setting. Ways to assess the qualityand depth of design and user research workincludes: demonstrated understanding oftarget issue’s historical context; depth ofpolitical economy analysis; complexity ofrelationships surfaced between influentialactors; sample size of potential usersengaged; range of user needs surfaced;and ability to map a coherent path toinfluencing target outcomes that engagesmore than one actor group.4. Develop strong social designcapacitiesGovernance data actors are interested inuser-centered design, but lack appropriateguidance on how to execute such a process.This is unsurprising given that design practicesfrom the corporate sector are ill-suitedfor the social sector; for example, robustpolitical analysis is rare or wholly absent.To enable appropriate adaption andapplication of design practices, donors can:Provide grantees with operationalguidance and resources—not justgeneral principles—on how to executedesign approaches in the social sector.For grantees working on issues of transparencyand accountability, a politicallygrounded approach is critical for designingappropriate interventions. Traditionaldesign approaches—often derived fromcorporate product or service design—donot account for complex governanceecosystems and processes and, as such, willfrustrate them. Capacity building for implementingdesign practices requires ‘learningby doing’; donors can help grantees by connectingthem to external experts in theseapproaches to train them and to shepherdhands-on learning processes.11

Encourage interactions & coalitionsacross traditional stakeholder divides5. Facilitate user peer exchange togrow the agency and influence ofthe data user communityWhile many data actors have bandedtogether around shared challenges andapproaches to work, current and potentialusers of governance data—who may beoperating in isolated contexts, or arecompeting over the same limited set offinancial resources—lack this cohesion.The result is a community of actors workingin similar ways around similar issues, butwith little coordination and hence, at times,redundant efforts. Moreover, influencinginternationally defined data agendas isdifficult withouta unified front. Donors can help to addressthese challenges in the following ways:Develop inter-donor knowledgesharing platforms to discuss andsupport coordination mechanisms fordata user grantees. Many donors are supportinggovernance data users working indifferent countries, across issues areas, andwith varying levels and types of datasets.Sharing information around grantees’ workcan help determine which of them mightbenefit from informal introductions to oneanother, or from exploring more formalpartnerships.Support grantees in translatingvirtual, broad-based stakeholdercommunities into intentionallyinteractive user networks. Many loosenetworks of governance data stakeholders,such as online communities of practice,operate without much guidance. They canbe disentangled and appropriate subsetscan be intentionally formed. One way ofdoing this is to bring together user groupswho are participating in established onlinecommunities and have them co-design newways of growing their influence. Donors cansupport this through asking appropriatelyresourced grantees to coordinate fora toadvance these objectives.6. Encourage cross-sector empathybuilding to strengthen learningprocesses and outcomes‘Othering’ is a natural human tendency,and stakeholders involved in governanceprocesses are no different. Many that sitoutside of government see governmentactors primarily as targets of advocacy andnot also as potential allies. Those in governmentview outside parties as actors pushingfor change with limited understanding ofinstitutional constraints or reform processes.This can prevent governance data actorsfrom constructively engaging with oneanother.Efforts aimed at increasing governmentaccountability are often led by actors that,paradoxically, are reluctant to engagewith government. Lack of engagementacross sectors can result from limitedunderstandings of the context in whichan unfamiliar counterpart operates. Thereare few incentives to encourage crosspollinationof ideas or ways of workingacross sectors, but donors can helpgrantees establish more constructiveways of learning about, and workingwith, different counterparts:Facilitate cross-sector understandingthrough immersive research andlearning methods. To design a compellingpitch to, or realistic ask of, government,governance data actors need to firstunderstand the interests, processes, andcapacities of the institutions they seek toinfluence. There are several ways of doingso. Some include institutional ethnographyand design research done by embedding ingovernment offices, or fellowship programsthat bring civil society leaders intogovernment for a time-bound postingor assignment. Two examples of thelatter approach are Code for Americain the United States and the Agentes deInnovación fellowship in Mexico. Deeperresearch or direct experience withgovernment can help civil society actorsbetter understand how government works.This will help them target the right actorsand processes, and develop compellingmessaging and proposals that appeal totheir counterparts’ interests and aremindful of their constraints.Offer evidence on the concretebenefits of cross-sector collaboration.Multi-stakeholder collaboration soundsgood in theory, and many are willing to try.But efforts are often abandoned once actorsexperience the pain of doing so without aclear sense of the payoff. After all, workingwith diverse actors with differing mandatesand interests can be a time-consuming,long, and challenging process.Donors can help grantees understandthe enablers and potential outcomes ofsuccessful multi-stakeholder efforts.They can do this by commissioning a reviewof successful past efforts or deep processresearch into ongoing efforts. The formerhelps build a body of evidence to encourageactors to collaborate, while the lattercan help provide insights on how to do so(e.g. specific protocols and mechanisms forpartnership).12

Improve learning practicesacross the community7. Encourage grantees to developlearning agendas based on howtarget communities of practiceactually operate“Contributing to learning” can take manyforms and organizations often go about itin ways that are familiar to them. Commonoutputs include: online repositories of casestudies, academic white papers shared at‘brown bag lunches’ or more formal ‘launch’events, and webinars to present findingsto an interested group of viewers. Theseapproaches are the norm for most organizations,but they may overlook actors thatare influential in their respective operatingcontexts who are not engaged in thesecommunities of practice.To address these challenges, donors canencourage grantees to first pinpointthe sources and channels their targetaudiences use to get inspiration,design projects, and troubleshootimplementations. Then, they caninterview audiences to understand whatmakes these channels appealing—is it theselection of topics, the editorial tone, thefrequency of new content, or other factors?From there, they can develop learningproducts and outreach strategies based onaudience preferences, not organizationalhabit.8. Set specific indicators and targetsfor tracking a grantee’s contributionsto broader community learning andprocess change.Although it will be difficult to determinecorrelation and causation between agrantee’s activities and wider systemimpact, setting targets and trackingprogress will help grantees prioritizeotherwise ‘fuzzy’ learning activities.While indicators will, in most instances,track outputs instead of outcomes, donorscan invest resources in analyzinglongitudinal changes in thought andpractice among target communities ofpractice. This measurement exercise cantake the form of a process evaluation whereincremental steps are defined and used forreflection at multiple points over time.Take, for example, an online communityof practice. Donors can specify ways ofmeasuring ‘community’ activity—such asthrough relationships formed and offlineengagements spurred between online participants.This will help to drive the initiativeto tie immediate outputs (e.g. platformand social media creation)to long-term behavior change.9. Steer learning efforts towardcommunity-wide process change(beyond organizational learningand peer exchange)Often, learning for organizations isconfined to internal parties or for exchangewith peer groups who are familiar (at times,only because they are geographically convenient).There are challenges to influencingbeyond one’s known sphere of influenceincluding the heavier investment needed toform new relationships, and the trade-offsthat arise when allocating limited resources.Credibility in the eyes of the wider communityof practice requires an ability to communicatehow one has changed the statusquo, but this takes time and incrementalsteps. Donors can help grantees changebroader processes in the following ways:Target support toward learningactivities and deliverables in scopesof work. Doing so would be especiallyimportant in issue areas where organizationsseem to be particularly siloed, thereare few other incentives or channels forknowledge exchange, and/or where lessonsfrom a specific organization would greatlybenefit a larger community of practice. Incommunities studied, learning efforts arecentral to their agendas, however, they lackincentives to share concrete learning withthe broader community in a significant way.The rewards of doing so are ambiguous,whereas the cost is clear: precious time13

LOOKING FORWARDdiverted from activities (e.g. fundraising)with more immediate, direct benefits.Donors have an opportunity to prioritizeeffective learning by supporting these activitiesin current and core strategic granteeworkplans.Introduce new ways of directingprecision in understanding processchange. Quantitative metrics and targetsare often used to assess process change.For example, to understand the success ofcommunity building efforts, donors mayselect indicators relating to volume of socialmedia engagement, of website visitors, orof case studies downloaded. Qualitativemetrics, such as participant responses on amulti-stakeholder workshop, are also used.While these metrics are easy for granteesto report on and donors to review, theyprovide limited insight into whether andhow different activities are contributing tolarger process change.Donors can encourage greaterambition in communicationsactivities armed at enabling widerprocess change. They can start, forexample, by assessing, whether granteesprioritize audiences because they are an‘easy fit’, or because they understand userbehaviors and considerations of greatestimpact. A grantee might target its webinarat the global open government community,a seemingly natural audience for its work.However, donors can ask for more granularityin audience selection based on thegrantee’s larger impact goals. For example,a government ministry that reserves abi-weekly timeslot to discuss capacitybuilding is more likely to integrate newlearning into its workflow. If learning is todrive impact, not just stimulate conversation,this ministry may be the most suitabletarget audience.The study highlighted the need for governance data communitiesto better understand the influential actors and political contexts theyseek to influence. Doing so may require wider adoption of strategicplanning and program design approaches that appropriately accountfor the complexity of governance ecosystems and processes.There is interest and appetite from governance data actors to buildupon the strong work that they are currently doing, and to try andtest new design approaches. Mechanisms for doing so includedemonstration projects that support organizations in applyinguser-centered and politically grounded design approaches inproduct development and implementation.Implementing such projects can happen through eager, influentialmulti-stakeholder initiatives that are invested in broader sectorlearning. This will help to ensure lessons from the demonstrationprojects are shared among wider communities of practices.14