Closing Remarks - CERIS

More documents

Recommendations

Info

Comparing Accountability and EvaluationAccountability1) Narrower concept with short termhorizons.2) Audit-based; it is about followingproper rules and procedures.3) Focus tends to be on outcomes, it ismore inward looking and has alimited horizon. Accountabilityinvolves practices like counting thenumber of client contacts, considerclient satisfaction scores/measures, andidentify the number of clientsconnected to employers.Evaluation1) Broader concept with longer termhorizons.2) Examines the bigger picture; looks atprograms/policies and their deeperimpacts on communities.3) Evaluation is applied research andconsequently it needs to be both realistic anduseful (Pawson and Tilley 1997); it is aboutevidence-based evaluations at a deeper levelthan happens in the case of simpleaccountability often involving simple countingexercises.4) Because evaluation is more outwardlooking and considers not justimmediate client impacts but broadersocietal effects it is also more complex.Hence, at least in contrast toaccountability, complexity is a primefeature of evaluation.5) Evaluation outcomes are influenced bya much broader range of factors thatstretch beyond the scope and control ofthe program and its deliverer. Thisincludes factors such as the state of theeconomy and labour market; changingmigration patterns and policies; thewarmth of societal welcome, and thelike. These factors contribute to thecomplexity of the evaluation exercise.The rather loose way in which accountability and evaluation have so often been used in practicehas too often placed increased expectations on an already over burdened nonprofit serviceproviding sector. In fulfilling their accountability duties nonprofit providers are often expected toengage in much larger evaluation exercises that have not been properly funded and which areoften beyond their resource and technical capacities to properly perform. Funders in manyinstances are not fully aware of the distinctions between accountability and evaluation exercises,nor often do they fully understand the capacity limitations of the agencies that they fund. Thissituation has resulted in a considerable degree of confusion and muddiness around the role ofaccountability and evaluation practices in the nonprofit/settlement sector that has beencounterproductive. Practitioners and policymakers alike have suffered from confusion around2
these concepts which has frustrated the abilities to achieve good accountability or evaluationpractices and outcomes. There is an urgent need to bring greater conceptual clarity to this issueand for recognition of the capacities of agencies to carry out increasingly complex evaluationexercises in a world of limited resources.Evaluation for Whom?A second issue is that, as in the case of accountability, with evaluation we need to pose the‘evaluation for whom’ question? Nonprofit agencies, as complex civil society organizationsserving a public purpose, have multiple points of responsibility. This includes responsibilities tofunders, boards of management, clients/community, volunteers and donors, staff, among others(Richmond and Shields 2004). Hence, accountability and evaluation should not be to just onepoint of responsibility. Yet settlement agencies and other nonprofit service providers are oftendominated by one dimensional forms of accountability and evaluation to funders (See: Wall2012). Evaluation has the potential to play a very important role in terms of connecting nonprofitservice provision to communities and clients with government ‘public accountability’ forprogram and policy in various areas of concern. However, narrow unidirectional accountabilityand evaluation has restricted the scope and view to the nonprofit agency itself (Richmond andShields 2004). Deeper multidimensional forms of accountability and evaluation that are properlyresources are required. This is also very important for helping to bring greater public oversight to‘government responsibilities’ for public financed programs that are delivered by nonprofits thatoperate at a considerable distanced from governments themselves (Richmond and Shields 2004).Organizational and Political Economy Dimensions of Evaluation in the Nonprofit SectorThe third broad point to be made is that there is always a need to be aware of the organizational,political and socio-economic context in which evaluation takes place (Pawson and Tilley 1997).There are three main points to be made here:a) The information that nonprofit service deliverer, including settlement agencies, has beenprimarily driven by funder ‘demands’/’needs’. This is complicated further by the fact thatprograms are often supported by multiple funders each with their own data requests, oftenrequiring different information forms and file structures. Often data collection is drivenby how money is spent (audit driven) rather than outcome centred which tends to be moreuseful for evaluation purposes.The irony is that most organizations collect lots of data – in fact there is a generalproblem of data overload/data glut – but very often it is the wrong kind of data or it isstructured in such a way that it is not useful for deeper evaluation purposes. Additionally,3
Page 1: “Evaluation Challenges and Settle
Page 5: environment where there is at least

Closing Remarks - CERIS

Create successful ePaper yourself

Delete template?

Save as template?