Evaluation of the Australian NGO Cooperation Program (ANCP)

ODE BRIEFAugust 2015Evaluation of the AustralianNGO Cooperation Program (ANCP)The ANCP is an annual grants program that providesmatched funding to accredited Australian NGOs tosupport their work in developing countries. Fundingsupports projects across a range of sectors includingeducation, health, water and sanitation, governance andeconomic development. Established in 1974 the ANCP isthe Department of Foreign Affairs and Trade’s (DFAT’s)longest running NGO program. It also represents thelargest program for Australian NGOs, with a 2014–15allocation of $134 million, constituting 2.7 percent ofAustralia’s Official Development Assistance budget*,and approximately one fifth of all NGO funding. As alarge global program ANCP activities span multiplecountries and regions with individual projects numberingin the hundreds.Assessing the impact of the ANCP on individualsand communities in developing countries across theglobe was beyond the scope of the evaluation. Rather,the Office of Development Effectiveness set out toassess the effectiveness of the ANCP as a mechanismto assist NGOs in reducing poverty and supportingsustainable development.Overall findingThe evaluation confirmed that the ANCP is a successfuland highly valued program. It identifies positive featureswhich might usefully inform the Australian Government’sapproach to a number of other development programsand partnerships. However it also highlights areas forimprovement, such as the need to address the complexityand limited transparency associated with fundingallocations and the opportunities for greater sharing ofknowledge and learning across the partnership.Vomboe Molly, World Vision Vanuatu Area Program Manager for Sanma, speaking with the village Chief on Malo Island.Photo Credit: DFAT/Julie Hart* The 2015-16 budget for ANCP is $127 million, representing more than 3% of Australia’s official development assistance.ODE BRIEF: Evaluation of the Australian NGO Cooperation Program (ANCP)www.ode.dfat.gov.au1

Evaluation findings and recommendationsThe ANCP offers flexibility and extends the reachof the Australian aid programThe ANCP is a flexible program which remains relevantto the Australian Government’s aid and foreign policypriorities, including those relating to economic diplomacyand the private sector. The evaluation found a highdegree of alignment between ANCP interventions and theAustralian aid program’s strategic goals. For example,77 per cent of ANCP funding is currently dedicated to theIndo-Pacific region. This alignment occurs without explicitdirection from DFAT.The ANCP delivers strongly on results; in 2013–14ANCP represented around 2.7 per cent of the aidbudget and delivered 18.2 per cent of outputs reportedin the Department’s aggregate development results.Furthermore the ANCP extends the reach of Australianaid to beneficiaries beyond the footprint of DFAT’sbilateral aid programs. It enables NGOs to delivereffective development activities from the grass rootsthrough to the institutional and policy levels.Whilst the ANCP represents a small percentage of theAustralian Government’s total aid expenditure, its reachand scale present many benefits to DFAT. These includemaintenance of residual capacity to scale up or respondto future shifts in the aid program’s sectoral or geographicfocus. Any attempt to narrow the program to currentpriority areas risks undermining ANCP’s flexibility, theprinciple of respect for NGOs’ organisational autonomy,and the benefits that flow from these key features.Accreditation is valuable and brings a range of benefitsThe accreditation process provides an efficient means ofselecting capable NGOs with well-established systemsand processes. It also provides a number of important,indirect benefits. Whilst accreditation is time andresource intensive, ANCP NGOs were unanimously of theview that the process strengthened their organisationalcapacity and was worthwhile, not least because it thenensured a minimum level of annual funding. WithinDFAT, ANCP accreditation is regarded as an indicator oforganisational sophistication and superior performanceand a number of other programs use ANCP accreditationstatus in place of a separate due diligence processeswhen selecting NGO partners.The evaluation found that ANCP funding bolstersthe credibility of member agencies and is helpfulin leveraging other funding within Australia andinternationally, from other parts of their organisations,from the private sector and from other donors. Asignificant factor in the ability of NGOS to leverage fromother sources is their accreditation status.Percent of total DFAT reportedaggregate development results 2013–14Reported by ANCPCSOs supported to track service provisionIncreased incomesIncreased access to financial servicesAccess to agricultural technologiesWomen survivors of violence receivingservices such as counsellingDisability services like prostheseand assistive devicesWASH management committees in whichwomen are equally representedIncreased knowledges of hygiene practicesIncreased access to safe waterIncreased access to basic sanitation0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%ODE BRIEF: Evaluation of the Australian NGO Cooperation Program (ANCP)www.ode.dfat.gov.au2

The funding allocation model is not adaptable tochanging circumstancesTo be eligible for ANCP funding NGOs must demonstratefinancial backing from the Australian community. Theuse of Recognised Development Expenditure (RDE) asboth a measure of support from the public and as abasis for the allocation of funds across agencies is along standing and well accepted ANCP practice. Theevaluation found that while there is general consistencybetween RDE levels and funding allocations somesignificant discrepancies also exist. The current fundingarrangements and calculations have been adequateto date, but this has been within the context of anexpanding program. The introduction of new NGOs intoANCP, a decline in funding, or a combination of bothpresents a challenge to DFAT and the NGOs alike.In addition the value of the current fund structure withtwo accreditation levels (Base and Full) and threefunding tiers (Base, Full and Partner) is unclear and thelanguage used to distinguish between the different tiersis unhelpful. Base and Full NGOs expressed frustrationat the perceived lack of transparency surroundingarrangements with the ten Partner NGOs.Recommendation: That DFAT explain the relationshipbetween accreditation levels (currently two) and fundingtiers (currently three) and detail how NGOs qualify for, andprogress through, these. Any difference in obligation orbenefit associated with each tier or level should be clearlyarticulated. DFAT should also ensure that the principle ofpartnership applies across all ANCP members and thatthis is reflected in a suitable naming convention.Recommendation: That DFAT revise the currentapproach to allocating funds across ANCP memberagencies with a view to formalising fundingarrangements through a transparent funding allocationpolicy. The policy should retain key elements of theestablished model, such as the use of recogniseddevelopment expenditure, but also enable DFATperformance assessments to impact on allocations.A scalable approach is required to ensure adaptabilityto changes in budget or other circumstances.Learning within the program is not optimisedDFAT, in conjunction with ANCP NGOs, has developedrobust and appropriate monitoring and reportingprocesses for ANCP. Like many other aid programs,monitoring and evaluation systems are focused ondesired results at the output level. Potential exists fora deeper assessment of results at the outcome levelor, put another way, what the ANCP has achieved asa whole. There is also room for improving knowledgeSummary of ANCP strengths and weaknessesPredictable/annual fundingFlexiblefundingAccreditationRecognisedDevelopmentExpenditure(RDE)MonitoringEvaluation andLearningFramework (MELF)No policy basis in fundingpredicibility (ie fundingcalculated annually)Risks associated withunexpected budget cuts(long term planningand relationships)Enables long termprogramming andstrategic approachesBuilds sustainablepartnerships withlocal organisationsRespects NGOautonomy for tacklingdevelopment issuesComplements othersources of fundingFacilitates innovationIncreases reach(Geographies, sectorsand communities)Benefits outweighcosts of accreditationImproves NGO systems,processes and policiesProvides credibility(”quality stamp”)Not an assessment ofthe impact of NGO’sdevelopment activitiesImpose a resourceintensive burden onsmaller agenciesAmbiguity aroundstandards/criteria to gainFull or Partner NGO statusMeasures supportfrom the Australian publicfor foreign aidContributes to cooperationand collaborationCo-funding limitsdependence ongovernment supportInsufficient transparencyaround fundingallocation(RDE only one factor)Insufficient clarity aboutthe benefits andresponsibilities ofPartner NGOsImproves NGO M&Eand reporting practisesPromotes learning inthematic areasProvides AggregatedDevelopment Resultsfor ANCPMELF potential forlearning is not fullyutilisedContributes to capacitydevelopment (NGOsand local partners)Helps to leverage(Financial and relationships)ANCP FeaturesStrengths/benefitsWeaknessODE BRIEF: Evaluation of the Australian NGO Cooperation Program (ANCP)www.ode.dfat.gov.au3

sharing and learning within the ANCP. The evaluationnotes the lead role of the Australian Council forInternational Development (ACFID) and ANCP PartnerAgency Collaboration (APAC) group of NGOs in this area.The focus on cross-cutting themes such as disability,child protection and gender (particularly as part of theaccreditation process) distinguishes ANCP from otherfunding models and has a positive effect on the NGOsector as a whole. ANCP has elevated the profile of thesethemes amongst in-country partner organisations withpotentially far-reaching effects.While the ANCP reporting framework does notnecessarily suit all NGOs it has provided a systematicapproach to reporting across the program, includingstandard indicators. There is a general consensus thatthe Monitoring, Evaluation and Learning Framework(MELF) enhances the effectiveness of NGOs’ workand has led to continuous improvement, some sharedlearning and improved reporting from field basedpartners. The MELF effectively captures the outputsof ANCP funding but not development outcomes. Itwill be important to ensure the forthcoming theory ofchange, currently being formulated by ANCP partners,provides sufficiently detailed outcomes to enablebetter performance assessment and communication ofachievement against these outcomes.Recommendation: That DFAT build upon the ANCPmonitoring, evaluation and learning framework(MELF) in order to strengthen the role of qualitative,quantitative and geographic data in generatingevidence for learning, policy and program improvement.This should include;a. development of a performance assessmentframework, based upon the forthcoming ANCPtheory of change, to aid reporting of outcomes at aprogram level;b. clear links to the Australian aid program’s high-leveltargets and other performance reporting processes;andc. introduction of a system of independent review andvalidation of the performance management andresults reporting systems used by the larger ANCPmembers.The MELF provides a lot of information but not all of thisis being used optimally. There is also limited learning,policy and program development from the MELF. Inparticular the thematic reviews are a rich source ofevidence and an excellent foundation for further learningacross the sector. There is an appetite for better use ofinformation among ANCP NGOs, and recognition that thiscould also benefit DFAT.Many recognise that useful information is beingproduced, however, it is also recognised that learningcould be better facilitated. One possibility is for ACFIDto play a greater role in supporting learning events orproviding online access to resources.Recommendation: That DFAT, ACFID and the ANCPNGOs commit to testing new approaches to improve thesharing of lessons between Australian and local NGOsand DFAT aid staff. This could be largely undertakenwithin existing resources, harnessing opportunities tobring people together through learning events and usingavailable technology to make existing evaluations andother studies more readily available. The ANCP theoryof change could be used to help define a focussedlearning agenda.Engagement across DFAT is variableThe level of engagement across DFAT with the ANCPis highly variable and understanding of ANCP rolesand responsibilities is often dependent on theindividuals involved. DFAT NGOs and Volunteers Branchacknowledged that the ability to clearly articulate keyareas of mutual benefit between DFAT and ANCP NGOscould increase the value of the ANCP partnershipssubstantially.The forthcoming ANCP theory of change will for the firsttime articulate the overall aims of the ANCP and thusplay a major role in helping to address some of themisunderstanding that exists around the program.Recommendation: Whilst maintaining the role of NGOand Volunteers branch in managing the program, DFATshould clarify the role of Posts in the ANCP with aview to establishing a consistent and minimum level ofresourcing and engagement between DFAT Posts andANCP members in-country.Office of Development EffectivenessThe Office of Development Effectiveness (ODE) is anindependent branch within the Department of ForeignAffairs and Trade (DFAT). ODE monitors the Australianaid program’s performance, evaluates its impact, andcontributes to international evidence and debateabout aid and development effectiveness. ODE’s workis overseen by the Independent Evaluation Committee(IEC), an advisory body that provides independentexpert advice on ODE’s evaluation strategy, work plan,analysis and reports.The full evaluation report and DFAT managementresponse can be accessed at www.ode.dfat.gov.au.ODE BRIEF: Evaluation of the Australian NGO Cooperation Program (ANCP)www.ode.dfat.gov.au