ANCP Philippines Cluster Evaluation Report - AusAID

More documents

Recommendations

Info

ANCPFindings3. FINDINGSThis cluster evaluation represents the fifth ANCP evaluation since 2005 using thecurrent cluster evaluation framework 18 . It may now be timely for AusAID to engagewith the ANGO sector to conduct the first of the planned ‘meta evaluations’. Whenthe cluster evaluation framework was designed it was anticipated that there would beperiodic synthesis of trends and key issues arising from individual cluster evaluations.This process would be most fruitful if conducted jointly by AusAID and the AustralianCouncil for International Development (ACFID) to ensure ownership of the process bythe ANGO sector.Recommendation1. AusAID should engage with the ANGO sector to conduct the first of the planned ‘metaevaluations’ to synthesise trends and key issues that have arisen from the clusterevaluations.The findings of this cluster evaluation are presented in line with the ANCP clusterevaluation framework (Appendix E). Individual reports for each of the four NGOinitiatives are presented in Appendices A – D. A synthesis of key issues is discussedin the following sections, based on the three performance dimensions that underpinthe ANCP cluster evaluation framework: organisational analysis, developmentstrategy, activity implementation.3.1 Overall RatingsThe evaluation team’s ratings are summarised in Figure 3 and in the followingparagraphs.AgencyIndicator1. Capacity to deliver development responseOrganisationalAnalysisDevelopment StrategyActivity Implementation2. Strategies for ensuring quality partnerships3. Analysis of geopolitical context4. Adequacy of design process5. Standard of funding proposal or design6. Efficiency of activity implementationABM CBM Reledev S.ArmyS GP S SSSSGPSUSUSS S S SS S S S S S USSSSUSS GP GP S7. Capacity for learning and continuousimprovement S S S US8. Effectiveness of development interventionSSSS S S S9. Strategies for sustainabilityFigure 3: Ratings by the evaluation team (HU=highly unsatisfactory; U=unsatisfactory; S=satisfactory;GP=good practice)SGPSUSUSUSS18A total of twenty-two accredited ANGOs have been evaluated in five ANCP cluster evaluations: Cambodia (2005),India (2006), North Asia (2007), Pacific (2007) and Philippines (2008). In addition, modified versions of the clusterevaluation framework have been implemented for Cooperation Agreements in Pakistan (CAER), Africa (APAC) andSolomon Islands (SINCA).ANCP Philippines Cluster Evaluation Report (ver. 2.0) 6
ANCPFindingsAs with earlier cluster evaluations, this evaluation team noted challenges facingANGOs arising from the existence of at least three rating scales. The clusterevaluation framework applies a four-point ordinal scale; the ANGO self-assessmentsapply a five-point ordinal scale; and AusAID is rolling out a six-point ordinal scale forthe purpose of ‘Quality at Implementation’ (QAI) reporting.Recommendation2. AusAID should rationalise the rating scales associated with the ANCP.Three of the four NGO initiatives sampled in this cluster evaluation were found to besatisfactory overall. One NGO partnership was assessed as unsatisfactory overall.Two of the four ANGOs were ‘base’ accredited agencies (Reledev and SAADO); andtended to be rated as weaker overall. This issue generated constructive discussionabout the application of the cluster evaluation framework to both full and baseaccredited agencies. In the past, it has been understood that while base accreditedagencies may be expected to have less developed organisational systems (asassessed through the accreditation process), they should be on a par with fullaccredited agencies in terms of development practice, community engagement andintervention planning. However, the details of this understanding remain ambiguousand should be elaborated.Recommendation3. AusAID should dialogue with the ANGO sector about the performance expectations offull and base ANGOs vis-à-vis the cluster evaluation framework.4. AusAID should provide cluster evaluation teams with guidance on which specificstandards in the cluster evaluation framework (if any) that should be applied differentlyfor base and full accredited ANGOs.Two of the four ANGOs (Reledev and SAADO) were involved in technical vocationaltraining for youth. One of the four (ABM) was improving household access to potablewater; and the fourth (CBM) was improving the capacity of local stakeholders tosupport children with disabilities. Only the ABM-supported project focussed inrural/remote areas; the other three emphasised urban beneficiaries. All four ANGOssampled were faith-based agencies. One of the four in-country partner organisations(The Propeller Club of Manila) was not a development agency 19 .3.2 Organisational AnalysisA range of organisational arrangements were evident within the cluster. Figure 4depicts the diversity against the well established ‘three zones of management’construct 20 . Any single circle/elliptical shape represents a discrete organisationalentity or team 21 . The ‘y’ axis in Figure 4 describes the ‘vertical’ structure within eachof the NGO partnerships. Figure 4 is elaborated below.19The Propeller Club of Manila is a service club loosely associated with the shipping industry. The club has enactedphilanthropic values using ANCP funding via the Salvation Army but is not formally created as a developmentagency.20Sarasohn, H. M. and C. B. Protzman (1948). The fundamentals of industrial management. See also Dransfield, S.B., N. I. Fisher, et al. (1999). “Using statistics and statistical thinking to improve organisational performance.”International Statistical Review 67: 99-150.21The oversimplification implied in Figure 4 is acknowledged, nevertheless, the diagram serves to illustrate thediversity of structural arrangements.ANCP Philippines Cluster Evaluation Report (ver. 2.0) 7
Page 6: ANCPExecutive Summarypartner. None
Page 11 and 12: ANCPList of AcronymsLIST OF ACRONYM
Page 13 and 14: ANCPIntroduction1. INTRODUCTION1.1
Page 15 and 16: ANCPMethodology• ACFID’s NGO Ef
Page 17: ANCPMethodologyA synthesis of overa
Page 21 and 22: ANCPFindingschildren. Both NORFIL a
Page 23 and 24: ANCPFindingsservice providers, resp
Page 25 and 26: ANCPFindingsThe evaluation team for
Page 27 and 28: ANCPFindingsrepresents a complex ba
Page 29 and 30: ANCPAppendix A: ABM-ECPANGOAnglican
Page 31: ANCPAppendix A: ABM-ECPECP has deve
Page 34 and 35: ANCPAppendix A: ABM-ECPrisk. It is
Page 36 and 37: ANCPAppendix A: ABM-ECPout every tw
Page 38 and 39: ANCPAppendix A: ABM-ECPmobilisation
Page 40 and 41: ANCPAppendix B: CBM-NORFILAPPENDIX
Page 42 and 43: ANCPCBR is a social strategy aiming
Page 44 and 45: ANCPcoordination and resource-shari
Page 46 and 47: ANCPclearly affect individuals, fam
Page 48 and 49: ANCPThat Norfil consider carefully
Page 50 and 51: ANCPas an issue, and through the ad
Page 52 and 53: ANCPAppendix C: Reledev-FPTIAPPENDI
Page 54 and 55: ANCPAppendix C: Reledev-FPTItrainin
Page 56 and 57: ANCPAppendix C: Reledev-FPTIinevita
Page 58 and 59: ANCPAppendix C: Reledev-FPTIThe eva
Page 60 and 61: ANCPAppendix C: Reledev-FPTIThe mon
Page 62 and 63: ANCPAppendix C: Reledev-FPTI• ind
Page 64 and 65: ANCPAppendix D: SAADO-Propeller Clu
Page 66 and 67: ANCPAppendix D: SAADO-Propeller Clu
Page 68 and 69:
ANCPAppendix D: SAADO-Propeller Clu
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
ANCP Appendix E: Cluster Evaluation
Page 78 and 79:
ANCPAppendix F: Summary of Intervie
Page 80 and 81:
ANCP Appendix F: Summary of Intervi
Page 82:
ANCP Appendix F: Summary of Intervi
show all

ANCP Philippines Cluster Evaluation Report - AusAID

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?