Measuring performance and effectiveness for MÃƒÂ¤ori ... - Te Puni Kokiri

More documents

Recommendations

Info

16 ‘Sole’ vs. ‘mixed’ Mäori – different shades of Mäori ethnicity There is debate in the literature on the merit of capturing outcome performance within ethnic groups by intra-ethnic variation to get a fuller picture of the complexity of outcomes across and within ethnic groups. In favour of such an analysis, Humpage (2002, pp.180-1) suggests: “Because ‘sole’ Mäori are more likely to be distributed among the deprived deciles than ‘mixed’ Mäori. Yet when the ‘sole’ Mäori category is used alone, figures are underestimated. Ideally, ‘sole; ‘mixed’ and ‘total’ Mäori categories should be combined to show the ‘bigger picture, starting with the simplest (‘total’) and gradually moving on to more complex and detailed (‘sole’ and ‘mixed’) analysis, because reliance on either external or internal indicators can favour one interpretation of the disparity issue… MOH (2001:24) asserted that combining these categories would avoid very simple ‘on average’ conclusions being made in an area where considerable complexity exists”. 3 However, Kukutai (2004) says such an approach is limited. In differentiating needs within the Mäori population, a critical variable is orientation towards the Mäori ethnic group (e.g. the group a person identifies most strongly with). This dynamic cannot be captured in a crude ‘sole’ Mäori versus ‘mixed’ Mäori dichotomy. Durie et al. (2003b, p.100) suggest “the only significant differences lie between Mäori who identify as Mäori (either solely or in combination with other ethnicities) and those who are descended from Mäori but identify as non-Mäori (Kukutaki, 2003)”. Te Puni Kökiri concludes therefore that any analysis involving disaggregation of this nature should be approached with caution. 3.4 COMPARATIVE IMPACT MEASURES OR RESULTS Multiple sources, including official requirements, MSD’s Performance Improvement Framework Review, and writings by Mäori academics, emphasise the importance of impact measures. Impacts are to be directly linked or attributable to the entity’s activities. Examples of impact measures for employment services There is a range of generic impact type measures in MSD’s 20012/15 and 2011/14 SOIs for their employment service Job Search Service and its Appropriation Tailored Sets of Services to help people into work or achieve independence (see Appendix Three). It is also notable that these measures are capable of disaggregation by ethnicity, but such an analysis is not provided. MSD’s Performance Improvement Framework (PIF) review noted difficulties in outcome monitoring and evaluation, and suggested some improvements in that regard (State Services Commission, May 2011b, pp.34-5). However, that review did not address MSD’s performance for Mäori. From an academic perspective, Gill and Dormer (2011, p. 297) suggest Work and Income staff generally perceive that paying benefit entitlements and placing people in jobs is not complex or difficult to measure, but the organisation’s role in social development is. Although “social development outcomes are 3 Sole Mäori are those who are identified in the data as having only Mäori ethnicity, ‘mixed’ Mäori are those identified in the data as having Mäori and at least one other ethnicity), and total Mäori (the total of all those with Mäori ethnicity). T E P U N I K Ö K I R I M E A S U R I N G P E R F O R M A N C E A N D E F F E C T I V E N E S S F O R M Ä O R I : K E Y T H E M E S F R O M T H E L I T E R A T U R E
quantified in the annual Social report, sense is made of this information via subjective evaluation (“gut feeling”) rather than by the application of a formal logic of cause and effect” (Dormer, 2010, p.55). Illustrative examples: potential impact measures for employment services for Mäori The literature identifies several meaningful impact measures for Mäori receiving employment services including measuring: • incremental steps to employment such as addressing health or training barriers and increasing participation and having this reflected in outcome measures and contracting guidelines (MSD, 2003a, 2008a, 2008b, 2008c); • the effectiveness of sequential interventions (Perry, 2007); • whether exiting a benefit marks a move to employment, and if so, the impact of that employment on individual and whänau/family wellbeing (see Englert, 2001; Stolte, 2006; MSD, 2007; Dixon and Crichton, 2007) and long-term employment and income prospects. Best way to measure the impact of an intervention for Mäori is through multiple perspectives Durie and Kingi (2010) emphasise the best way to measure the impact of an intervention for Mäori is through data from multiple perspectives, such as the Mäori client, the service provider, and family/whänau member, with quantified weightings for their responses. By obtaining the perspectives of these three key stakeholders a more balanced impression of outcome could be obtained. It is possible to measure impact status at several points in the service delivery process: at assessment, while in receipt of the service and on exiting the service. 3.5 COMPARATIVE ANALYSIS IN PARTICULAR TYPES OF MEASURES The whole range of performance measures included in figure one (see above, Part 1) lend themselves to comparative measurement. It is worth discussing briefly the literature that relates to measures of coverage and accessibility; customer satisfaction; and the comparative performance of different service delivery sites. Coverage and service accessibility Coverage or reach are important measures of service adequacy for Mäori, as they focus on customer accessibility or uptake of services. “Coverage really matters. Outputs must reach the groups or area where need exist”, and “poor ability to target effective outputs on areas of need will limit value for money” (The Treasury, 2008, p.11 and p.5). The domestic literature on public services shows that Mäori are more likely to experience greater barriers to accessing state services than some other population groups (State Services Commission, 2011a, pp.6, 12) 4 . Examples of coverage measures include: percentage of population in need of receiving the output; percentage of target group who meet entry criteria; percentage of target group who did not access or use the service; and transit time (or other barrier to use). 17 4 See also: Ministerial Advisory Committee on a Mäori Perspective for the Department of Social Welfare, 1988; Te Puni Kökiri, 1996a, 1996b; Ministerial Inquiry into Department of Work and Income, 2000. C O M P A R A T I V E M E A S U R E S – G O O D P R A C T I C E F O R M Ä O R I
Page 1 and 2: MÄTAURANGA KNOWLEDGE Ka möhio, ka
Page 3 and 4: C O N T E N T S 2 EXECUTIVE SUMMARY
Page 5 and 6: (Gill and Russell, 2011, p.8). As D
Page 7 and 8: PART 1 : INTRODUCTION 1.1 PURPOSE T
Page 9 and 10: Table One - examples of performance
Page 11 and 12: 2.2 NO ONE-SIZE-FITS-ALL - PERFORMA
Page 13 and 14: “In brief there is a prima facie
Page 15 and 16: PART 3 : COMPARATIVE MEASURES - GOO
Page 17: 3.3 CAPTURING VARIATION WITHIN MÄO
Page 21 and 22: Raising performance between sites/o
Page 23 and 24: New Zealand (2002) explains that hi
Page 25 and 26: 4.6 MEASURES FOR MÄORI AS PART OF
Page 27 and 28: PART 5: KEY IMPLEMENTATION ISSUES F
Page 29 and 30: 5.4 THE NEED FOR FORETHOUGHT Much o
Page 31 and 32: GLOSSARY Aspirational goals Expecte
Page 33 and 34: A P P E N D I C E S
Page 35 and 36: The “Mäori Perspective” Report
Page 37 and 38: Illustrative examples of Hua Oranga
Page 39 and 40: Table Six - Measures for employment
Page 41 and 42: Donnell, A., and Social Programme E
Page 43 and 44: Kukutai, T. (2004) ‘The Problem o
Page 45 and 46: State Service Commission (2003a) Pu
Page 48: Te Puni Kökiri, Te Puni Kökiri Ho

Measuring performance and effectiveness for MÃƒÂ¤ori ... - Te Puni Kokiri

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?