Inside the Black Box - Chapin Hall at the University of Chicago

More documents

Recommendations

Info

literature often notes the need for reliable, consistent, and appropriate performance measures and data in order to improve program performance and achieve success (Miles, et al., 2010; Bearer-Friend, 2009; Bellotti, Rosenberg, Sattar, Esposito, & Ziegler, 2010; Holzer & Nightingale, 2009; Shore & Shore, 2009). Nevertheless, efforts to develop common integrated performance measures across various types of workforce development programs are challenged by the need to incorporate differences in program populations, models, and outcome goals (Wilson, 2005). Existing process measures are more rarely described in the literature than are outcome and impact measures. A recent United States Government Accountability Office report describes the need for greater investment in rigorous evaluation of workforce development programs’ impact and investigation of how the workforce system can be modified to enhance its services (U.S. GAO, 2009). Recommendations for process measures, however, are quite limited in the existing literature, especially measures that transcend differences in program service-delivery models and participants. Some nonexperimental studies have found how differences in program design and populations relate to program impacts. For example, Heinrich, Mueser, and Troske (2008) found that women and adult program participants had higher earnings than participants in a dislocated worker program. An MDRC report from 2006 explored how different service-delivery strategies significantly influence retention and advancement. Finally, a report from the Joyce Foundation’s Transitional Jobs Reentry Demonstration, which is testing employment programs for former prisoners, found the transitional job group was much more likely to work than the control group early on, but that the differences faded as men left their subsidized jobs (Redcross et al., 2010). Additional information about process measures and influences on program outcomes will likely be gleaned from studies currently underway. MDRC is conducting an experimental study of the Workforce Investment Act (WIA), to be completed in 2015, that will explore how services are implemented and how differences in implementation affect participants’ employment, earnings, and related outcomes. The Public/Private Ventures “Benchmarking Project” is currently identifying meaningful outcome benchmarks for the workforce development field to better understand what constitutes “good” performance, considering many influences on program outcomes such as program population, services, and organizational characteristics like age and size (Miles et al., 2010). The literature does, in some cases, go beyond enumerating the process measures collected to describe how programs may benefit from better use of process data and improved measures. One report highlights the role data and use of performance measures can have to tailor programs to meet the needs of both participants and employers and see increased retention and placement rates (Clymer, 2003). A recent report by Eberts (2009) looked at how performance measures can be improved for the nation’s workforce Chapin Hall at the University of Chicago 12
development system and found that in order to measure accurately the value added of the services programs provide when looking at placement, retention and wage outcomes, local labor market conditions and the personal characteristics and work history of the participants need to be taken into account. Moreover, combinations of process and outcome data may help organizations assess their programs and make changes to better serve program participants. In particular, one report found that participants with multiple barriers benefitted from periodic assessment of needs, growth, success, and long-term outcomes (Council for Adult and Experiential Learning (CAEL), 2006). Another noted that interim evaluations during enrollment could be based on combinations of assessment, program data, and administrative data such as participant employment and educational outcomes (Abt Associates and Metis Associates, 2005). Despite the limited coverage of measuring workforce development program practices, there is a wealth of literature on what these promising practices are, and their potential impact on participant and program outcomes. Promising Program Practices and Characteristics The workforce development literature is rich with descriptions of promising practices and characteristics of successful workforce development organizations, highlighting many different factors across the ecological model levels that likely influence success across the variety of workforce development programs. Program Elements Support Services. The literature emphasizes the provision of support services and case management—including child care, transportation, access to housing and cash assistance, food stamps, health care services, and referrals to substance abuse and mental health services—over any other factor in helping participants overcome barriers to attendance and work (Purmort, 2010; Klein- Collins, 2002; Maguire, Freely, Clumer, & Conway, 2010). For long-term unemployed individuals, research shows the advantage of a human development approach, which draws on psychology and sociology and focuses on understanding the process of change and growth in individuals (Herr & Wagner, 2007). Flexibility. Flexibility in programming and service delivery is another crucial component of program success. Organizations that support a program’s ability to alter services in order to adapt to both business and participant needs are more likely to be efficient and successful. Flexibility in hours, location, and method of service delivery are desired characteristics in programs’ offerings (Hendra, et al., 2010; Strawn, 2010a; Holzer & Nightingale, 2009; Rynell, 2001; Maguire, et al., 2010). Family Involvement. Family engagement in programs—including training multiple family members, assessing and intervening in family issues at the time of program enrollment, involving family Chapin Hall at the University of Chicago 13
Page 1 and 2: Inside the Black Box: What Makes Wo
Page 3 and 4: Acknowledgments We would like to th
Page 5 and 6: Policy and Advocacy Efforts .......
Page 7 and 8: Executive Summary The workforce dev
Page 9 and 10: 1. Interface with other required da
Page 11 and 12: While acknowledging funding as a pr
Page 13 and 14: Introduction The workforce developm
Page 15 and 16: Figure 1: Interaction of Influences
Page 17: Review of Recent Literature A revie
Page 21 and 22: for contributing to program success
Page 23 and 24: capture all of the participants’
Page 25 and 26: The organizations also reported tha
Page 27 and 28: that needs to be collected. Another
Page 29 and 30: Recommendation 2: Develop longitudi
Page 31 and 32: seldom systematically collected by
Page 33 and 34: With regard to the planning and dev
Page 35 and 36: models in order to best serve their
Page 37 and 38: organizations are interested in col
Page 39 and 40: Poor attitude, poor work history, o
Page 41 and 42: Organizations often use a wide rang
Page 43 and 44: …a more holistic approach to work
Page 45 and 46: in the industry. Professional crede
Page 47 and 48: Success Factors: Workforce Provider
Page 49 and 50: education or no work experience. Ot
Page 51 and 52: equirements. As part of these effor
Page 53 and 54: given as the reason organizations d
Page 55 and 56: passion to see, to transform this c
Page 57 and 58: permanent job opportunities. These
Page 59 and 60: Meet the Needs of Participants Many
Page 61 and 62: participant is linked to an opening
Page 63 and 64: Provide Business Services to Engage
Page 65 and 66: The organizations in the study all
Page 67 and 68: community, such as the services the
Page 69 and 70:
depend more heavily on other fundin
Page 71 and 72:
Sometimes a legislation that is out
Page 73 and 74:
on the participants they serve. Als
Page 75 and 76:
Community engagement is also essent
Page 77 and 78:
Website Description Length of Progr
Page 79 and 80:
Eligibility Participants Served Par
Page 81 and 82:
Participants Served ninth grade lev
Page 83 and 84:
Length of Program Size of Program E
Page 85 and 86:
Size of Program Eligibility Partici
Page 87 and 88:
Eligibility Participants Served Ind
Page 89 and 90:
Bragg, D.D., Kim, E., & Barnett, E.
Page 91 and 92:
Herr, T., & Wagner, S. L. (2007). B
Page 93 and 94:
NGA Center for Best Practices, Nati
Page 95 and 96:
The Workforce Boards of Metropolita
Page 97 and 98:
The qualitative design of this proj
Page 99:
1313 East 60th Street Chicago, IL 6
show all

Inside the Black Box - Chapin Hall at the University of Chicago

Create successful ePaper yourself

Delete template?

Save as template?