Inside the Black Box - Chapin Hall at the University of Chicago

Recommendation 2: Develop longitudinal measures on individuals to assess outcomes over
time.
Workforce programs in the study noted the need to track participant outcomes over a longer period of
time than currently required by funders. However, programs are not currently funded for long-term
tracking of outcomes. Additional support would be needed to allow programs to follow up with
participants for a longer period of time. Furthermore, longitudinal information obtained from linking
existing administrative records, such as states’ employment and wage data systems, may give programs
some of the employment, wage, and educational attainment outcomes they want to track.
Recommendation 3: Analyze outcomes by subpopulations, including participant
characteristics and barriers to employment, in order to develop risk-adjusted performance
expectations.
Current measures do not account for variation in achieving outcomes among those participants who are
hardest to serve—those that have multiple barriers to employment. Measures and data targeting specific
subpopulations or those with employment barriers would allow better understanding of outcomes for
these participants and performance expectations to be adjusted. This subpopulation analysis is feasible if
data is available on all program participants, including their characteristics and other factors that may
identify barriers to employment.
Recommendation 4: Expand what is measured in program processes and outcomes,
including measures within the individual client, program, organization, and contextual
measures of external engagement.
More comprehensive data on a fuller array of measures related to program success will provide a better
understanding of participants’ needs, progress, and outcomes as well as program process and
performance. Much of the data for these measures may already be collected by programs, but may not be
part of a data system or formal data collection effort. Some information can be obtained through linked
administrative data, such as data on other public benefits and services, which can be used to determine
economic self-sufficiency and family well-being. Other measures, however, would require additional data
collection. The following are suggestions for types of data that would help to give a fuller picture of the
relative success of workforce development programs and their effects on participants, along with the
feasibility of incorporating these measures.
Initial assessment and continual development of hard skills and soft skills
This information is essential in order to understand the needs and skills of those entering programs as well
as monitoring their development as they progress through the program. These measures could be easily
Chapin Hall at the University of Chicago 23