Inside the Black Box - Chapin Hall at the University of Chicago

The qualitative design of this project used a case study approach based on semi-open-ended, in-depth
interviews with organization staff. Interviews were based on the workforce development literature about
influences on service delivery, promising practices, and measurement and outcomes. Fifteen interviews
were conducted with 16 workforce program staff. One interview was conducted with two interviewees;
the rest took place with researchers and a single interviewee. Interviewees included executive directors,
vice presidents, program directors, and other directors within the organizations. Organizations
participated in two to three interviews that lasted between one and two hours each. At the end of the
interviews, interviewers requested documents that were referenced during the interview related to
program description, process measurement, and evaluation.
Follow-up communication with each organization occurred at least four points in time. The first took
place within a few days of the interview to request the additional documents mentioned during the
interview, but not provided at the time of the interview. A follow-up email and then a phone call took
place several months following to pose any additional questions and request additional documentation
about program description, process measurement, and evaluation. The final contact occurred two months
later, as programs were asked to edit and approve a short program description to be included in the
appendix of this report.
All interviews were digitally recorded and transcribed verbatim. Supporting documentation of program
description, process measurement, and evaluation were also coded. ATLAS.ti 6 was used to conduct
qualitative coding, which was based on a set of broad, not mutually exclusive, coding categories designed
to correspond to the topics and themes of the interview protocol and the framework of the ecological
model. Each transcript was coded twice, by two separate researchers, to improve inter-rater reliability.
The researchers then met to resolve discrepancies.
Analyses of the transcript coding was conducted to develop themes and findings related to program
offerings, participants, practices and service delivery strategies, and use of data and information.
Qualitative data matrices of code content and contextual narrative were created to validate the themes
across interviews and organizations and further develop the findings. No specific program or interviewee
is identified in the report, and no program was overrepresented.
Chapin Hall at the University of Chicago 91