Improving the Effectiveness of Juvenile Justice Programs: A New

DJJDP already had excellent offender management
tools (a disposition matrix, a validated risk assessment
instrument, an excellent needs assessment instrument,
and a disposition grid) that it had used effectively to assign
offenders to appropriate placement options and reduce
admissions to the state’s Youth Development Centers by
68 percent (North Carolina Department of Juvenile Justice
and Delinquency Prevention, 2008). Adding the SPEP tool
to DJJDP’s repertoire permitted a statewide evaluation of
the presumptive effectiveness of the community-based
programs the state was increasingly using for juvenile
offenders.
To assess the validity of the overall SPEP scores and the
ratings on each of the factors that contribute to those
scores, a recidivism analysis was conducted using data for
juveniles served by 50 SPEP-rated prevention programs
and 113 SPEP-rated programs for juveniles under court
supervision in the community (Lipsey, Howell, and Tidd,
2007). Risk-adjusted recidivism rates for these juveniles
were generated with statistical models that predicted
recidivism based on risk and prior delinquency history.
These models were used to estimate the recidivism rate
of the juveniles under conditions where they all had the
same initial risk for recidivism. Given equal risk, it was
expected that actual recidivism rates would be lower for
juveniles served by programs with high SPEP ratings than
those served by programs with low SPEP ratings. These
analyses found that the SPEP scores were moderately
correlated with the risk-adjusted recidivism rates, with
larger relationships found for the court supervision cases
than for the prevention cases.
2. Arizona
Former OJJDP Deputy Administrator Rob Lubitz was
familiar with the North Carolina SPEP Project. After his
appointment as director of the Juvenile Justice Services
Division (JJSD) in Arizona, Mr. Lubitz championed
SPEP implementation across all Arizona court services
programs. JJSD staff began implementing the SPEP
rating scheme for their contract service providers in five
pilot counties in the fall of 2006. Follow-up activities
were then aimed at prompting providers to plan program
improvements that would elevate their SPEP scores.
Subsequently, the SPEP has been expanded to all JJSDfunded
programs in the state.
Based on information obtained during the contracting
process and by direct contact with the providers, it was
possible to classify nearly all the Arizona programs
as representative of therapeutic program types that
were included in the research that supported the
SPEP and thus to apply the SPEP ratings to them. The
exceptions were brief behavior-specific programs—
short educational programs on topics related to juvenile
behavior problems—for which there is insufficient
research on which to base a SPEP. The Arizona JJSD has
a well-developed data system that includes detailed risk
assessment scores capable of supporting an especially
differentiated SPEP risk rating.
Obtaining data about the duration of service
and number of contact hours by the respective
programs for each juvenile proved more
challenging.
Obtaining data about the duration of service and number
of contact hours by the respective programs for each
juvenile proved more challenging. JJSD did not collect
service information in this form, but the financial records
did identify the billable service units provided to each
juvenile. JJSD staff was able to use that data to establish
service start and end dates and to convert the service
units into contact hours. There was no reasonable source
for quality of service ratings for each program, however,
so the JJSD staff embarked on a project to develop a
rating scheme based on information that could be required
as part of the contracting process, supplemented by site
visits as needed.
JJSD’s experience with the SPEP implementation
stimulated some additional innovations. To oblige
providers to attend to the SPEP ratings for their programs
as a programmatic diagnostic tool, JJSD staff developed a
format for program improvement plans that had to be filed
by each provider during the contracting cycle. The SPEP
experience also highlighted the importance of matching
programs with offender needs and motivated JJSD staff to
develop an improved needs assessment instrument.
Improving the Effectiveness of Juvenile Justice Programs: A New Perspective on Evidence-Based Practice
33