Improving the Effectiveness of Juvenile Justice Programs: A New

effectiveness found in the meta-analysis and the basis for
rating them in the SPEP are as follows.
Type of program. The SPEP covers only program types
that take a therapeutic approach, as defined in the
program categories used in the meta-analysis (e.g., family
counseling, mentoring, cognitive-behavioral therapy,
vocational training). The relative effectiveness of each
program type for reducing recidivism that was found in
the statistical analysis was used to categorize program
types as having, on average, high, medium, or low effects
on recidivism, keeping in mind that even the low program
types nonetheless have positive average effects. The total
number of points—which represents the proportionate
contribution of program type to predicting recidivism
effects—is distributed across these categories so that the
maximum number of program type points goes to those in
the high category with discounted scores given to program
types in the medium and low categories.
To determine which program type a local program
represents, and thus what its SPEP score is on that
factor, descriptive information about the nature of the
services it provides must be examined. That information is
compared with the descriptions in a glossary of program
types that was developed from the descriptions provided
in the corresponding research studies included in the
meta-analysis. The local program is then identified with
regard to the program type it represents and, depending
on whether that program type is classified as having low,
medium, or high effectiveness, the corresponding SPEP
rating is assigned. If a program does not match any of the
program types in the glossary, it means that insufficient
research exists for estimating the effectiveness of that
type of program.
Many programs involve combinations of services that may
represent different program types. In those cases, primary
and supplementary services are distinguished and, if the
supplementary services are of a different type from the
primary service, but of a type shown to be effective in the
research, bonus points are awarded for it.
Amount of treatment. Service amount is divided into
duration and total contact hours, with the latter receiving
somewhat more points in light of its slightly stronger
relationship to outcomes. Service duration is assessed
as the time (e.g., number of weeks) between the date
of service intake and the date of service termination for
each juvenile with a closed case who was served by
the program over the period of time to which the SPEP
is applied (e.g., SPEP ratings might be made annually).
Similarly, total contact hours are assessed as the
number of hours of direct exposure each juvenile had to
substantive program activities. In both cases, these values
must be determined from actual service records, not
estimated subjectively.
The SPEP ratings for these service dimensions assign
a greater or lesser proportion of the points available for
amount of service according to the proportion of the
juveniles served with service duration or contact hours
that reach or exceed specified target values. Those target
values are set at the average found in the corresponding
research studies for programs of that type. This is based
on the assumption that, if the amount of service provided
at least reaches the average reported in the respective
research studies, the program should attain at least the
average effects on recidivism found for that program type.
Quality of treatment. The quality of the treatment
implementation is the most difficult SPEP factor to rate
on the basis of actual program data. This factor, as it is
represented in the research studies and analyzed in the
meta-analysis, refers to the extent to which the program
was implemented as intended for every juvenile recipient.
Such information is not generally collected as part of
the management information or client-tracking systems
used by juvenile justice agencies and may have to be
developed in order to support full SPEP ratings. Drawing
on the representation of this factor in the research studies,
we identify the key dimensions of implementation quality
as (1) a written protocol describing the intended service,
(2) provision of training on the intended service for those
delivering it, (3) a regular procedure for monitoring service
to assess whether it is being delivered as intended, and
(4) a procedure for taking corrective action when service
delivery strays from what is intended. Note that these are
not dimensions of clinical quality, which may be important
but are not captured well in the research on which the
SPEP is based. Rather, these are organizational matters
that can be assessed in terms of the operating procedures
established and maintained by the provider delivering the
program being rated.
30 Improving the Effectiveness of Juvenile Justice Programs: A New Perspective on Evidence-Based Practice