Critical Appraisal of Clinical Prediction Rules That Aim to Optimize ...

effect of rule status (positive or negative)
on response to treatment (manipulation
versus placebo). 32 The 2
validation RCTs both used longer,
more clinically relevant maximum
follow-up times of 12 weeks 32 and 6
months. 26
Discussion
We found that all derivation-level
CPRs used single-arm study designs
to derive predictors, raising doubts
about the veracity of current CPRs.
In total, 15 CPRs to aid selection of a
range of treatments for musculoskeletal
conditions were found. However,
only 1 CPR for selecting spinal
manipulation for LBP 30 had reached
the validation stage of CPR development,
with the CPR predicting
response to treatment in a narrow
validation study but not treatment effects
in a broad validation study. 26,28,32
Although single-arm study designs
can be a preliminary step in developing
prediction rules by identifying
potential candidate variables, they
are not able to differentiate between
predictors of response to treatment
and predictors of outcome regardless
of treatment. These studies do
not include a control group, so they
cannot provide information on treatment
effects or on factors that modify
treatment effects. The predictive
factors identified in these studies,
therefore, have a higher risk of being
merely nonspecific predictors
of outcome or prognostic factors.
Neither of the 2 previous systematic
reviews 16,17 on this topic discussed
this issue of use of a singlearm
trial design at the derivation
level. One review stated that “most
of the derivation studies were of
high quality,” 17(p40) a claim we
would argue is potentially misleading,
considering no derivation studies
used a study design that allows
specific identification of treatment
effect modifiers.
Clinical Prediction Rules for Musculoskeletal Conditions
Figure.
Flow chart describing the results of the literature search. CPRclinical prediction rule.
One potential justification for using
prognostic factors identified in
single-arm trials to develop a treatment
CPR is that the prognostics factors
also may be treatment effect
modifiers. Although this can happen,
30 there are cases where it does
not. 43 There also are examples
where the same clinical feature predicts
poor prognosis yet predicts a
good response to treatment. 44 This
uncertain relationship makes it essential
to carefully interpret the results
of studies reporting a treatment
CPR from single-arm trials. Although
these studies are hypothesis generating,
variables identified in single-arm
trial designs run a greater risk of not
being significant in a subsequent
controlled study. Moreover, for
some CPR candidate variables, there
are many existing data sets from
RCTs that would provide a better
evaluation of the variable as an effect
modifier than a single-arm study. For
example, 5 of the CPRs included age
as a variable, 25,33,36,39,40 and as age is
almost always measured in RCTs, a
more robust evaluation of age as a
treatment effect modifier would be
possible from secondary analysis of
individual trials or from the pooled
data from several trials using a metaregression
approach. Having said
that, to properly develop a CPR, an
RCT designed specifically for the
purpose of CPR development, with
appropriate sample size and a priori
analysis, is necessary. 22
June 2010 Volume 90 Number 6 Physical Therapy f 847
Downloaded from
http://ptjournal.apta.org/ by guest on May 30, 2013