sign130

Brain injury rehabilitation in adults
1.4 METHoDoloGICAl lIMITATIoNS INHERENT IN REHABIlITATIoN lITERATuRE
4 |
There are inherent difficulties in appraising the quality of rehabilitation studies by traditional evidence
based methods. This is particularly true of the complex, experience-based treatments that predominate in
rehabilitation over medically-oriented treatments such as pharmacotherapy and surgery. 14 Interventions
that involve explicit teaching, behaviour change, and/or environmental manipulations cannot typically be
hidden from the patient or the therapist, thus the removal of bias by using standard blinding procedures,
such as placebo treatment, is not straightforward. Unlike medical treatments which may be aimed at specific
symptoms, rehabilitation interventions usually target multiple or complex outcomes at the levels of activity
and participation. Identification of a primary outcome for such treatments may be impossible and even
inappropriate. Goals associated with successful treatment will vary across participants, meaning that simple
outcome measures may not provide universal and objective metrics of improvement. Moreover, a highly
meaningful intervention may appear meaningless if the wrong outcome measure is selected. Rehabilitation
interventions are often delivered by members of multiple disciplines working synergistically, complicating
the application of quality appraisal standards that do not incorporate such factors. 15
1.4.1 THE CHOICE OF STUDy DESIGN IN REHABILITATION RESEARCH
Randomised controlled trials (RCTs) where patients are randomly assigned to at least two comparison groups
are best able to control for threats to the internal validity of studies and ensure pre-treatment equivalence of
experimental and control groups, strengthening the basis for statistical inference. Completing an RCT with
an adequate sample size, appropriate randomisation techniques to account for variability in the diagnostic
conditions and a combination of patient, service and/or system level outcome measures is difficult due to
competition for rehabilitation research funding and the individual nature of brain injuries.
There are also ethical constraints in using RCTs, particularly with severely affected patients for whom clinicians
believe there are no realistic alternative interventions to specialised care. Notably for conditions in which
multidisciplinary rehabilitation has become the standard of care without systematic evidence to support it
in practice, denying services randomly in order to conduct an RCT could be considered unethical. 16
Few rehabilitation-focused, observational studies control for selection bias. A systematic review of
multidisciplinary rehabilitation services in post-acute care across a range of populations concluded that
adjusting for case mix when examining the influence of two or more interventions on an outcome does
not necessarily sufficiently reduce the selection bias associated with assignment to each intervention
group. 16 Adjusting for the probability of receiving the treatment is essential in non-randomised studies of
the effectiveness of rehabilitation services.
The majority of studies published describing patients with brain injury use single-case design or are small
case series, reflecting the individual nature of rehabilitation interventions, the challenges of using more
complex designs and the relative simplicity of conducting single case studies. While RCTs may suffer from
problems with applicability of results or heterogeneity of included patients or wider population, “studies of
individuals and small case series can be optimal for exploring a new treatment, for titrating therapies, for
documenting a promising variation in behavioural therapies, for enhancing knowledge of generalisation of
treatment to a new group, and to enhance understanding of why some patients respond to a treatment of
known (average) effectiveness whereas others do not, that is, for extending results of an RCT.” 14
The disadvantages associated with single case design studies are well reported. These include the difficulty in
drawing cause-and-effect conclusions (limited internal validity), possible biases when interpreting outcomes
due to observer bias and bias in data collection, and crucially, the problem of generalising findings from a
single individual to a group or wider population (limited external validity). While researchers can take steps
to attempt to limit the biases associated with this design there remain difficulties in assessing behaviours
which do not reverse back to baseline after withdrawal of treatment, indicating that the treatment may not
have been the key variable affecting change. Single case studies are usually ranked at the bottom of the
traditional hierarchy of evidence. 17