Education and Training in Developmental Disabilities - Division on ...

Gresham, Kern, Barreras, Thornton, & Crews,
2008; Hwang & Hughes, 2000; Vaughn et al.,
2003; White et al., 2007). None of this research
has discussed whether the studies met
criteria for be<strong>in</strong>g evidence-based practice.
Def<strong>in</strong><strong>in</strong>g Evidence-Based Practices
National policies, such as No Child Left
Beh<strong>in</strong>d (NCLB) require that teachers use
evidence-based practices <strong>in</strong> their classrooms.
Also, new regulations <strong>in</strong> the reauthorization
of IDEA <strong>in</strong> 2004 require that evidence-based
practices be used to ensure <strong>in</strong>dividuals with
disabilities receive the highest quality <strong>in</strong>struction.
However, there has never been a clear
def<strong>in</strong>ition of what evidence-based practice is.
Therefore, <strong>in</strong> January 2003, the Council for
Exceptional Children (CEC) <strong>Division</strong> for Research
established a task force to address the
issue of evidence-based practices. A special
issue of Exceptional Children (2005) was dedicated
to establish<strong>in</strong>g criteria for evidencebased
practice <strong>in</strong> special education.
First, let’s clarify what a practice is. Accord<strong>in</strong>g
to Horner, Carr, Halle, McGee, <strong>and</strong> Wolery
(2005), a “practice refers to a curriculum,
behavioral <strong>in</strong>tervention, systems change, or
educational approach designed for use by
families, educators, or students with the express
expectation that implementation will result
<strong>in</strong> measurable, educational, social, behavioral,
or physical benefit.” (p. 175)
In the 2005 special issue of Exceptional
Children, Odom et al. set the context for the
development of research quality <strong>in</strong>dicators
<strong>and</strong> guidel<strong>in</strong>es for evidence of effective practices
provided by different methodologies <strong>in</strong>
special education, <strong>in</strong>clud<strong>in</strong>g group experimental
or quasi-experimental research, s<strong>in</strong>gle
subject research, correlational research <strong>and</strong>
qualitative research. For the purpose of this
article, we will focus on two types of research
methodologies: group experimental or quasiexperimental
<strong>and</strong> s<strong>in</strong>gle subject research.
These two methodologies are largely used to
identify cause-effect relationships between <strong>in</strong>terventions
<strong>and</strong> target behaviors, therefore,
they are more appropriate for identify<strong>in</strong>g
evidence-based practices.
Specifically, for group experimental <strong>and</strong>
quasi-experimental research articles <strong>and</strong> reports,
Gersten et al. (2005) presented the fol-
low<strong>in</strong>g quality <strong>in</strong>dicators: (1) participants <strong>in</strong> a
given study are sufficiently discussed <strong>and</strong> their
disability conditions are confirmed; (2) r<strong>and</strong>om
assignment to study conditions are attempted
<strong>and</strong> when r<strong>and</strong>omization is not feasible,
other alternatives are used to ensure
participants are comparable across conditions;
(3) sufficient <strong>in</strong>formation is provided
regard<strong>in</strong>g the <strong>in</strong>terventionists <strong>and</strong> procedures
implemented to ensure they are comparable
across conditions; (4) <strong>in</strong>tervention strategies
are implemented with fidelity, (5) multiple
outcome measures are used to capture the
<strong>in</strong>tervention’s effect, (6) data analysis <strong>in</strong>clude
effect size calculations <strong>in</strong> addition to <strong>in</strong>ferential
statistics (p. 152).
Furthermore, the panel suggested that a
practice is considered evidence-based “when
there are at least four acceptable quality studies
or two high quality studies that support the
practice <strong>and</strong> the weighted effect size is significantly
greater than zero.” (Gersten et. al.,
2005, p. 162)
When evaluat<strong>in</strong>g research studies us<strong>in</strong>g
the s<strong>in</strong>gle subject methodology, Horner et al.
(2005) stated that “s<strong>in</strong>gle-subject research
documents a practice as evidence based when
(a) the practice is operationally def<strong>in</strong>ed,
(b) the context <strong>in</strong> which the practice is to be
used is def<strong>in</strong>ed; (c) the practice is implemented
with fidelity; (d) results from s<strong>in</strong>gle
subject research document the practice to be
functionally related to change <strong>in</strong> dependent
measures, <strong>and</strong> (e) the experimental effects
are replicated across a sufficient number of
studies, researchers, <strong>and</strong> participants to allow
confidence <strong>in</strong> the f<strong>in</strong>d<strong>in</strong>gs” (p. 175-176).
Additionally, documentation of an evidencebased
practice typically requires multiple s<strong>in</strong>gle
subject studies. “A practice may be considered
evidence-based when (a) m<strong>in</strong>imum of
five s<strong>in</strong>gle subject studies that meet m<strong>in</strong>imally
acceptable methodological criteria <strong>and</strong> document
experimental control have been published
<strong>in</strong> peer-reviewed journals, (b) the studies
are conducted by at least three different
researchers across at least three different geographical
locations, <strong>and</strong> (c) the five or more
studies <strong>in</strong>clude a total of at least 20 participants”
(Horner et al., 2005, p.176).
Although there have been a few review articles
published on this topic <strong>in</strong> recent years
(e.g. Matson, Matson, & Rivet, 2007; McCon-
Evidence-Based Social Skills Interventions / 319