pdf - Institute for Policy Research - Northwestern University

P. Reese
Quantitative Methods for Policy Research
44
IPR faculty listen to a Q-Center
colloquium on fractional treatment rules.
correspondence is what theory predicts.
But to achieve it in the complex social
settings in which these within-study
comparisons were carried out suggests that
regression discontinuity results might be
more generally robust than some critics
contend.
Research Methods for Education
Larry V. Hedges, Board of Trustees
Professor of Statistics and Social
Policy, is reanalyzing surveys with
nationally representative samples to
develop reference values of intraclass
correlations. These data can then be used
to help plan experiments in education.
For example, one study with graduate
student Eric Hedberg
provides a compilation of
intraclass correlation values
of academic achievement
and related covariate effects
that could be used for
planning group randomized
experiments in education.
This project has funding
from the Interagency
Educational Research
Initiative (IERI). IERI
is a collaborative effort
of the National Science
Foundation, Institute of
Education Sciences (IES), and National
Institute of Child Health and Human
Development to support scientific
research that investigates the effectiveness
of educational interventions in reading,
mathematics, and the sciences.
In another project supported by IES,
Hedges is developing improved statistical
methods for analyzing and reporting
multilevel experiments in education. He
is also working on more efficient designs
for such experiments that require the
assignment of fewer schools. Such designs
should reduce the costs of educational
experiments and thus make them more
feasible to conduct.
For those designs involving cluster
randomization, Hedges has defined three
effect sizes—and computing estimates
of those effect sizes and their standard
errors—from information that is likely
to be reported in journal articles. He
has also provided a simple correction to
the t-statistic that would be computed if
clustering were incorrectly ignored.
Social psychologist Thomas D. Cook
and William Shadish of the University
of California, Merced, held three oneweek
workshops in summer 2006 for
84 educational researchers mainly from
universities, contract research firms, and
school districts. They will hold three
more in 2007. All of the workshops are
supported by the Spencer Foundation.
In these workshops, the two organizers
cover the most empirically viable quasiexperimental
practices such as regression
discontinuity designs and interrupted
time series. They lecture on theory and
practice, supplementing their discussions
with as many examples as possible from
education, highlighting the advantages
and disadvantages of using them. They
also rely on empirical research that
compares the results of randomized
experiments to quasi-experiments that
shared the same intervention group.
In addition, Cook continues work on
quasi-experimentation in education that
is supported by the Spencer Foundation.
Many researchers believe that
randomized experimentation is usually
the best methodology for investigating
issues in education. However, it is not
always feasible. The usually advocated
alternative—quasi-experimentation—
has recently come under attack from
scholars who contrast the results from
a randomized experiment and a quasiexperiment
on the same topic, where
the quasi-experiment shares the same
intervention as the experiment. Thus,