The Interactive Whiteboards, Pedagogy and Pupil Performance ...

8.1.2. Difference-in-Differences (School Fixed Effects)
By using data from before and after the installation of IWBs as part of London
Challenge, we can compare changes in pupil outcomes for departments that
experience large increases in IWB availability, and departments that did not. The
advantages of this methodology are that any constant factors, observable or
unobservable, that may affect overall school performance are eliminated. We do
require the assumption that any unmeasured time-varying factors affect departments
with a large increase in IWB installation (we can think of these as the treatment
group) as much as they do departments with little increase in IWB installation (these
could be thought of as the control group of schools).
This approach estimates the effect of the installation of IWBs by exploiting different
growths in IWB availability between October 2003 and October 2004 across schools
in the sample. Consider the following equation of school achievement in a single
subject:
yst
= γ
s
+ λt
+ M
st
β + ε
(1)
st
where y st is the average achievement in school s in period t; γ s is a time-invariant
effect of school s on pupil outcomes; λ t the school-invariant trend in pupil outcomes
and β is the estimate of the effect of the level of IWB installations, M st , in the
department at school s at time t.
If we assume that trends in achievement would have been the same in the absence
of our treatment (the increase in IWB installations in some departments) then we can
exploit between school differences in installation patterns to estimate the effect on
pupil outcomes:
∆yst
= ( λ
t
− λt
) + ( M
st
− M
st
) β + ε
(2)
−1 −1
st
This basic first difference equation sets up the principle behind the difference-indifferences
(DID) approach, used throughout this paper to estimate the effect of IWB
installation on pupil outcomes. However, because we might be concerned that
changes in the pupil populations at schools is confounding estimates, a pupil level
model is estimated, even though the intervention is a school-level treatment. We do
not have observations for pupils at both points in time (2003/04 and 2004/05), so we
use school fixed effects rather than a first difference approach. Expected pupil
outcomes are conditional on a vector of pupil level characteristics X ist , including prior
attainment. All school characteristics and inputs are assumed be time-invariant,
except for the treatment M st and the year group composition Z st :
yist
= λ
t
+ M
st
β + Z
st
χ + X
istδ
+ γ
s
+ ε
(3)
ist
The equivalent model in a multi-level framework is a random effects model, which
parameterises the distribution of the school effects by assuming they are normally
distributed. The equivalent equation is:
yist
= λ
t
+ M
st
β + Z
st
χ + X
istδ
+ uist
, where uist
= η
st
+ ε
(4)
ist
In certain circumstances this would be the more efficient approach. However, our
sample of schools is quite small here, so the normality of school effects assumption
might not be valid. We test the validity of the random effects (multi-level) estimates
by comparing them to the fixed effects model in equation (3) using a Hausman
specification test. The results indicate that the random effects approach is
inappropriate for the English regressions and so is not reported.
64