ITT ITT

Munchën, 24th April 2013
Causal inference based on potential
outcomes: non-compliance and
instrumental variables
Leonardo Grilli
Teaching staff Erasmus mobility
LUDWIG-
MAXIMILIANS-
UNIVERSITÄT
MÜNCHEN
grilli@disia.unifi.it
local.disia.unifi.it/grilli
2
Introduction
Instrumental variables techniques are a key tool for drawing
causal inferences in difficult settings such as
Randomized controlled trials with non-compliance (the subjects
do not perfectly adhere to the assigned treatment)
Observational studies with confounded treatments, where the
available covariates are not enough to reasonably assume
unconfoundedness (i.e. relevant unobserved confounders
affect both the treatment and the outcome)
In such settings, standard methods based on unconfoundedness
(stratification, regression, matching) yield biased results
We will initially focus on the case of controlled experiments
with non-compliance, then we will consider the broader case of
observational studies (sometimes called ‘natural experiments’)
Experiments with non-compliance
4
3
Experiments with non-compliance
• Types of non-compliance
• Notation
• Causal effects
• Compliance behavior
Randomization implies that the treatment assignment
Z is independent of all the pre-assignment variables
(including the potential outcomes) thus
unconfoundedness holds
Unfortunately, in case of non-compliance (nonadherence)
the actual treatment D
is not the same as the assigned treatment Z and
it is a post-assignment variable: conditioning on D is
harmful since it violates the unconfoundedness
assumption

Types of compliance
Types of non-compliance /cont
5
Compliance can be
Binary (all-or-none): e.g. the patient assigned to a
new therapy only has two options: to participate or to
not participate
Partial: e.g. the patient assigned to a treatment
entailing the take of one pill per day for 30 days may
take 0 pills (she does not participate), or 30 pills (full
compliance), or 0

Causal effects
Compliance behaviour
9
10
For a given individual in a given experiment, all the
causal effects of Z on post-assignment variables are
defined a priori in terms of potential outcomes
Z on D: D(1) D(0)
Z on Y: Y(1, D(1)) Y(0, D(0))
Since Z is randomised both effects are estimable
The average causal effect of Z on Y is known as
Intention-To-Treat effect (ITT)
1, (1)
0, (0)
ITT E Y D Y D
In the ITT there is no conditioning on the value of D: each subject is left
free to react to Z on the basis of her compliance behaviour
Z and D binary 4 compliance states (types of
reaction to treatment assignment)
D(1) D(0) D(1)- D(0) Type Prob.
1 0 1 Complier (C) C
0 0 0 Never taker (NT) NT
1 1 0 Always taker (AT) AT
0 1 -1 Defier (D) D
1
C NT AT D
Compliance behaviour /cont
Latent and observable groups
11
12
The labelling of individuals as C, NT, AT and D on the basis of
the reaction to treatment assignment is
Unobservable, i.e. the groups C, NT, AT and D are latent groups
Relative to the behaviour of individuals in the actual experiment,
not in general (e.g. a Never Taker in an experiment may be a
Complier in another experiment)
It is not affected by the actual assignment so it is a preassignment
variable (this is a key property: since Z is
randomised, the composition of the population in terms of
compliance behaviour is the same at any level of Z)
For AT and NT there is no information to estimate the causal
effect of D on Y since Z is unable to change D
General
D obs
Z 0 1
0 NT /C AT /D
1 NT /D AT /C
Monotonicity
( no Defiers)
D obs
Z 0 1
0 NT /C AT
1 NT AT /C
Treatment available only to
individuals assigned to
treatment ( no D, no AT)
D obs
Z 0 1
0 NT /C .
1 NT C
Membership to latent groups cannot be established (except for certain
combinations of Z e D obs depending on the design)
Nevertheless, if Z is randomised and the latent groups are no more than 3,
we can estimate the probabilities of group membership (see next example)

Latent and observable groups /cont
13
Hypothetical experiment with random assignment Pr(Z=1)=0.5 and monotonicity
Percentages of groups
Assigned Control arm
Treatment arm
Population Taken Control Treatment Control Treatment
50 Compliers 25 25
30 Never Takers 15 15
20 Always Takers 10 10
100 Observed 40 10 15 35
50 50
The observed proportions allow us to recover the proportions of
the 3 latent groups:
Prop(AT) = 10/50 = 0.2 Prop (NT) = 15/50 = 0.3
Prop(C) = 1-0.2-0.3 = 0.5
14
Complier Average Causal Effect
• Definition of CACE
• Estimation
• Instrumental Variables
• Comparison with Intention-To-Treat (ITT) effect
• Traditional (and biased) estimators
15
CACE (or LATE)
Data contain information to estimate the causal
effect on the compliers, known as
CACE = Complier Average Causal Effect, or
LATE = Local Average Treatment Effect
• Angrist J., Imbens G.W, Rubin D.B. (1996) Identification of
Causal Effects using Instrumental Variables. JASA 91, 444-
472.
• Imbens, G.W. and Rubin, D.B. (1997) Bayesian inference for
causal effects in randomized experiments with noncompliance.
Annals of Statistics 25, 305–327, 1997.
• Mealli F., and D.B. Rubin (2002) Assumptions when Analyzing
Randomized Experiments with Noncompliance and Missing
Outcomes. Health Services and Outcomes Research
Methodology 3, 225-232.
Joshua Angrist
Guido Imbens
Donald Rubin
16
From ITT to CACE
Assume SUTVA
ITT (Intention-To-Treat effect)
ITT ITT ITT ITT ITT
C C NT NT AT AT D D
ITTgroup E Y i
1, Di (1) Yi 0, Di
(0) | i group
P i group
group
(essentially, SUTVA rules out interactions
among units, so we can write expressions
holding for a single, arbitrary unit)
1, (1)
0, (0)
ITT E Y i
Di Yi Di
Decomposition of the ITT across the four latent compliance groups:

17
Exclusion restriction
Assume Exclusion restriction:
Y(1, d) Y(0, d) for d 0,1
i
i
• it rules out a direct effect of Z on Y (it captures the notion underlying
instrumental variables procedures that any effect of Z on Y must be via
an effect of Z on D)
• it relates quantities that can never be jointly observed, thus it is not
directly verifiable form the data (although it has testable implications)
• it is violated in case of a placebo effect (indeed, placebo-controlled
experiments, blinding and double-blinding aim at making the exclusion
restriction more plausible)
• the standard version of the exclusion restriction refers to all the units, and
thus it applies to all the latent groups (compliers, never takers, etc.), but in
some settings it may be useful to relax the exclusion restriction for a
specific group, for example for the never takers
18
Exclusion restriction /cont
The exclusion restriction has two consequences
1. When Y is studied as a function of the actual treatment D,
the assignment Z is superfluous, thus we can write
Y d
i
( ) instead of Y( z, d)
2. The ITT for never takers and always takers is null
ITT
NT
ITT 0
The consequence #2 implies (along with SUTVA) that the
decomposition of the ITT simplifies as follows:
AT
ITT ITT
ITT
C C D D
i
Strong monotonicity
CACE (or LATE)
19
Assume Strong monotonicity:
1) Non-null average effect of Z on D
E D(1) D(0)
0
i i C D
2) Monotonicity (No defiers):
Remark: 1 & 2 imply
0
ITT ITT C
C C
D
E D(1) D(0)
0
i i C
Part 1 states that Z moves D
Part 2 states that Z only moves D in one direction
Along with SUTVA and exclusion restriction, strong monotonicity implies that
with 0
20
From the decomposition of the ITT in the last line of the previous slide,
it follows that
ITT EYi1, Di(1) Yi0, Di(0)
CACE ITT
C
E D(1) D(0)
C i i
CACE is defined as the causal effect of D on Y for the unobservable
group of compliers
we have shown that, assuming (i) SUTVA, (ii) exclusion restriction and
(iii) strong monotonicity, CACE is the ratio between two causal effects:
the effect of Z on Y (ITT) and the effect of Z on D
CACE is a population quantity: up to now we did not consider
estimation (note: we did not invoke unconfoundedness or
randomization of Z, since this relevant only for estimation)

21
Estimation of CACE
We have shown that under the assumptions (i) SUTVA, (ii)
exclusion restriction and (iii) strong monotonicity
ITT E Yi 1, Di(1) Yi 0, Di(0) EYi 1, Di(1) EYi 0, Di(0)
CACE
E D(1) D(0) E D(1) E D(0)
C i i i i
Under random assignment of Z, a consistent estimator of CACE is
obtained by estimating the expected values using the
corresponding sampling means:
obs
Y
Z
Y
CACE D
obs
D
obs
[ 1] [ Z0]
obs
[ Z1] [ Z0]
22
ITT vs CACE
If Z is randomized we can consistently estimate
ITT (effect of Z on Y)
• Whole population
• Assumptions: SUTVA
CACE (effect of D on Y)
• Compliers (non-observable sub-population)
• Assumptions: SUTVA, Exclusion restriction and Strong monotonicity
If we are interested in the effect of D on Y, then CACE is more
relevant than ITT, even if it requires stronger assumptions and it refers
to a non-observable sub-population
Advantages of ITT
Disadvantages of ITT
23
Estimation of ITT is more robust than estimation of CACE since it
is based on fewer assumptions
Under the usual assumptions, CACE = ITT/pr(complier), thus
|CACE| > |ITT|
The ITT is a conservative estimate of CACE, so if we find a
relevant ITT effect, there is also a relevant CACE effect
The ITT estimate can be used to test the sharp null hypothesis
of no causal effect (no effect will be declared when no effect
exists)
24
The conservativeness of the ITT effect makes it a dangerous
effect measure when the goal is evaluating a treatment’s
safety (in fact, the ITT could be low because individuals assigned to new
drug stopped taking it before developing serious adverse effects)
The ITT effect is conservative in placebo-controlled trials, but it
is not in experiments comparing two active treatments
Since ITT = CACE pr(complier), the ITT effect mixes two
components: the causal effect of interest and the compliance
rate the ITT has a lower external validity than CACE

25
Generalizing the ITT
The ITT summarizes the overall effect of an intervention it is
important to generalize the ITT to other hypothetical
experiments
Since ITT = CACE pr(complier), generalizing the ITT entails
generalizing the CACE and the compliance rate separately
generalizing the CACE poses the usual difficulties for generalizing a
causal effect (e.g. the effect may be influenced by the type of
implementation, or by the features of the target population)
the compliance rate may be rather different in another experiment (e.g.
applying the intervention to a wider population usually entails a lower
compliance rate; e.g. if the treatment has proved to be effective, in
future experiments there will likely be a higher compliance rate);
moreover, increasing the compliance rate could be a feasible policy
goal
26
Instrumental variables (IV)
A variable Z is called instrument if it
satisfies two conditions:
• Relevance: Z affects the endogenous
regressor D (hence the arrow Z D)
• Exclusion restriction: Z affects Y only
through D (the only path from Z to Y
passes through D)
The idea underlying Instrumental Variables is that the
relationship between the outcome Y and the
endogenous regressor D (affected by omitted
variables U) can be estimated without bias by
exploiting the part of variability of D that is
uncorrelated with the omitted variables, namely the
variability of D induced by the instrument Z
Z
OLS
U
D Y
cov( Y, D) cov( Y, D)
var( D) cov( DD , )
IV
cov( Y, Z)
cov( D, Z)
CACE and instrumental variables (IV)
Natural experiments
27
The CACE (or LATE) estimator is equal to the IV estimator
Indeed, when the IV variable is binary the IV estimator of the
effect of D on Y has the following expression (Wald)
obs
obs obs
cov( Yi
, Z ) Y
i [ Z1] Y[ Z0]
IV CACE
cov( obs
obs obs
D , Z ) D D
i
i
[ Z1] [ Z0]
The IV estimator has been introduced in Econometrics a long
time ago (Philip G. Wright, 1928), but the interpretation in terms
of potential outcomes is recent (Imbens & Angrist, 1994)
Under the assumptions (i) SUTVA, (ii) exclusion restriction and
(iii) strong monotonicity, the IV estimator is estimating the CACE,
a quantity with a valid casual interpretation
28
Instrumental variables are generated by randomized
experiments and also by natural experiments, namely
“situations where the forces of nature or government policy
have conspired to produce an environment somewhat akin to a
randomized experiment.” (Angrist & Kruger, 2001)
An example of natural experiment is the lottery for draft for
the U.S. army (Angrist, Imbens, Rubin 1996) illustrated later
Imbens, Guido W. and Joshua D. Angrist (1994). Identification and Estimation of Local
Average Treatment Effects. Econometrica. 62(2), 467-75.
Angrist, J. and A. Krueger (2001) Instrumental variables and the search for
identification: From supply and demand to natural experiments, Journal of Economic
Perspectives, 15(4), 69–85.

Looking for instrumental variables
29
Angrist & Kruger (2001) wrote:
A good instrument is correlated with the endogenous regressor
for reasons the researcher can verify and explain, but
uncorrelated with the outcome variable for reasons beyond its
effect on the endogenous regressor.
Maddala (1977, p. 154) rightfully asks, "Where do you get
such a variable?" Like most econometrics texts, he does not
provide an answer.
In our view, good instruments often come from detailed
knowledge of the economic mechanism and institutions
determining the regressor of interest.
30
Examples
• Artificial example
• Lottery for draft – U.S. army
31
Traditional (and biased) estimators
As-treated estimator:
Per-protocol estimator:
Y
Y
Y
obs obs
obs
[ D 1] obs
[ D 0]
Y
[ Z1, D 1] [ Z0, D 0]
Those estimators are very simple but generally biased: in fact,
they condition on D, which is a post-treatment variable
WARNING: Unconfoundedness ensures the balancing across
treatment groups for pre-treatment variables but not for posttreatment
variables
obs
obs
obs
obs
Example
Hypothetical experiment with random assignment Pr(Z=1)=0.5 and (i) SUTVA, (ii)
exclusion restriction and (iii) strong monotonicity
Percentages of groups
Mean outcome
Assigned Control arm Treatment arm Control arm Treatment arm
Population Taken Control Treatment Control Treatment Control Treatment Control Treatment
50 Compliers 25 25 500 600
30 Never Takers 15 15 480 480
20 Always Takers 10 10 550 550
100 Observed 40 10 15 35 492.5 550.0 480.0 585.7
50 50
504.0 554.0
obs obs
Mean of treated 577.8
As-treated: Y obs Y
obs
[ D 1] [ D 0]
Mean of control 489.1
Per-protocol:
As-treated effect 88.7
obs
obs
Y obs Y
Per-protocol effect 93.2
obs
[ Z1, D 1] [ Z0, D 0]
Intention-to-treat effect 50.0
obs obs
Proportion of compliers 0.5
Intention-to-treat: Y[ Z1] Y[ Z0]
CACE 100.0
ITT is an underestimate of CACE (lower proportion of compliers higher bias)
The As-treated and Per-protocol effects are over- or under-estimates of CACE
depending on the mean outcome of Never Takers and Always Takers (see next ex.)

Example (mean outcome AT: from 550 to 650)
Hypothetical experiment with random assignment Pr(Z=1)=0.5 and (i) SUTVA, (ii)
exclusion restriction and (iii) strong monotonicity
Percentages of groups
Mean outcome
Assigned Control arm Treatment arm Control arm Treatment arm
Population Taken Control Treatment Control Treatment Control Treatment Control Treatment
50 Compliers 25 25 500 600
30 Never Takers 15 15 480 480
20 Always Takers 10 10 650 650
100 Observed 40 10 15 35 492.5 650.0 480.0 614.3
50 50
524.0 574.0
As-treated:
Per-protocol:
Intention-to-treat:
Y
Y
Y
Y
obs obs
obs
obs
[ D 1] [ D 0]
obs
obs
obs Y
obs
[ Z1, D 1] [ Z0, D 0]
obs obs
[ Z1] Y[ Z0]
Mean of treated 622.2
Mean of control 489.1
As-treated effect 133.1
Per-protocol effect 121.8
Intention-to-treat effect 50.0
Proportion of compliers 0.5
CACE 100.0
34
Example: lottery for draft – U.S. army /1
AIR 1996: Angrist J., Imbens G.W, Rubin D.B., 1996. Identification of Causal
Effects using Instrumental Variables. JASA 91, 444-472.
In the years1970-1973, draft to U.S. army for Vietnam war was based on
a lottery:
For any cohort, each date of birth was randomly assigned a number
from 1 to 365
Males with a number lower than a certain threshold were drafted, e.g.
males born in 1950 were called to be drafted until the number 195
Variables:
Z = 1 if the individual had a number lower than the threshold (i.e. he
was called to be drafted)
D = 1 if the individual actually drafted
Y = 1 if the individual died in the period 1974-1983 (civilian mortality,
i.e. for causes not directly related to war)
Example: lottery for draft – U.S. army /2
Estimates and comments from AIR 1996 p 453
35
The assumptions for a valid causal interpretation of the IV estimand are
(AIR 1996 p 452):
SUTVA: The veteran status of any man at risk of being drafted in the
lottery was not affected by the draft status of others at risk of being
drafted, and, similarly, that the civilian mortality of any such man was
not affected by the draft status of others
Ignorable Assignment: Assignment of draft status was random
Exclusion restriction: Civilian mortality risk was not affected by draft
status once veteran status is taken into account
Nonzero Average Causal Effect of Z on D: Having a low lottery
number increases the average probability of service
Monotonicity assumption: There is no one who would have served if
given a high lottery, but not if given a low lottery number
Some men with low lottery numbers changed their educational plans so as to retain draft
deferments and avoid the conscription. If so, then the exclusion restriction could be violated, because
draft status may have affected civilian outcomes through channels other than veteran status.
(men born in 1950)
Of the men with low lottery numbers (Z = 1), 35.3% actually served in the military. Of those
who had high lottery numbers (Z = 0), only 19.3% served in the military. Random assignment of
draft status suggests that draft status had a causal effect that increased the probability of
serving by an estimated 15.9% on average.
Similarly, of those with low lottery numbers, 2.04% died between 1974 and 1983, compared
to 1.95% of those who had high lottery numbers. The difference of 0.09% can be interpreted
as an estimate of the average causal effect of draft status on civilian mortality.
Assuming that these estimated causal effects are population averages, the ratio of these two
causal effects of draft status is, under the Assumptions 1-5, the causal effect of military service
on civilian mortality for the 15.9% who were induced by the draft to serve in the military. For
this group, the average causal effect is 0.56%.

Example: lottery for draft – U.S. army /4
Further references
37
38
This application highlights the fact that the IV
estimator does not require observations on
individuals; sample averages of outcomes and
treatment indicators by values of the instruments are
sufficient
In addition, in this application, the sample averages
for computing the IV estimator are drawn from
different data sets
Jo, B. (2002) Estimation of intervention effects with noncompliance: Alternative
model specifications. JEBS, 27, 385–409.
Mealli F., G. Imbens, S. Ferro, A. Biggeri (2004) Analyzing a Randomized Trial on
Breast Self Examination with Noncompliance and Missing Outcomes, Biostatistics, 5,
2, 207-222.
Jo B., Asparouhov T. & Muthén B. (2008) Intention-to-treat analysis in cluster
randomized trials with noncompliance. Statistics in Medicine, 27, 5565–5577
Little, R., Long, Q., & Lin, X. (2009) A comparison of methods for estimating the
causal effect of a treatment in randomized clinical trials subject to noncompliance.
Biometrics, 65, 640–649.
Schochet P.Z. & Chiang H.S. (2011) Estimation and Identification of the Complier
Average Causal Effect Parameter in Education RCTs. JEBS, 36, pp. 307–345.
Frumento P., Mealli F., Pacini B., and Rubin D.B. (2012) Evaluating the Effect of
Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing
Outcome Data, Journal of the American Statistical Association, 107, 450-466.