MATCHINGWhen and how?

OUTLINE FOR TODAYWhat if no randomization?Matching set-up and assumptionsThe dimensionality problem and propensity scorematchingPropensity score matching estimationWrap-up and next class

MATCHINGAnother possibility is if the assignment to the treatmentis done not randomly, but on the basis of observablesMatching methods allow you to construct comparisongroups when the assignment to the treatment is doneon the basis of observable variables.

VERY MAJOR CAVEATWarning: Matching STILL does not allow to control forselection bias that arises when the assignment to thetreatment is done on the basis of non-observables.

KEY QUESTION HEREWhat is the effect of treatment on the treatedwhen the assignment to the treatment isbased on observable variables?

= E[Y 1 (u)|D = 1] E[Y 0 (u)|D = 1]UNCONFOUNDEDNESSˆ¯d =[ˆ¯ Y 1 |D = 1][ Y ˆ¯ 0 |D = 0]AKA conditional independenceLet X denote a matrix in which each row is a vector of[Y 0 (u)|D = 1]=[Y 0 (u)|D = 0]pre-treatment observable variables for individual iatchingAssignment to treatment is unconfoundedgiven pre-treatment variables X if(Y Ti ,Y Ci ) ? T i |X i

IN WORDSUnconfoundedness is equivalent to saying that:(1) within each cell defined by X: treatment is random(2) the selection into treatment dependsonly on the observables X

[Y 0 (u)|D = 1]=[Y 0 (u)|D = 0]matchingATE VS TOTUnconfoundedness: Allows you to estimate ATE(Yi T ,Yi C ) ? T i |X iTo estimate TOT we need a weaker assumption:Y Ci? T i |X i

ONE APPROACHUnconfoundedness suggests the following strategy forthe estimation of the average treatment effect !:Stratify the data into cells defined byeach particular value of XWithin each cell (i.e. conditioning on X) computethe difference between the average outcomesof the treated and the controlsAverage these differences with respect to thedistribution of X in the population of treated units.

WHAT IS THE PROBLEM?The Dimensionality ProblemMethod will not work if:The sample is smallThe set of covariates is largeMany of the covariates have manyvalues or are continuous

LACK OF COMMONSUPPORTExamples of dimensionality problem:How many cells do we have with 2 binary X variables?And with 3 binary X variables?And with K binary X variables?How about if we have 2 variables that take 7 values each?As the number of cells grows, we’ll getlack of common support:cells containing onlytreated observationscells containing onlycontrol observations

PROPENSITY SCOREMATCHING“The propensity score allows to convert themultidimensional setup of matching intoa one-dimensional setup.In that way, it allows to reduce thedimensionality problem.”Rosenbaum and Rubin (1983)

matchingMATCHING USING PSM(Y Ti ,Y CDefinition: The propensity score is theYiCi ) ? T i |X i? T i |X iconditional probability of receiving thetreatment given the pre-treatment variables:P(X)=Pr(T = 1|X)

YiC? T i |X iLEMMAS ON PSMP(X)=Pr(T = 1|X)Lemma 1:If p(X) is the propensity score, thenT ? X|P(X)(Y T “Given ,Y C ) ? the T |X propensity ) (Y T ,Yscore, C ) ? the T |P(X) pretreatmentvariables are balanced betweenbeneficiaries and non- beneficiaries”1

P(X)=Pr(T = 1|X)LEMMAS ON PSMT ? X|P(X)Lemma 2:(Y T ,Y C ) ? T |X ) (Y T ,Y C ) ? T |P(X)“Suppose that assignment to treatment isunconfounded given the pre-treatment variablesX. Then assignment to treatment isunconfounded given the propensity score p(X).”1

SOLVING DIMENSIONALITYPROBLEMThe balancing property of the propensityscore (Lemma 1) ensures that:Observations with the same propensity scorehave the same distribution of observablecovariates independently of treatment status; andfor a given propensity score, assignment to treatmentis “random” and therefore treatment and controlunits are observationally identical on average

IMPLEMENTATIONOUTLINEStrategy for the estimation ofthe average treatment effect !Step 1: Estimate the propensity scoreThis step is necessary because the “true”propensity score is unknown and thereforethe propensity score has to be estimatedStep 2: Estimate the average treatmenteffect given the propensity score

COMMON SUPPORTAGAINIdea behind propensity score matching:estimation of treatment effects requires acareful matching of treated and controls.If treated and controls are very different in termsof observables this matching is not sufficientlyclose and reliable or it may even be impossibleThe comparison of the estimated propensity scoresacross treated and controls provides a useful diagnostictool to evaluate how similar are treated and controls.

Propensity Score MatchingFigure 4.1 Example of Common SupportDensityDensity of scoresfor nonparticipantsDensity of scoresfor participantsGoodDensitySource: Authors’ representation.Propensity score0 Region of common support1Figure 4.2 Example of Poor Balancing and Weak Common SupportPropensity score0 Region of common support1Source: Authors’ representation.Density of scoresfor nonparticipantsDensity of scoresfor participantsBadFigure 4.2 Example of Poor Balancing and Weak Common SupportDensityDensity of scoresfor nonparticipantsDensity of scoresfor participantsDensityPropensity score0 Region of common support1Source: Authors’ representation.Propensity score

HOW TO CHECK FORSUPPORT?The range of variation of propensity scoresshould be the same for treated and controlsCount how many controls have a propensity scorelower than the minimum or higher than the maximumof the propensity scores of the treated and vice versa.Frequency of propensity scores isthe same for treated and controlDraw histograms of the estimated propensityscores for the treated and controlsThe bins correspond to the blocks constructedfor the estimation of propensity scores

IMPLEMENTATIONOUTLINE (IN PRINCIPLE)Strategy for the estimation ofthe average treatment effect !Step 1: Estimate the propensity scoreStep 2: Estimate the average treatmenteffect given the propensity scorematch treated and controls with exactlythe same (estimated) propensity scorecompute the effect of treatment for eachvalue of the (estimated) propensity scoreobtain the average of these conditional effects

IN PRACTICEUnfeasible in practice because it is rare to findtwo units with exactly the same propensity scoreThe closest we can get to an exact matchingis to match each treated unit with thenearest control in terms of propensity score

WHAT IS NEAREST?“Nearest” can be defined in many ways.These different ways correspondence todifferent ways of doing matching:Nearest neighbor matching on the ScoreStratification on the ScoreRadius matching on the ScoreKernel matching on the ScoreWeighting on the basis of the Score

THINGS TO KEEP IN MINDMatching requires large samples and good quality dataMatching at baseline can be very useful:Know the assignment rule and match based on itcombine with other techniques (i.e. diff-in-diff)Ex-post matching is risky:If there is no baseline, be careful!Matching on endogenous ex-post variablesgives bad results.

SUMMARY OF STEPSCollect representative & highly comparable survey of non-participants andparticipantsPool the two samples and estimated a logit (or probit) model of programparticipationRestrict samples to assure common support (important source of bias inobservational studies)For each participant find a sample of non-participants that have similarpropensity scoresCompare the outcome indicators. The difference is the estimate of the gain dueto the program for that observation.Calculate the mean of these individual gains to obtain the average overall gain.

NEXT TIMEGo over problem set 1Discuss Jalan and Ravallion (2003). “Does PipedWater Reduce Diarrhea for Children in Rural India?”