Propensity Score Matching - Statpower

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExamplePropensity Score MatchingJames H. SteigerDepartment of Psychology and Human DevelopmentVanderbilt UniversityMultilevel Regression Modeling, 2009MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExamplePropensity Score Matching1 Introduction2 Modeling the Covariates3 Subclassification4 MatchingIntroductionWhy Match?5 Balancing ScoresDefinitionMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMatching and SubclassificationIn previous discussions, we learned about selection bias and, inparticular, the dangers of attempting to control forpost-treatment covariates while assessing causality.Near the end of Chapter 10, Gelman & Hill discuss the methodsof matching and subclassification as aids to causal inference inobservational studies.The basic idea behind the methods is that, if you can identifyrelevant covariates so that ignorability is reasonable, you canassess causality by controlling for the covariates statistically.Such control can take several forms:You can examine conditional distributions, conditionalizedon classifications on the covariate(s).You can match treatment and controls, and comparematched groupsYou can model the covariates along with the treatmentMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMatching and SubclassificationIn previous discussions, we learned about selection bias and, inparticular, the dangers of attempting to control forpost-treatment covariates while assessing causality.Near the end of Chapter 10, Gelman & Hill discuss the methodsof matching and subclassification as aids to causal inference inobservational studies.The basic idea behind the methods is that, if you can identifyrelevant covariates so that ignorability is reasonable, you canassess causality by controlling for the covariates statistically.Such control can take several forms:You can examine conditional distributions, conditionalizedon classifications on the covariate(s).You can match treatment and controls, and comparematched groupsYou can model the covariates along with the treatmentMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMatching and SubclassificationIn previous discussions, we learned about selection bias and, inparticular, the dangers of attempting to control forpost-treatment covariates while assessing causality.Near the end of Chapter 10, Gelman & Hill discuss the methodsof matching and subclassification as aids to causal inference inobservational studies.The basic idea behind the methods is that, if you can identifyrelevant covariates so that ignorability is reasonable, you canassess causality by controlling for the covariates statistically.Such control can take several forms:You can examine conditional distributions, conditionalizedon classifications on the covariate(s).You can match treatment and controls, and comparematched groupsYou can model the covariates along with the treatmentMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleModeling the CovariatesThe problem with modeling the covariates is that, dependinghow influential the covariates are,with even minor modelmisspecification the estimate of the effect of the treatment maybe seriously biased.Since ignorability requires that all relevant covariates beaccounted for, the “curse of dimensionality” quickly becomes afactor. A huge number of models is conceivable, and so thelikelihood of misspecification is high.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleSubclassification on a Single CovariateGelman & Hill (p. 204) illustrate subclassification with a simpleexample.Suppose that the effectiveness of an educational intervention forimproving kids’ test scores was investigated in an observationalsetting where mothers chose whether or not to have theirchildren participate, and randomization was not possible.Selection bias is a fundamental problem in such a study.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleSubclassification on a Single CovariateSelection bias occurs when the treatment condition (e.g.,experimental vs. control) of a participant is not independent ofconfounding covariates which are also correlated with theoutcome.For example, if mothers’ high achievement motivation causesthem to select into the experimental group, and also causesthem to react to their children in a way that affects theoutcome, then the results of the study will be biased.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleSubclassification on a Single CovariateSuppose, for the sake of argument, that there is only oneconfounding covariate in the study, and it is the level ofeducation of the mother.One way of controlling for the impact of this covariate is tocreate subclassifications, within which the covariate has thesame value in experimental treatment and control groups.Here are some illustrative data from Gelman & Hill .MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDifficulties with SubclassificationSubclassification has advantages:It forces overlapIt imposes roughly the same covariate distribution withinsubclassesHowever, it has disadvantages as well:When categorizing a continuous covariate, someinformation will be lostThe strategy is very difficult to implement with severalcovariates at onceMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDifficulties with SubclassificationSubclassification has advantages:It forces overlapIt imposes roughly the same covariate distribution withinsubclassesHowever, it has disadvantages as well:When categorizing a continuous covariate, someinformation will be lostThe strategy is very difficult to implement with severalcovariates at onceMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDifficulties with SubclassificationSubclassification has advantages:It forces overlapIt imposes roughly the same covariate distribution withinsubclassesHowever, it has disadvantages as well:When categorizing a continuous covariate, someinformation will be lostThe strategy is very difficult to implement with severalcovariates at onceMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDifficulties with SubclassificationSubclassification has advantages:It forces overlapIt imposes roughly the same covariate distribution withinsubclassesHowever, it has disadvantages as well:When categorizing a continuous covariate, someinformation will be lostThe strategy is very difficult to implement with severalcovariates at onceMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMatchingIntroductionWhy Match?Matching refers to a variety of procedures that restrict andreorganize the original sample in preparation for a statisticalanalysis. The simplest version is to do one-to-one matching,with each treatment observation being paired with a controlthat is as much like it as possible on the relevant covariates.With multiple covariates, matching can be done on a “nearestneighbor” basis. For example, a treatment observation ismatched according to the minimum Mahalanobis distance,which is, for two vectors of scores x (1) and x (2) on thecovariates,(x (1) − x (2) ) ′ Σ −1 (x (1) − x (2) ) (2)MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleWhy Match?IntroductionWhy Match?MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleBalancing ScoresDefinitionCoarseness and FinenessRosenbaum and Rubin(1983) introduced the notion of abalancing score, a function of the covariates that may be used inplace of all the covariates to achieve balancing.Balancing ScoreGiven treatment T and one or more covariates in X , abalancing score b(X ) satisfies the condition that the conditionaldistribution of X given b(X ) is the same for treated (T = 1)and control (T = 0), that isX ⊥T | b(X ) (3)MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleBalancing ScoresDefinitionCoarseness and FinenessThere are many possible balancing scores. For example, X itselfis a balancing score.Balancing scores can be characterized in terms of coarseness orfineness.Coarseness — Fineness of a Balancing ScoreA balancing score a(x) is said to be coarser than balancingscore b(x) if a(x) = f (b(x)) for some function f . In such a case,we can also say that b(x) is finer than a(x).MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleThe Propensity ScoreDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsRosenbaum and Rubin (1983) concentrate on a particularbalancing score, the propensity score.The Propensity ScoreGiven a treatment T and a set of covariates X , the propensityscore e(x) is defined ase(x) = Pr(T = 1|X = x) (4)MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleThe Strong Ignorability AssumptionDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsIn deriving the key optimalit property of propensity scoresRosenbaum and Rubin (1983) assume strong ignorability of Tgiven X .Strong IgnorabilityThe property of strong ignorability of T given X holds if, forpotential outcomes y 1 and y 0 , the distribution of these potentialoutcomes is conditionally independent of T given X , and forany value of the covariates, there is a possibility of a unitreceiving the treatment or not receiving the treatment. That is,(y 1 , y 0 )⊥T |X (5)and0 < Pr(T = 1|X = x) < 1 ∀x (6)MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMathematical PropertiesDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsRosenbaum and Rubin (1983, p. 43–44) proved the followingtheorems:1 The propensity score is a balancing score2 Any score that is “finer” than the propensity score is abalancing score; moreover, X is the finest balancing scoreand the propensity score is the coarsest3 If treatment assignment is strongly ignorable given X , thenit is strongly ignorable given any balancing score b(X )4 At any given value of a balancing score, the differencebetween the treatment and control means is an unbiasedestimate of the average treatment effect at that value of thebalancing score if treatment assignment is stronglyignorable. Consequently, with strongly ignorable treatmentassignment, pair matching on a balancing score,subclassification on a balancing score and covarianceadjustment on a balancing score can all produce unbiasedestimates of treatment effects,5 Using sample estimates of balancing scores can producesample balance on X .MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMathematical PropertiesDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsRosenbaum and Rubin (1983, p. 43–44) proved the followingtheorems:1 The propensity score is a balancing score2 Any score that is “finer” than the propensity score is abalancing score; moreover, X is the finest balancing scoreand the propensity score is the coarsest3 If treatment assignment is strongly ignorable given X , thenit is strongly ignorable given any balancing score b(X )4 At any given value of a balancing score, the differencebetween the treatment and control means is an unbiasedestimate of the average treatment effect at that value of thebalancing score if treatment assignment is stronglyignorable. Consequently, with strongly ignorable treatmentassignment, pair matching on a balancing score,subclassification on a balancing score and covarianceadjustment on a balancing score can all produce unbiasedestimates of treatment effects,5 Using sample estimates of balancing scores can producesample balance on X .MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMathematical PropertiesDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsRosenbaum and Rubin (1983, p. 43–44) proved the followingtheorems:1 The propensity score is a balancing score2 Any score that is “finer” than the propensity score is abalancing score; moreover, X is the finest balancing scoreand the propensity score is the coarsest3 If treatment assignment is strongly ignorable given X , thenit is strongly ignorable given any balancing score b(X )4 At any given value of a balancing score, the differencebetween the treatment and control means is an unbiasedestimate of the average treatment effect at that value of thebalancing score if treatment assignment is stronglyignorable. Consequently, with strongly ignorable treatmentassignment, pair matching on a balancing score,subclassification on a balancing score and covarianceadjustment on a balancing score can all produce unbiasedestimates of treatment effects,5 Using sample estimates of balancing scores can producesample balance on X .MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMathematical PropertiesDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsRosenbaum and Rubin (1983, p. 43–44) proved the followingtheorems:1 The propensity score is a balancing score2 Any score that is “finer” than the propensity score is abalancing score; moreover, X is the finest balancing scoreand the propensity score is the coarsest3 If treatment assignment is strongly ignorable given X , thenit is strongly ignorable given any balancing score b(X )4 At any given value of a balancing score, the differencebetween the treatment and control means is an unbiasedestimate of the average treatment effect at that value of thebalancing score if treatment assignment is stronglyignorable. Consequently, with strongly ignorable treatmentassignment, pair matching on a balancing score,subclassification on a balancing score and covarianceadjustment on a balancing score can all produce unbiasedestimates of treatment effects,5 Using sample estimates of balancing scores can producesample balance on X .MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMathematical PropertiesDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsRosenbaum and Rubin (1983, p. 43–44) proved the followingtheorems:1 The propensity score is a balancing score2 Any score that is “finer” than the propensity score is abalancing score; moreover, X is the finest balancing scoreand the propensity score is the coarsest3 If treatment assignment is strongly ignorable given X , thenit is strongly ignorable given any balancing score b(X )4 At any given value of a balancing score, the differencebetween the treatment and control means is an unbiasedestimate of the average treatment effect at that value of thebalancing score if treatment assignment is stronglyignorable. Consequently, with strongly ignorable treatmentassignment, pair matching on a balancing score,subclassification on a balancing score and covarianceadjustment on a balancing score can all produce unbiasedestimates of treatment effects,5 Using sample estimates of balancing scores can producesample balance on X .MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleKey ImplicationsDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsIf strong ignorability holds, and treatment and control groupsare matched perfectly on their propensity scores, then thedifference in means between treatment and control groups is anunbiased estimate of treatment effects.Moreover, subclassification or covariance adjustment can alsoyield unbiased treatment effects.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleKey QuestionsDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsPropensity scores have some nice properties that, in principle,seem to solve a very vexing problem. However, before jumpingon the very large propensity score bandwagon, we need to recall1 The propensity score is a parameter, i.e., a probability. Wenever know it precisely. We only know sample estimates ofit.2 Propensity scores are guaranteed to yield unbiased causaleffects only if strong ignorability holds.For now, let’s move on to a discussion of the practical aspects ofcalculating and using sample estimates of propensity scores.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleKey QuestionsDefinition of a Propensity ScoreKey AssumptionMathematical PropertiesKey ImplicationsKey QuestionsPropensity scores have some nice properties that, in principle,seem to solve a very vexing problem. However, before jumpingon the very large propensity score bandwagon, we need to recall1 The propensity score is a parameter, i.e., a probability. Wenever know it precisely. We only know sample estimates ofit.2 Propensity scores are guaranteed to yield unbiased causaleffects only if strong ignorability holds.For now, let’s move on to a discussion of the practical aspects ofcalculating and using sample estimates of propensity scores.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleIntroductionIntroductionHelpful SoftwareExact MatchingSubclassificationNearest Neighbor MatchingOptimal MatchingMany approaches to matching are possible, and quite a few areautomated in R packages.A key aspect of all of them is that you never use the outcomevariable during matching!We shall briefly discuss several methods, then illustrate themwith a computational example.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleMatchIt and ZeligIntroductionHelpful SoftwareExact MatchingSubclassificationNearest Neighbor MatchingOptimal MatchingGary King and his associates have been actively involved indeveloping software called MatchIt and Zelig to facilitatematching and the modeling process.Zelig subsumes a number of modeling procedures under acommon framework, making modeling a more user-friendlyexercise. It automates a number of useful methods for analyzingmodel fit. MatchIt, likewise, automates matching procedures,and provides methods for evaluating their success.After installing these two packages, you will have a number ofmatching procedures, and related analytic methods, at yourdisposal.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleExact MatchingIntroductionHelpful SoftwareExact MatchingSubclassificationNearest Neighbor MatchingOptimal MatchingThe simplest version of matching is exact.This technique matches each treated unit to all possible controlunits with exactly the same values on all the covariates, formingsubclasses such that within each subclass all units (treatmentand control) have the same covariate values.Exact matching is implemented in MatchIt usingmethod = "exact".MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleSubclassificationIntroductionHelpful SoftwareExact MatchingSubclassificationNearest Neighbor MatchingOptimal MatchingWhen there are many covariates (or some covariates can take alarge number of values), finding sufficient exact matches willoften be impossible.The goal of subclassification is to form subclasses, such that ineach the distribution (rather than the exact values) of covariatesfor the treated and control groups are as similar as possible.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleNearest Neighbor MatchingIntroductionHelpful SoftwareExact MatchingSubclassificationNearest Neighbor MatchingOptimal MatchingNearest neighbor matching selects the r (default=1) bestcontrol matches for each individual in the treatment group(excluding those discarded using the discard option).Matching is done using a distance measure specified by thedistance option (default=logit). Matches are chosen for eachtreated unit one at a time, with the order specified by them.order command (default=largest to smallest). At eachmatching step we choose the control unit that is not yetmatched but is closest to the treated unit on the distancemeasure. Nearest neighbor matching is implemented in MatchItusing the method = "nearest" option.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOptimal MatchingIntroductionHelpful SoftwareExact MatchingSubclassificationNearest Neighbor MatchingOptimal MatchingThe default nearest neighbor matching method in MatchIt is“greedy” matching, where the closest control match for eachtreated unit is chosen one at a time, without trying to minimizea global distance measure. In contrast, “optimal” matching findsthe matched samples with the smallest average absolutedistance across all the matched pairs.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleOverviewDeciding on Relevant CovariatesUsing propensity scores, in practice, involves several steps:1 Decide on the relevant covariates X2 Develop a model for predicting Pr(T = 1) from X3 Obtain sample propensity scores ê(x) from the model4 Use a matching procedure to obtain samples with T = 1and T = 0 that are matched on ê.5 Assess the success of the matching procedure6 If the matching procedure has not been successful, go backto step 2 and update the model, otherwise proceed7 Perform the desired parametric analysis on thepreprocessed (matched) dataMultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDeciding on Relevant CovariatesDeciding on Relevant CovariatesAll variables in X that would have been included in aparametric model without preprocessing should be included inthe matching procedure.To minimize omitted variable bias, these should “include allvariables that affect both the treatment assignment and,controlling for the treatment, the dependent variable.” (Ho,Imai, King, & Stuart,2007, p. 216) Keep in mind that, to avoidposttreatment bias, we should exclude variables affected by thetreatment.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDeciding on Relevant CovariatesDeciding on Relevant Covariates – A CautionAs Ho, Imai, King, and Stuart (2007, p. 216–217) point out, theemphasis in the literature has been to include a virtual grab-bagof all covariates deemed even slightly relevant, and users need tobe aware this point of view may be incorrect. The view is thatthis will decrease bias more than it will increase error variance.MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleDeciding on Relevant CovariatesDeciding on Relevant Covariates – A CautionA Caution“However, the theoretical literature has focused primarily on thecase where the pool of potential control units is considerablylarger than the set of treated units. Some researchers seem tohave incorrectly generalized this advice to all data sets. If, as isoften the case, the pool of potential control units is not muchlarger than the pool of treated units, then always including allavailable control variables is bad advice. Instead, the familiareconometric rules apply about the trade-off between the bias ofexcluding relevant variables and the inefficiency of includingirrelevant ones: researchers should not include everypretreatment covariate available.”MultilevelPropensity Score Matching

IntroductionModeling the CovariatesSubclassificationMatchingBalancing ScoresThe Propensity ScoreMatching MethodsUsing Propensity Scores – A General StrategyAn ExampleThe Lalonde DataThe Lalonde DataLalonde(1986) constructed an observational study in order tocompare it with an actual randomized study that had alreadybeen done. A homework question deals with this study inconsiderable detail.Gelman & Hill have one set of Lalonde data, while the MatchItlibrary has another. The MatchIt demos will use their versionof the data.MultilevelPropensity Score Matching