On the Identification of Misspecified Propensity Scores - School of ...

More documents

Recommendations

Info

1 Introduction Propensity score matching is a widely used approach for the estimation of treatment effects. See Heckman, LaLonde, and Smith [7] for a detailed survey of its use in the treatment effect literature. The method is based on the following well-known result of Rosenbaum and Rubin [13]: if selection into the program is independent of the potential outcomes of interest conditional on a vector of covariates, then selection into the program is also independent of the potential outcomes conditional on the propensity score, where the propensity score denotes the probability of selection into the program conditional on the same vector of covariates. The high dimensionality of the vector of observed covariates often forces researchers to adopt a finite- dimensional approximation to the propensity score, in lieu of a more flexible nonparametric model. Imposing such restrictions on the functional form for the propensity score raises the possibility that the propensity score model is misspecified. If the propensity score model is misspecified, then in general the estimator of the propensity score will be inconsistent for the true propensity score and the resulting propensity score matching estimator may be inconsistent for the treatment effect of interest. First suggested by Heckman, Ichimura, Smith, and Todd [5], it is now common to calculate estimates of the densities of the propensity score conditional on the participation decision. These estimates are calculated in order to determine the region of common support on which to perform matching. We show that these conditional densities provide a means for examining whether the propensity score is misspecified. In particular, we derive a restriction between the density of the propensity score model among participants and the density among nonpar- ticipants. Failure of this restriction to hold for the estimated conditional densities provides evidence that the propensity score model is misspecified. We show that this restriction between the two conditional densities is equivalent to a particular orthogonality restriction and derive a formal test based upon it. Unlike other tests for correct specification of the propensity score, the resulting test does not require nonparametric estimation of high dimensional objects. As a result, our test does not suffer from the curse of dimensionality and has dramatically greater power than competing tests for many alternatives. We provide striking evidence of this fact in a limited Monte Carlo study. The principle drawback of this approach is that our test does not have power against certain alternatives, but we argue that these alternatives are rather exceptional. Following [12], applied researchers often employ heuristic balancing score tests for misspecification of the propensity score to guide the choice of specification of the parametric propensity score. Specifically, [12] suggests examining whether the observable characteristics of the pop- ulation are independent of participation conditional on the propensity score. See e.g. [3], [8], [15], and [17] for applications of such balancing score tests. The formal misspecification test 2
presented in this paper may be considered a complement to this practice. Our paper proceeds as follows. In Section 2, we first provide a summary of the method of propensity score matching. We then present in Section 3 the restriction upon which our test is based. In Section 4, we use this result to derive an asymptotic test of misspecification and examine its finite sample properties in a Monte Carlo study. Finally, Section 5 concludes. 2 A Review of Matching Before proceeding, it is useful to review the key theoretical underpinnings of matching as a means of program evaluation. There are two groups of individuals, participants and nonpar- ticipants, in a program of interest. Participation in the program of interest is denoted by the dummy variable D, with D = 1 if the individual chooses to participate and D = 0 otherwise. Individuals in each of these two groups are associated with observed characteristics X. Two commonly used metrics for evaluating the effect of participation in a program are the average treatment effect, given by E[Y1 −Y0], and the average treatment effect on the treated, given by E[Y1 − Y0|D = 1], where Y1 is the potential outcome in the case of participation and Y0 is the potential outcome in the case of nonparticipation. The difficulty with estimation of this object lies with the following missing data problem: the econometrician never observes the counter- factual outcome Y1−D, which precludes direct estimation of E[Y0|D = 1] and E[Y1|D = 0]. The method of matching resolves this difficulty by matching each participant with a non- participant that is similar in terms of observed characteristics X. As described in Rosenbaum and Rubin [13], matching formally requires that: A1. (Y0, Y1) ⊥ D | X. A2. 0 of A2 is that E[Y0|D = 1, X = x] and E[Y1|D = 0, X = x] are well defined for all x ∈ supp(X). Hence, matching suggests estimation of E[Y0|D = 1] and E[Y1|D = 0] by a two-step procedure in which E[Y0|D = 1, X] and E[Y1|D = 0, X] are first estimated by exploiting A1, and then integrated with respect to the empirical distribution of X in order to obtain an estimate of E[Y1 − Y0] or integrated with respect to the empirical distribution of X conditional on D = 1 to obtain an estimate of E[Y1 − Y0|D = 1]. Following Heckman, Ichimura, Smith, and Todd [5], it is clear that one can relax the first condition to only require mean independence instead of full independence. Using this procedure, it is in 3
Page 1: On the Identification of Misspecifi
Page 5 and 6: 3 The Restriction In order to ensur
Page 7 and 8: that misspecification is not very s
Page 9 and 10: where ˆεi := Yi − Q(Xi, ˆ θn)
Page 11 and 12: Throughout the simulations presente
Page 13 and 14: 5 Conclusion [TABLE 8 ABOUT HERE] [
Page 15 and 16: Ichimura, and Todd [6]. In addition
Page 17 and 18: To check the second condition of th
Page 19 and 20: where M = sup θ∈Θ[D−Q(x, θ)]
Page 21 and 22: [15] S, B. (2004): “An Evaluation
Page 23 and 24: FIGURE 2 E D P S Note: ----- ˆα
Page 25 and 26: TABLE 2: P c=0.05 Proportion of Rej

On the Identification of Misspecified Propensity Scores - School of ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?