Political Analysis Advance Access originally published online on September 16, 2005
Political Analysis 2006 14(1):37-62; doi:10.1093/pan/mpj001
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
Comparing Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment
Department of Political Science, Temple University, 453 Gladfelter Hall, 1115 West Berks Street, Philadelphia, PA 19122
Yale University, Institution for Social and Policy Studies, P.O. Box 208209, 77 Prospect Street, New Haven, CT 06520
e-mail: kevin.arceneaux{at}temple.edu (corresponding author)
e-mail: alan.gerber{at}yale.edu
e-mail: donald.green{at}yale.edu
In the social sciences, randomized experimentation is the optimal research design for establishing causation. However, for a number of practical reasons, researchers are sometimes unable to conduct experiments and must rely on observational data. In an effort to develop estimators that can approximate experimental results using observational data, scholars have given increasing attention to matching. In this article, we test the performance of matching by gauging the success with which matching approximates experimental results. The voter mobilization experiment presented here comprises a large number of observations (60,000 randomly assigned to the treatment group and nearly two million assigned to the control group) and a rich set of covariates. This study is analyzed in two ways. The first method, instrumental variables estimation, takes advantage of random assignment in order to produce consistent estimates. The second method, matching estimation, ignores random assignment and analyzes the data as though they were nonexperimental. Matching is found to produce biased results in this application because even a rich set of covariates is insufficient to control for preexisting differences between the treatment and control group. Matching, in fact, produces estimates that are no more accurate than those generated by ordinary least squares regression. The experimental findings show that brief paid get-out-the-vote phone calls do not increase turnout, while matching and regression show a large and significant effect.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. E. McNulty, C. M. Dowling, and M. H. Ariotti Driving Saints to Sin: How Increasing the Difficulty of Voting Dissuades Even the Most Motivated Voters Political Analysis, October 1, 2009; 17(4): 435 - 455. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. A. Freedman and R. A. Berk Weighting Regressions by Propensity Scores Eval Rev, August 1, 2008; 32(4): 392 - 409. [Abstract] [PDF]
|
|
|
|
 |
|
 T. Dunning Improving Causal Inference: Strengths and Limitations of Natural Experiments Political Research Quarterly, June 1, 2008; 61(2): 282 - 293. [Abstract] [PDF]
|
|
|
|
|
|
D. A. Freedman Statistical Models for Causation: What Inferential Leverage Do They Provide? Eval Rev, December 1, 2006; 30(6): 691 - 713. [Abstract] [PDF]
|
|