Political Analysis Advance Access originally published online on November 2, 2005
Political Analysis 2006 14(2):131-159; doi:10.1093/pan/mpj004
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
The Dangers of Extreme Counterfactuals
Department of Government and Institute for Quantitative Social Science, Harvard University, 1737 Cambridge Street, Cambridge MA 02138
Department of Political Science, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0521
e-mail: king{at}harvard.edu
e-mail: zeng{at}ucsd.edu
We address the problem that occurs when inferences about counterfactuals—predictions, "what-if" questions, and causal effects—are attempted far from the available data. The danger of these extreme counterfactuals is that substantive conclusions drawn from statistical models that fit the data well turn out to be based largely on speculation hidden in convenient modeling assumptions that few would be willing to defend. Yet existing statistical strategies provide few reliable means of identifying extreme counterfactuals. We offer a proof that inferences farther from the data allow more model dependence and then develop easy-to-apply methods to evaluate how model dependent our answers would be to specified counterfactuals. These methods require neither sensitivity testing over specified classes of models nor evaluating any specific modeling assumptions. If an analysis fails the simple tests we offer, then we know that substantive results are sensitive to at least some modeling choices that are not based on empirical evidence. Free software that accompanies this article implements all the methods developed.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. Barabas Not the Next IRA: How Health Savings Accounts Shape Public Opinion Journal of Health Politics Policy and Law, April 1, 2009; 34(2): 181 - 217. [Abstract] [PDF]
|
|
|
|
 |
|
 G. King and L. Zeng Empirical versus Theoretical Claims about Extreme Counterfactuals: A Response Political Analysis, January 15, 2009; (2009) mpn010v3. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Sambanis and A. Michaelides A Comment on Diagnostic Tools for Counterfactual Inference Political Analysis, January 1, 2009; 17(1): 89 - 106. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Colaresi and S. C. Carey To Kill or to Protect: Security Forces, Domestic Institutions, and Genocide Journal of Conflict Resolution, February 1, 2008; 52(1): 39 - 67. [Abstract] [PDF]
|
|
|
|
|
|
D. E. Ho, K. Imai, G. King, and E. A. Stuart Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference Political Analysis, July 1, 2007; 15(3): 199 - 236. [Abstract] [Full Text] [PDF]
|
|