Political Analysis Advance Access originally published online on September 14, 2007
Political Analysis 2008 16(2):138-152; doi:10.1093/pan/mpm025
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches
Yale University, Department of Political Science, 77 Prospect Street, New Haven, CT 06520-8209 e-mail: donald.green{at}yale.edu
UCLA, Department of Political Science, 4289 Bunche Hall Box 951472, Los Angeles, CA 90095-1472
e-mail: lvavreck{at}ucla.edu (corresponding author)
Analysts of cluster-randomized field experiments have an array of estimation techniques to choose from. Using Monte Carlo simulation, we evaluate the properties of point estimates and standard errors (SEs) generated by ordinary least squares (OLS) as applied to both individual-level and cluster-level data. We also compare OLS to alternative random effects estimators, such as generalized least squares (GLS). Our simulations assess efficiency across a variety of scenarios involving varying sample sizes and numbers of clusters. Our results confirm that conventional OLS SEs are severely biased downward and that, for all estimators, gains in efficiency come mainly from increasing the number of clusters, not increasing the number of individuals within clusters. We find relatively minor differences across alternative estimation approaches, but GLS seems to enjoy a slight edge in terms of the efficiency of its point estimates and the accuracy of its SEs. We illustrate the application of alternative estimation approaches using a clustered experiment in which Rock the Vote TV advertisements were used to encourage young voters in 85 cable TV markets to vote in the 2004 presidential election.
Authors' note: We thank Rock the Vote for permission to use their public service announcements in this field experiment. The authors are grateful to Alan Gerber for suggestions throughout the design phase of this project. We are also grateful to Dan Kotin and Margaret Coblentz, who worked with cable operators, distributed the advertisements, and assembled the data. We thank Terence Leong for his programming expertise. Replication materials are available on the Political Analysis Web site.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. C. Andersen and P. B. Mortensen Policy Stability and Organizational Performance: Is There a Relationship? J. Public Adm. Res. Theory., May 14, 2009; (2009) mup005v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Arceneaux and D. W. Nickerson Modeling Certainty with Clustered Data: A Comparison of Methods Political Analysis, April 10, 2009; (2009) mpp004v1. [Abstract] [Full Text] [PDF]
|
|