Political Analysis - recent issues

Pass the Pork: Measuring Legislator Shares in Congress

Lauderdale, B. E. — 2008-07-10

Linear regression models are frequently used to analyze distributive politics in the U.S. Congress; however, authors have used a variety of specifications with different implicit assumptions about how bicameralism shapes legislative bargaining. I derive a model that describes district or state spending authorizations as the aggregation of spending secured by multiple legislators working on behalf of overlapping constituencies. This bicameral shares model allows the disaggregation of House and Senate influence through simultaneous estimation of the relative bargaining power of the two chambers and the advantages that accrue to legislators holding partisan, committee, and other relevant affiliations. In the 2005 transportation bill, the model better predicts the functional form of small state advantage than recently employed specifications in the literature.

Strategic Interaction and Interstate Crises: A Bayesian Quantal Response Estimator for Incomplete Information Games

Esarey, J., Mukherjee, B., Moore, W. H. — 2008-07-10

Private information characteristics like resolve and audience costs are powerful influences over strategic international behavior, especially crisis bargaining. As a consequence, states face asymmetric information when interacting with one another and will presumably try to learn about each others' private characteristics by observing each others' behavior. A satisfying statistical treatment would account for the existence of asymmetric information and model the learning process. This study develops a formal and statistical framework for incomplete information games that we term the Bayesian Quantal Response Equilibrium Model (BQRE model). Our BQRE model offers three advantages over existing work: it directly incorporates asymmetric information into the statistical model's structure, estimates the influence of private information characteristics on behavior, and mimics the temporal learning process that we believe takes place in international politics.

Modeling Committee Chair Selection in the U.S. House of Representatives

Cann, D. M. — 2008-07-10

For many years, committee chairs have been selected on the basis of seniority. Recent work has suggested that alternative factors, specifically financial support of party goals and party unity, have diminished the importance of seniority in committee chair selection. However, previous work has either failed to quantify these effects or has done so with inappropriate methods. This paper argues for the use of a Bayesian conditional logit estimator to correctly model committee chair selection in the U.S. House of Representatives. Results show a declining commitment to seniority throughout the Republican era and support the importance of fundraising as a determinant of committee chair selection. This paper shows that two other factors, financial support of party goals and party unity, have essentially replaced seniority as the central criteria for selecting committee chairs.

Model Specification in Instrumental-Variables Regression

Dunning, T. — 2008-07-10

In many applications of instrumental-variables regression, researchers seek to defend the plausibility of a key assumption: the instrumental variable is independent of the error term in a linear regression model. Although fulfilling this exogeneity criterion is necessary for a valid application of the instrumental-variables approach, it is not sufficient. In the regression context, the identification of causal effects depends not just on the exogeneity of the instrument but also on the validity of the underlying model. In this article, I focus on one feature of such models: the assumption that variation in the endogenous regressor that is related to the instrumental variable has the same effect as variation that is unrelated to the instrument. In many applications, this assumption may be quite strong, but relaxing it can limit our ability to estimate parameters of interest. After discussing two substantive examples, I develop analytic results (simulations are reported elsewhere). I also present a specification test that may be useful for determining the relevance of these issues in a given application.

Legislative Productivity of the U.S. Congress, 1789-2004

Grant, J. T., Kelly, N. J. — 2008-07-10

We measure legislative productivity for the entire history of the U.S. Congress. Current measures of legislative productivity are problematic because they measure productivity for a limited number of decades and because they are based on different aspects of productivity. We provide a methodology for measuring (1) a Legislative Productivity Index (LPI) and (2) a Major Legislation Index (MLI). We use the W-CALC algorithm of Stimson (1999, Public opinion in America: Moods, cycles, and swings. 2nd ed. Boulder, CO: Westview Press) to combine information from previously used indicators of productivity into measures of the LPI and the MLI. We provide examinations of content, convergent, and construct validity. The construct validity model includes potential determinants of legislative productivity. We conclude that the LPI and the MLI are superior measures of productivity than other measures used in the literature.

Two Sides of the Same Coin? Employing Granger Causality Tests in a Time Series Cross-Section Framework

Hood, M. V., Kidd, Q., Morris, I. L. — 2008-07-10

In this paper, we introduce a recently developed methodology for assessing the assumption of causal homogeneity in a time series cross-section Granger framework. Following a description of the procedure and the analytical contexts for which it is appropriate, we implement this new approach to examine the transformation of the post-World War II party system in the South. Specifically, we analyze the causal relationship between black mobilization and GOP growth in the region. We find that black mobilization Granger caused Republican growth throughout the South, whereas Republican growth Granger caused black mobilization only in the deep South. We discuss the substantive significance of our results and conclude with guidelines for the appropriate use of this procedure and suggestions for future extensions of the method.

Wouldn't It Be Nice ...? The Automatic Unbiasedness of OLS (and GLS)

Luskin, R. C. — 2008-07-10

The Constrained Instability of Majority Rule: Experiments on the Robustness of the Uncovered Set

Bianco, W. T., Lynch, M. S., Miller, G. J., Sened, I. — 2008-04-28

The uncovered set has frequently been proposed as a solution concept for majority rule settings. This paper tests this proposition using a new technique for estimating uncovered sets and a series of experiments, including five-player computer-mediated experiments and 35-player paper-format experiments. The results support the theoretic appeal of the uncovered set. Outcomes overwhelmingly lie in or near the uncovered set. Furthermore, when preferences shift, outcomes track the uncovered set. Although outcomes tend to occur within the uncovered set, they are not necessarily stable; majority dominance relationships still produce instability, albeit constrained by the uncovered set.

Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches

Green, D. P., Vavreck, L. — 2008-04-28

Analysts of cluster-randomized field experiments have an array of estimation techniques to choose from. Using Monte Carlo simulation, we evaluate the properties of point estimates and standard errors (SEs) generated by ordinary least squares (OLS) as applied to both individual-level and cluster-level data. We also compare OLS to alternative random effects estimators, such as generalized least squares (GLS). Our simulations assess efficiency across a variety of scenarios involving varying sample sizes and numbers of clusters. Our results confirm that conventional OLS SEs are severely biased downward and that, for all estimators, gains in efficiency come mainly from increasing the number of clusters, not increasing the number of individuals within clusters. We find relatively minor differences across alternative estimation approaches, but GLS seems to enjoy a slight edge in terms of the efficiency of its point estimates and the accuracy of its SEs. We illustrate the application of alternative estimation approaches using a clustered experiment in which Rock the Vote TV advertisements were used to encourage young voters in 85 cable TV markets to vote in the 2004 presidential election.

Is Partial-Dimension Convergence a Problem for Inferences from MCMC Algorithms?

Gill, J. — 2008-04-28

Increasingly, political science researchers are turning to Markov chain Monte Carlo methods to solve inferential problems with complex models and problematic data. This is an enormously powerful set of tools based on replacing difficult or impossible analytical work with simulated empirical draws from the distributions of interest. Although practitioners are generally aware of the importance of convergence of the Markov chain, many are not fully aware of the difficulties in fully assessing convergence across multiple dimensions. In most applied circumstances, every parameter dimension must be converged for the others to converge. The usual culprit is slow mixing of the Markov chain and therefore slow convergence towards the target distribution. This work demonstrates the partial convergence problem for the two dominant algorithms and illustrates these issues with empirical examples.

Testing the Predictions of the Multidimensional Spatial Voting Model with Roll Call Data

Jeong, G.-H. — 2008-04-28

This paper develops a procedure for locating proposals and legislators in a multidimensional policy space by applying agenda-constrained ideal point estimation. Placing proposals and legislators on the same scale allows an empirical test of the predictions of the spatial voting model. I illustrate this procedure by testing the predictive power of the uncovered set—a solution concept of the multidimensional spatial voting model—using roll call data from the U.S. Senate. Since empirical tests of the predictive power of the uncovered set have been limited to experimental data, this is the first empirical test of the concept's predictive power using real-world data.

Comparing Legislators and Legislatures: The Dynamics of Legislative Gridlock Reconsidered

Chiou, F.-Y., Rothenberg, L. S. — 2008-04-28

Although political methodologists are well aware of measurement issues and the problems that can be created, such concerns are not always front and center when we are doing substantive research. Here, we show how choices in measuring legislative preferences have influenced our understanding of what determines legislative outputs. Specifically, we replicate and extend Binder's highly influential analysis (Binder, Sarah A. 1999. The dynamics of legislative gridlock, 1947–96. American Political Science Review 93:519–33; see also Binder, Sarah A. 2003. Stalemate: Causes and consequences of legislative gridlock. Washington, DC: Brookings Institution) of legislative gridlock, which emphasizes how partisan, electoral, and institutional characteristics generate major legislative initiatives. Binder purports to show that examining the proportion, rather than the absolute number, of key policy proposals passed leads to the inference that these features, rather than divided government, are crucial for explaining gridlock. However, we demonstrate that this finding is undermined by flaws in preference measurement. Binder's results are a function of using W-NOMINATE scores never designed for comparing Senate to House members or for analyzing multiple Congresses jointly. When preferences are more appropriately measured with common space scores (Poole, Keith T. 1998. Recovering a basic space from a set of issue scales. American Journal of Political Science 42:964–93), there is no evidence that the factors that she highlights matter.

Taking the Measure of Congress: Reply to Chiou and Rothenberg

Binder, S. A. — 2008-04-28

Chiou and Rothenberg raise important questions about how to measure key concepts in the study of legislative stalemate in the U.S. Congress. In challenging my choice of measures to capture bicameral differences, Chiou and Rothenberg argue that my findings are the artifact of measurement error. In this reply, I review the hurdles involved in measuring policy views over time and across institutions and suggest that the preferred measure of Chiou and Rothenberg falls short for measuring bicameral differences. Second, I assess the extent to which measurement choices affect the robustness of my findings about the determinants of gridlock. Drawing on new measures and model specifications, I show that my results are robust to alternative specifications. I conclude with an assessment of the broader challenges posed by how we measure critical concepts in the study of congressional performance.

The Search for Comparability: Response to Binder

Chiou, F.-Y., Rothenberg, L. S. — 2008-04-28

Binder (n.d., Taking the measure of Congress: Reply to Chiou and Rothenberg. Political Analysis. Forthcoming) highlights areas of agreement and disagreement with our discussion of preference measurement and legislative gridlock. We now both agree that W-NOMINATE scores—employed in Binder (1999, The dynamics of legislative gridlock. American Political Science Review 9:519–33) to measure key independent variables, including bicameral differences—should not be used when examining multichamber legislatures over time. We continue to disagree over whether Common Space scores or Binder's conference vote measure is superior. In this response, we show that, although several of the theoretical and statistical objections that Binder (n.d.) raises to our Common Space measure do not apply, they are all relevant for her conference vote analog. Additionally, we detail how, despite protests to the contrary, the conference vote measure is plagued by insufficient data. Finally, we demonstrate how new efforts to show that Binder's (1999) results continue to hold are not robust.

Editor's Note

Zorn, C. — 2008-04-15

Expert Opinion, Agency Characteristics, and Agency Preferences

Clinton, J. D., Lewis, D. E. — 2008-04-15

The study of bureaucracies and their relationship to political actors is central to understanding the policy process in the United States. Studying this aspect of American politics is difficult because theories of agency behavior, effectiveness, and control often require measures of administrative agencies' policy preferences, and appropriate measures are hard to find for a broad spectrum of agencies. We propose a method for measuring agency preferences based upon an expert survey of agency preferences for 82 executive agencies in existence between 1988 and 2005. We use a multirater item response model to provide a principled structure for combining subjective ratings based on scholarly and journalistic expertise with objective data on agency characteristics. We compare the resulting agency preference estimates and standard errors to existing alternative measures, discussing both the advantages and limitations of the method.

Statistical Backwards Induction: A Simple Method for Estimating Recursive Strategic Models

Bas, M. A., Signorino, C. S., Walker, R. W. — 2008-04-15

We present a simple method for estimating regressions based on recursive extensive-form games. Our procedure, which can be implemented in most standard statistical packages, involves sequentially estimating standard logits (or probits) in a manner analogous to backwards induction. We demonstrate that the technique produces consistent parameter estimates and show how to calculate consistent standard errors. To illustrate the method, we replicate Leblang's (2003) study of speculative attacks by financial markets and government responses to these attacks.

Bayesian and Likelihood Inference for 2 x 2 Ecological Tables: An Incomplete-Data Approach

Imai, K., Lu, Y., Strauss, A. — 2008-04-15

Ecological inference is a statistical problem where aggregate-level data are used to make inferences about individual-level behavior. In this article, we conduct a theoretical and empirical study of Bayesian and likelihood inference for 2 x 2 ecological tables by applying the general statistical framework of incomplete data. We first show that the ecological inference problem can be decomposed into three factors: distributional effects, which address the possible misspecification of parametric modeling assumptions about the unknown distribution of missing data; contextual effects, which represent the possible correlation between missing data and observed variables; and aggregation effects, which are directly related to the loss of information caused by data aggregation. We then examine how these three factors affect inference and offer new statistical methods to address each of them. To deal with distributional effects, we propose a nonparametric Bayesian model based on a Dirichlet process prior, which relaxes common parametric assumptions. We also identify the statistical adjustments necessary to account for contextual effects. Finally, although little can be done to cope with aggregation effects, we offer a method to quantify the magnitude of such effects in order to formally assess its severity. We use simulated and real data sets to empirically investigate the consequences of these three factors and to evaluate the performance of our proposed methods. C code, along with an easy-to-use R interface, is publicly available for implementing our proposed methods (Imai, Lu, and Strauss, forthcoming).

Ignoramus, Ignorabimus? On Uncertainty in Ecological Inference

Elff, M., Gschwend, T., Johnston, R. J. — 2008-04-15

Models of ecological inference (EI) have to rely on crucial assumptions about the individual-level data-generating process, which cannot be tested because of the unavailability of these data. However, these assumptions may be violated by the unknown data and this may lead to serious bias of estimates and predictions. The amount of bias, however, cannot be assessed without information that is unavailable in typical applications of EI. We therefore construct a model that at least approximately accounts for the additional, nonsampling error that may result from possible bias incurred by an EI procedure, a model that builds on the Principle of Maximum Entropy. By means of a systematic simulation experiment, we examine the performance of prediction intervals based on this second-stage Maximum Entropy model. The results of this simulation study suggest that these prediction intervals are at least approximately correct if all possible configurations of the unknown data are taken into account. Finally, we apply our method to a real-world example, where we actually know the true values and are able to assess the performance of our method: the prediction of district-level percentages of split-ticket voting in the 1996 General Election of New Zealand. It turns out that in 95.5% of the New Zealand voting districts, the actual percentage of split-ticket votes lies inside the 95% prediction intervals constructed by our method.

A Robust Transformation Procedure for Interpreting Political Text

Martin, L. W., Vanberg, G. — 2008-04-15

In a recent article in the American Political Science Review, Laver, Benoit, and Garry (2003, "Extracting policy positions from political texts using words as data," 97:311–331) propose a new method for conducting content analysis. Their Wordscores approach, by automating text-coding procedures, represents an advance in content analysis that will potentially have a large long-term impact on research across the discipline. To allow substantive interpretation, the scores produced by the Wordscores procedure require transformation. In this note, we address several shortcomings in the transformation procedure introduced in the original program. We demonstrate that the original transformation distorts the metric on which content scores are placed—hindering the ability of scholars to make meaningful comparisons across texts—and that it is very sensitive to the texts that are scored—opening up the possibility that researchers may generate, inadvertently or not, results that depend on the texts they choose to include in their analyses. We propose a transformation procedure that solves these problems.

Compared to What? A Comment on "A Robust Transformation Procedure for Interpreting Political Text" by Martin and Vanberg

Benoit, K., Laver, M. — 2008-04-15

Reply to Benoit and Laver

Martin, L. W., Vanberg, G. — 2008-04-15