Political Analysis - recent issues

Modeling Macro-Political Dynamics

Brandt, P. T., Freeman, J. R. — 2009-04-24

Analyzing macro-political processes is complicated by four interrelated problems: model scale, endogeneity, persistence, and specification uncertainty. These problems are endemic in the study of political economy, public opinion, international relations, and other kinds of macro-political research. We show how a Bayesian structural time series approach addresses them. Our illustration is a structurally identified, nine-equation model of the U.S. political-economic system. It combines key features of the model of Erikson, MacKuen, and Stimson (2002) of the American macropolity with those of a leading macroeconomic model of the United States (Sims and Zha, 1998; Leeper, Sims, and Zha, 1996). This Bayesian structural model, with a loosely informed prior, yields the best performance in terms of model fit and dynamics. This model 1) confirms existing results about the countercyclical nature of monetary policy (Williams 1990); 2) reveals informational sources of approval dynamics: innovations in information variables affect consumer sentiment and approval and the impacts on consumer sentiment feed-forward into subsequent approval changes; 3) finds that the real economy does not have any major impacts on key macropolity variables; and 4) concludes, contrary to Erikson, MacKuen, and Stimson (2002), that macropartisanship does not depend on the evolution of the real economy in the short or medium term and only very weakly on informational variables in the long term.

Treatment Spillover Effects across Survey Experiments

Transue, J. E., Lee, D. J., Aldrich, J. H. — 2009-04-24

Embedding experiments within surveys has reinvigorated survey research. Several survey experiments are generally embedded within a survey, and analysts treat each of these experiments as self-contained. We investigate whether experiments are self-contained or if earlier treatments affect later experiments, which we call "experimental spillover." We consider two types of bias that might be introduced by spillover: mean and inference biases. Using a simple procedure, we test for experimental spillover in two data sets: the 1991 Race and Politics Survey and a survey containing several experiments pertaining to foreign policy attitudes. We find some evidence of spillover and suggest solutions to avoid bias.

The Microfoundations of Mass Polarization

Levendusky, M. S. — 2009-04-24

Although there has been considerable attention to the question of how much polarization there is in the mass electorate, there has been much less attention paid to the mechanism that causes polarization. I provide evidence demonstrating the occurrence of individual-level conversion—individual Democrats and Republicans becoming more liberal and conservative. Although over the short term most of the observed changes are quite small and cannot be distinguished from measurement error, over time and many respondents, these movements aggregate to generate polarization. Small individual-level preference shifts provide an important foundation for aggregate polarization.

Modeling Certainty with Clustered Data: A Comparison of Methods

Arceneaux, K., Nickerson, D. W. — 2009-04-24

Political scientists often analyze data in which the observational units are clustered into politically or socially meaningful groups with an interest in estimating the effects that group-level factors have on individual-level behavior. Even in the presence of low levels of intracluster correlation, it is well known among statisticians that ignoring the clustered nature of such data overstates the precision estimates for group-level effects. Although a number of methods that account for clustering are available, their precision estimates are poorly understood, making it difficult for researchers to choose among approaches. In this paper, we explicate and compare commonly used methods (clustered robust standard errors (SEs), random effects, hierarchical linear model, and aggregated ordinary least squares) of estimating the SEs for group-level effects. We demonstrate analytically and with the help of empirical examples that under ideal conditions there is no meaningful difference in the SEs generated by these methods. We conclude with advice on the ways in which analysts can increase the efficiency of clustered designs.

Political Science, Biometric Theory, and Twin Studies: A Methodological Introduction

Medland, S. E., Hatemi, P. K. — 2009-04-24

As political scientists begin to incorporate biological influences as explanatory factors in political behavior, the need to present a methodological road map for utilizing biometric genetic theory and twin data is apparent. The classical twin design (CTD) remains the most popular design for initial examinations of the source of variance among social and political behaviors, and a vast majority of advanced variance components models as well as some molecular analyses are extensions of the CTD. Thus, it is appropriate to begin a series of works with the CTD and its most common variants. The CTD has strong roots in biometrical genetic theory and provides estimates of the correlations between observed traits of monozygotic and dizygotic twins in terms of underlying genetic and environmental influences. The majority of these analyses utilize SEMs of observed covariances for both twin types to assess the relative importance of these "latent" factors.

A New Measure of Policy Spending Priorities in the American States

Jacoby, W. G., Schneider, S. K. — 2009-03-06

In this paper, we develop and test a general measure of policy expenditures in the American states. Our approach is to construct a spatial proximity model of yearly state program spending. The empirical analysis reveals that state spending patterns vary along a clear and readily-interpretable unidimensional continuum which differentiates policies that provide particularized benefits to needy constituencies from policies that provide broader collective goods. Based upon standard evaluative criteria, the variable created from our model possesses some highly desirable characteristics. And, it compares favorably to other measures of state policy activity. The net result is a yearly score for each state which summarizes that state's spending across all major program areas. More generally, we believe that our variable can be interpreted as valid and reliable representational measurement of state policy priorities. In this capacity, it could occupy an important position within models of state politics.

Discretion Rather than Rules: Choice of Instruments to Control Bureaucratic Policy Making

Gailmard, S. — 2009-03-06

In this paper I investigate the trade-off a legislature faces in the choice of instruments to ensure accountability by bureaucrats with private information. The legislature can either design a state-contingent incentive scheme or "menu law" to elicit the bureau's information or it can simply limit the set of choices open to the bureaucrat and let it choose as it wishes (an action restriction). I show that the optimal action restriction is simply a connected interval of the policy space. However, this class of instruments is not optimal without some sort of limitation on the set of levers of control available to the legislature. I then analyze one such limitation salient in politics, the legislative principal's inability to commit to honor a schedule of (state contingent) policy choices and transfer payments for a menu law. In this case the optimal action restriction outperforms (in terms of the legislature's welfare) the best available menu law.

Sensitive Questions, Truthful Answers? Modeling the List Experiment with LISTIT

Corstange, D. — 2009-03-06

Standard estimation procedures assume that empirical observations are accurate reflections of the true values of the dependent variable, but this assumption is dubious when modeling self-reported data on sensitive topics. List experiments (a.k.a. item count techniques) can nullify incentives for respondents to misrepresent themselves to interviewers, but current data analysis techniques are limited to difference-in-means tests. I present a revised procedure and statistical estimator called LISTIT that enable multivariate modeling of list experiment data. Monte Carlo simulations and a field test in Lebanon explore the behavior of this estimator.

Measurement Equivalence of Nationalism and Constructive Patriotism in the ISSP: 34 Countries in a Comparative Perspective

Davidov, E. — 2009-03-06

Studies on national identity differentiate between nationalistic attitudes and constructive patriotism (CP) as two more specific expressions of national identity and as theoretically two distinct concepts. After a brief discussion of the theoretical literature, the following questions are examined: (1) Can nationalism and CP be empirically identified as two distinct concepts?; (2) Is their meaning fully or partially invariant across countries?; and (3) Is it possible to compare their means across countries? Data from the International Social Survey Program (ISSP) 2003 National Identity Module are utilized to answer these questions in a sample of 34 countries. Items to measure nationalism and CP are chosen based on the literature, and a series of confirmatory factor analyses to test for configural, measurement (metric), and scalar invariance are performed. Full or partial metric invariance is a necessary condition for equivalence of meaning across cultures and for a meaningful comparison of associations with other theoretical constructs. Scalar invariance is a necessary condition for comparison of means across countries. Findings reveal that nationalism and CP emerge as two distinct constructs. However, in some countries, some items that were intended to measure one construct also measure the other construct. Furthermore, configural and metric invariance are found across the full set of 34 countries. Consequently, researchers may now use the ISSP data to study relationships among nationalism, CP, and other theoretical constructs across these nations. However, the analysis did not support scalar invariance, making it problematic for comparing the means of nationalism and CP across countries.

Revisiting Adjusted ADA Scores for the U.S. Congress, 1947-2007

Anderson, S., Habel, P. — 2009-03-06

This paper replicates and extends Groseclose, Levitt, and Snyder, "Comparing Interest Group Scores Across Time and Chambers: Adjusted ADA Scores for the U.S. Congress," which appeared in the American Political Science Review (1999/93:33–50). We replicate the most recent unpublished extension by Dr. Groseclose and research assistants for years 1947–1999, and then we extend the analysis to include years 2000 through 2007. We make available inflation-adjusted ADA scores from 1947 through 2007, allowing scholars to incorporate the most recent interest group scores into their analyses.

A Comment on Diagnostic Tools for Counterfactual Inference

Sambanis, N., Michaelides, A. — 2009-03-06

We evaluate two diagnostic tools used to determine if counterfactual analysis requires extrapolation. Counterfactuals based on extrapolation are model dependent and might not support empirically valid inferences. The diagnostics help researchers identify those counterfactual "what if" questions that are empirically plausible. We show, through simple Monte Carlo experiments, that these diagnostics will often detect extrapolation, suggesting that there is a risk of biased counterfactual inference when there is no such risk of extrapolation bias in the data. This is because the diagnostics are affected by what we call the n/k problem: as the number of data points relative to the number of explanatory variables decreases, the diagnostics are more likely to detect the risk of extrapolation bias even when such risk does not exist. We conclude that the diagnostics provide too severe a test for many data sets used in political science.

Empirical versus Theoretical Claims about Extreme Counterfactuals: A Response

King, G., Zeng, L. — 2009-03-06

In response to the data-based measures of model dependence proposed in King and Zeng (2006), Sambanis and Michaelides (2008) propose alternative measures that rely upon assumptions untestable in observational data. If these assumptions are correct, then their measures are appropriate and ours, based solely on the empirical data, may be too conservative. If instead, and as is usually the case, the researcher is not certain of the precise functional form of the data generating process, the distribution from which the data are drawn, and the applicability of these modeling assumptions to new counterfactuals, then the data-based measures proposed in King and Zeng (2006) are much preferred. After all, the point of model dependence checks is to verify empirically, rather than to stipulate by assumption, the effects of modeling assumptions on counterfactual inferences.

Introduction to the Special Issue: The Statistical Analysis of Political Text

Monroe, B. L., Schrodt, P. A. — 2009-02-25

Understanding Wordscores

Lowe, W. — 2009-02-25

Wordscores is a widely used procedure for inferring policy positions, or scores, for new documents on the basis of scores for words derived from documents with known scores. It is computationally straightforward, requires no distributional assumptions, but has unresolved practical and theoretical problems. In applications, estimated document scores are on the wrong scale and the theoretical development does not specify a statistical model, so it is unclear what assumptions the method makes about political text and how to tell whether they fit particular text analysis applications. The first part of the paper demonstrates that badly scaled document score estimates reflect deeper problems with the method. The second part shows how to understand Wordscores as an approximation to correspondence analysis which itself approximates a statistical ideal point model for words. Problems with the method are identified with the conditions under which these layers of approximation fail to ensure consistent and unbiased estimation of the parameters of the ideal point model.

Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict

Monroe, B. L., Colaresi, M. P., Quinn, K. M. — 2009-02-25

Entries in the burgeoning "text-as-data" movement are often accompanied by lists or visualizations of how word (or other lexical feature) usage differs across some pair or set of documents. These are intended either to establish some target semantic concept (like the content of partisan frames) to estimate word-specific measures that feed forward into another analysis (like locating parties in ideological space) or both. We discuss a variety of techniques for selecting words that capture partisan, or other, differences in political speech and for evaluating the relative importance of those words. We introduce and emphasize several new approaches based on Bayesian shrinkage and regularization. We illustrate the relative utility of these approaches with analyses of partisan, gender, and distributive speech in the U.S. Senate.

Does Deliberation Matter in FOMC Monetary Policymaking? The Volcker Revolution of 1979

Bailey, A., Schonhardt-Bailey, C. — 2009-02-25

In monetary policy, decision makers seek to influence the expectations of agents in ways that can avoid making abrupt, dramatic, and unexpected decisions. Yet in October 1979, Chairman Paul Volcker led the Federal Reserve's Federal Open Market Committee (FOMC) unanimously to shift its course in managing U.S. monetary policy, which in turn eventually brought the era of high inflation to an end. Although some analysts argue that "the presence and influence of one individual"—namely, Volcker—is sufficient to explain the policy shift, this overlooks an important feature of monetary policymaking. FOMC chairmen—however, omnipotent they may appear—do not act alone. They require the agreement of other committee members, and in the 1979 revolution, the decision was unanimous. How, then, did Chairman Volcker manage to bring a previously divided committee to a consensus in October 1979, and moreover, how did he retain the support of the committee throughout the following year in the face of mounting political and economic pressure against the Fed? We use automated content analysis to examine the discourse of the FOMC (with this discourse recorded in the verbatim transcripts of meetings). In applying this methodology, we assess the force of the arguments used by Chairman Volcker and find that deliberation in the FOMC did indeed "matter" both in 1979 and 1980. Specifically, Volcker led his colleagues in coming to understand and apply the idea of credible commitment in U.S. monetary policymaking.

Parsing, Semantic Networks, and Political Authority Using Syntactic Analysis to Extract Semantic Relations from Dutch Newspaper Articles

van Atteveldt, W., Kleinnijenhuis, J., Ruigrok, N. — 2009-02-25

Analysis of political communication is an important aspect of political research. Thematic content analysis has yielded considerable success both with manual and automatic coding, but Semantic Network Analysis has proven more difficult, both for humans and for the computer. This article presents a system for an automated Semantic Network Analysis of Dutch texts. The system automatically extracts relations between political actors based on the output of syntactic analysis of Dutch newspaper articles. Specifically, the system uses pattern matching to find source constructions and determine the semantic agent and patient of relations, and name matching and anaphora resolution to identify political actors. The performance of the system is judged by comparing the extracted relations to manual codings of the same material. Results on the level of measurement indicate acceptable performance. We also estimate performance at the levels of analysis by using a case study of media authority, resulting in good correlations between the theoretical variables derived from the automatic and manual analysis. Finally, we test a number of substantive hypotheses with regression models using the automatic and manual output, resulting in highly similar models in each case. This suggests that our method has sufficient performance to be used to answer relevant political questions in a valid way.

Lexical Cohesion Analysis of Political Speech

Klebanov, B. B., Diermeier, D., Beigman, E. — 2009-02-25

This article presents a novel automatic method of text analysis aimed at discovering patterns of lexical cohesion in political speech. The unit of analysis are groups of words with related meanings; the software is based on the results of a multiperson annotation experiment that captures reliably identified connections between words in a text. We illustrate the advantages of such a representation by juxtaposing results of a detailed hand-made analysis of Margaret Thatcher's rhetoric with analysis based on the automatically detected groups of words. We both corroborate previous findings regarding Thatcher's rhetorical tools and illuminate additional elements thereof. We suggest that lexical cohesion analysis is a promising technique to bridge the gap between quantitative and qualitative analyses of text as political material, by establishing units that are both robust enough to enable comprehensive coverage and coherent enough to support direct interpretation.

Coding Disaggregated Intrastate Conflict: Machine Processing the Behavior of Substate Actors Over Time and Space

Shellman, S. M. — 2009-02-25

This article describes a new machine-coded event data set specifically designed to study the spatially, temporally, and tactically disaggregated actions of multiple state and nonstate actors in a systematic fashion. The project develops an extensive set of dictionaries for multiple actors and employs a new coding scheme to organize information on such actors and their behavior. The author describes the machine content-analysis methods used to collect the data and the newly developed coding scheme.