Compound Dilemmas: Democracy, Collective Action and Superpower Rivalry

Introduction (Draft)

For half of the twentieth century, rivalry between the United States and the Union of Soviet Socialist Republics defined the political landscape, both domestic and international. Now, a decade after the dissolution of the Soviet Union brought an abrupt end to the Cold War, much of this landscape seems puzzling, even peculiar.

The most striking aspect of the Cold War was its ever-changing tenor: periods of crisis and tension alternated with periods of détente and cooperation. Yet, whenever hostility became so intense that war seemed imminent, some way was found to avoid any direct confrontation between Soviet and American military forces. Also, whenever cooperation between the two leaders became unusually close, forces within one or both countries would act to restore a more normal level of suspicion. As a whole, the system of superpower rivalry exhibited a tendency towards self-correction.

In this book we use the concepts and analytical tools of rational choice theory to explain these systemic regularities. One basic dilemma of collective action is of particular importance to our argument. It often happens that a group of rational individuals, each of whom is pursuing his or her own self-interest, arrives at a collective outcome that none of them finds acceptable. Yet, their attempts to coordinate their efforts to achieve a more desirable outcome may falter if enough of the individuals refuse to contribute to the collective good. Even a few shirkers may ruin the outcome for all.

Solution of any one dilemma of collective action typically creates or aggravates still other problems. For example, any group that has successfully decided to delegate authority to some agent to act in their name must also find some way to monitor and sanction the actions of that agent. Yet doing so is costly. So even if everyone would benefit from close supervision of the agent, no individual, acting alone, may have an interest in paying the costs of monitoring or sanctioning. Cascading sequences of problems typify all aspects of collective action, which goes a long way towards explaining why the word dilemma is so closely associated with this form of human endeavor.

We argue that the long-term stability of superpower rivalry was a consequence of compounded dilemmas that were, ultimately, grounded in a persistent pattern of actor expectations. From shortly after the end of World War II right up until the collapse of the Soviet Union in 1991, most influential policymakers in both the United States and the Soviet Union believed that the other power posed the primary threat to their own nation’s security. Our analysis assumes that individual policy advocates in both states were rational, in the sense that they pursued their interests in an effective manner. Their interests also included collective goals, especially protection from the threat posed by the rival. Even so, policy advocates differed widely in the means through which they thought this goal could be best pursued. Each country’s security policy during the Cold War was driven by a complex amalgam of national, institutional, and individual interests. Despite ever-present temptations to seek parochial advantage, the collective goal of national security was maintained. However, this success necessarily complicated other collective action problems and created new dilemmas. In short, the policy makers of these rival states faced compound dilemmas of collective action.

Instances of compound dilemmas abound in the historical record. Each superpower sought allies to help offset the threat posed by the other superpower. Each provided arms and support to governments or rebel groups fighting the other state (or its allies). By the early 1950s the U.S. had literally surrounded Soviet territory with military bases and intelligence posts. Superpower rivalry reached into the remotest corners of the world, as otherwise insignificant political changes were interpreted as advantageous to one or the other of the main protagonists. Perhaps the most peculiar manifestation of this complex brew of allies, client states, and proxy forces concerned a brief naval battle between rival Cuban factions that took place on a lake in central Africa (Kwitny 1984: 71-85). Still, despite their many proxy wars, the superpowers scrupulously avoided direct combat operations against the other.

Superpower rivalry also found expression in a nuclear arms race with both quantitative and qualitative components. Through a series of moves and countermoves, the superpowers deployed so many nuclear weapons that scientists seriously debated whether the "nuclear winter" their use might trigger would kill off all life on this planet or just make human civilization no longer viable (Ehrlich et al 1984). Why political leaders would channel scarce resources to produce such an obviously excessive level of armaments is a question that virtually cries out for explanation.

Despite their fundamental antagonism, the superpowers were able to cooperate on certain matters of mutual concern. A growing realization of the dangers of nuclear war led their leaders to sign several arms control agreements, but these agreements didn’t really reduce either side’s arsenals. In effect, strategic arms control was an exercise in the joint management of U.S.-Soviet relations, not an effort to eliminate nuclear weapons. Each agreement was used to support arguments in favor of developing new kinds of weapons, meant to offset any advantage that may have been gained by the adversary in that agreement. In Smoke’s (1987: 180) wonderfully evocative image, "If the arms race is visualized as something like a freight train speeding down the tracks, then the SALT process is best visualized as something running alongside it." (emphasis in original)

This extended process of negotiation legitimated and institutionalized the practice of espionage, enshrined in the recognition that each side has a right to use "national technical means" to collect information on the other’s weapons systems. Even the practice of encrypting test data was banned. This meant that the leaders of these rival powers had come to realize that both sides had an interest in making sure neither side could keep the results of weapon tests secret. Still, each pushed the edges of their agreements by testing or deploying weapons systems that fell within the realm of ambiguity left open by the carefully worded texts. Successive treaties become longer, as each side tried to close loopholes that might be exploited by the other while protecting ambiguities that they themselves might later exploit.

Successive arms control agreements included increasingly stringent verification procedures (see Gallagher 1999, Rueckert 1998). Eventually, both parties agreed to allow the other to establish regular monitoring stations outside its military production facilities. In the final years of the Cold War, the military forces of these two rival superpowers routinely exchanged information, including notifying each other whenever they conducted exercises that might be misconstrued as preparation for an attack. It’s hard to imagine how an attitude of unwavering suspicion could have been sustained under those circumstances.

Given all this tension, something had to break. In the end it was the Soviet Union that collapsed, taking the Cold War with it into the pages of history. The wide-ranging reform process instigated by the Communist Party General Secretary Gorbachev produced this outcome, but this was clearly not his intention. As events spiraled out of his control, Gorbachev’s increasingly radical reforms continued to be dismissed by some influential American critics as a ruse to obtain access to Western technology or economic assistance. After four decades of remarkable resilience to dramatic changes in events, the system of superpower rivalry finally dissipated only after one of its protagonists dissolved into its constituent parts. Long-established habits of suspicion and fear proved remarkably difficult to break.

These examples of compound dilemmas pose a set of puzzles in need of explanation. Resolution of collective action dilemmas at the domestic level made it possible for rivals to maintain a primarily conflictual relationship with each other at the international level. This success in turn created a need for international cooperation, which proved increasingly difficult because of domestic opposition. Under conditions of international rivalry, dilemmas of collective action are compounded in much the same way that debts accumulate under compound interest. That is, dilemmas of collective action are compounded when the solution to one dilemma leads to an increase in the number or severity of dilemmas that must subsequently be confronted.

We explore a theme common to all of these dilemmas. In each instance, seemingly irrational outcomes can be explained as the unintended, aggregate-level consequences of actions taken by individually rational policy advocates in one or both countries. Specifically, we assert that these patterns result from the relatively efficient operation of institutions established and used by sophisticated goal-directed policy actors within both states. Policy actors need access to information about the likely behavior of the other state, especially if they want to anticipate future threats rather than waiting to react after the fact. This mutual effort to anticipate the other’s behavior effectively fuses their foreign policies into a single rivalry system, established and sustained by a close intertwining of domestic and international politics.

Any application of rational choice theory requires auxiliary assumptions concerning the source of preferences (Simon 1985; Snidal 1985). We define a pair of states as rivals whenever most of the influential policy makers in each state believe that the other state poses the primary threat to their own nation’s security. This consensus creates the basis for a system of enduring rivalry. (We discuss other approaches to the study of enduring rivalries in chapter 1.)

One of the reasons why this rivalry was so stable for so long was that both sides had access to sufficient information about the capabilities of the other side. Of course, important surprises and misunderstandings did occur, especially early in the Cold War, but by the time their relationship had matured into a stable rivalry system each superpower was sufficiently well-informed about the other to form reasonably accurate expectations.

We argue that the high premium placed on the procurement of good information about the other superpower created conditions under which the foreign policies of each state approximated the strategies that would have been pursued by a unitary rational actor capable of forming rational expectations of the other’s behavior. Thus, the superpowers were able to achieve the equivalent of a rational expectations equilibrium, in the sense that each was able to make, on the whole, unbiased forecasts of the other’s behavior (see Begg 1982; Sheffrin 1983). The technical meaning of this term will be detailed in subsequent chapters, but the essence of rational expectations is consistency and predictability (within broad constraints, as exact prediction is rarely feasible in complex social settings).

We present empirical results that support these implications, but our analysis does not provide a complete test of collective action theory in the context of superpower rivalry. One limitation is that we do not directly test either of two basic assumptions. First, determination of the depth and breadth of the "Cold War consensus" at different times is beyond the scope of this work. We will, however, make reference to classic works on that topic. Second, we make no effort to "prove" that policy makers are rational. We realize this assertion remains controversial, but we have found it to be a useful departure point for our own analysis. The assumption of individual rationality enables social scientists to draw conclusions about the aggregate consequences of their behavior under different institutional contexts, and empirical investigation is properly focused on these aggregate-level implications.

Unfortunately, the term rational has come to have many meanings, only one of which is relevant for our purposes. In hopes of reducing the chances that our conclusions might be misinterpreted, we want to clarify exactly how we use this highly contested term. When we say U.S. or Soviet policy in the Cold War was rational, we mean only that it approximated a rational expectations equilibrium, in a technical sense to be specified in chapters 3 and 4. Consistency is the key criterion here, along with responsiveness to each other’s behavior.

We make no claim that this form of consistent responsiveness was normatively desirable, either from the perspective of either state’s population or from the perspective of the world as a whole. An aggregate-level equivalent of a self-fulfilling fantasy may have been in operation. That is, each rival may have prepared weapons it felt was necessary to deter attack by the other side whose weapons profile was in turn shaped by its own fears of attack (see Wolfe 1984). For the purposes of this analysis, "rationality" must be interpreted within the constraints set by policymakers’ widespread presumption of fundamental antagonism between these two superpowers.

By saying that domestic collective action problems were "solved" or that information was used "efficiently" we do not mean to suggest that these outcomes were optimal. There is no implication that these policies achieved maximum economic growth for either society or provided for the general welfare. Indeed, long-term weakness of the Soviet economy and the devastating environmental degradation that occurred there demonstrate that the Soviets sustained their end of the rivalry only by accepting very high social costs. Had all-out nuclear war broken out between these two rivals the level of devastation would have been unfathomable. However, there were many reasons why this terrible outcome did not occur, reasons engrained in the basic structure of this rivalry system.

We also do not presume that any individual policy actor or social planner consciously devised a solution to these domestic collective action problems. One of the central lessons of rational choice theory is that collective outcomes are often unintended by political participants. Just as market participants do not have the goal of increasing national income when they engage in their individual decisions, it is not necessary to assume that all political participants in a rivalry are driven by selfless goals of national interests. No doubt some of them are, but for the most part politicians seek re-election, bureaucrats seek additional resources, and weapons manufacturers seek profits. Our story is about how these diffuse interests came together and produced "rational" policy towards a rival.

Chapter 1 locates our analysis in the context of international relations theory. We explain how our conceptualization of superpower rivalry differs from the related concepts of an "enduring rivalry" and a "two-level game." Chapter 2 lays out the domestic foundations of superpower rivalry. We examine how the compound dilemmas of collective action and principal-agent relations were manifested in the American political context. Since access to information plays a crucial role in our analysis, the CIA and related intelligence agencies play a prominent role in that story.

The next three chapters constitute the analytical core of the book. In Chapter 3 we develop a rational expectations model of superpower rivalry. This model extends a long-standing tradition of formal models of arms races and related phenomena at the international level. Chapter 4 tests the propositions derived from this model, through the application of vector autoregression techniques to time series data on U.S. and Soviet military expenditures and an events data based measure of diplomatic behavior. (Controversies surrounding the measurement of Soviet military expenditures are reviewed in a Data Appendix, which also includes all data series used in this analysis. Technical details concerning our statistical analyses are collected in the Methodological Appendix.) In Chapter 5 we return to the domestic level of analysis to test an organizational process model based on the sequence of steps by which the U.S. military budget has been determined.

In Chapter 6 we re-visit the international context of superpower rivalry, with particular emphasis on the sources and limitations of the arms control process. The collective action problems involved in establishing a comprehensive pattern of cooperation at the international level were much more difficult to solve than the domestic collective action dilemmas that enabled these states to be rivals in the first place. Even partial resolution of these international dilemmas elicited determined resistance from domestic actors in both states. Had the process of arms control continued much further the very basis of rivalry might have been placed at risk, but it was for precisely this reason that continued success in arms control negotiations became increasingly difficult to achieve. We examine how these compound dilemmas contributed to the dissolution of the superpower rivalry system.

Chapter 7 discusses prospects for similar rivalries in the post-Cold War era. We evaluate some of the lingering after-effects on patterns of policy debate within the only superpower to survive rivalry intact. This concluding chapter also draws out implications for the analysis of the ethnic rivalries that seem so common in today’s world.

In Chapters 3 through 5 we use the methodological tools of rational choice models and time-series statistical analysis to formulate and test some specific implications of our argument. Throughout the book we rely on extended quotations from historical sources to support our interpretation of superpower rivalry. We have found it especially useful to quote from the works of historians who have examined the vast body of material made available from recently-opened Soviet archives. However, we make no claim that these quotations "prove" our argument or unequivocally validate our empirical results. We use them only to locate our analysis within the ever-expanding array of interpretations of this important and still-controversial historical period.

The definitive book on the origins, stability, consequences, and ultimate demise of the Cold War remains to be written. In this book we pursue a single line of argument, and so we cannot pretend to explain everything. We share Waltz’s (1979) deep appreciation of the power of a simple, parsimonious theoretical argument tightly focused on explaining the "big picture." Waltz opts for a structural theory based exclusively on the characteristics of the international system as a whole. We are concerned with understanding a different kind of structure, one in which factors at different levels of analysis were tightly interwoven.

The Cold War is long over, but contemporary and future policy makers may inadvertently discover other dysfunctional ways to solve domestic dilemmas of collective action by creating new and more dangerous dilemmas at the international level, or vice versa. The exact characteristics of superpower rivalry are unlikely to be repeated in any future rivalry. Still, this intertwining of domestic and international factors has obvious relevance to a world in which globalization is erasing distinctions between domestic and international politics.

Table of Contents

INTRODUCTION

CHAPTER 1. Rivalry in International Relations

The Cold War as an Enduring Rivalry
The Cold War as an International System
The Focusing Effect of Rivalry
Rivalry as a Two-Level Game

CHAPTER 2. Democracy and Rivalry

Dilemmas of Collective Action in a Democratic Polity
Dilemmas of Delegation and Organization
Institutions and Information
Collective Action in Authoritarian and Democratic Systems

CHAPTER 3. Rational Expectations and the Arms Race

From Richardson to Rational Choice
A Rational Expectations Arms Race Model

CHAPTER 4. Dynamics of Superpower Rivalry

A Vector Autoregression Model of Military Expenditures
Diplomatic Relations and Economic Conditions
A Bayesian VAR  Model
Stability in the Rivalry System

CHAPTER 5. Organizations and the Domestic Politics of Rivalry

Contrasting Perspectives on Budgetary Politics
A VAR Model of the U.S. Defense Budgeting Process
Evaluation

CHAPTER 6. Cooperation and the End of Rivalry

Sources and Limitations of Cooperation Between Rivals
Transformation and Termination of the Rivalry System

CHAPTER 7. Lessons for the Post-Cold War World

Implications for Democratic Politics
Implications for Other Rivalry Systems
Implications for Analysis

APPENDIX A. Data Appendix

APPENDIX B. Methodological Appendix

REFERENCES

NOTES

Copyright 2000 by Michael D. McGinnis and John T. Williams