Qualitative methods in international relations

Millennial Reflections on International Studies

Volumn , Issue , 2009, Pages 432-454

  Levy, Jack S ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85059558298     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (174)

1. Giovanni Sartori, “Concept Misinformation in Comparative Politics”, American Political Science Review 64, no. 4 (1970): 1033-53
2. Idiographic inquiry aims to describe, understand, and interpret individual events or a temporally and spatially bounded series of events, whereas nomothetic inquiry aims to generalize about the relationships between variables and, to the extent possible, construct lawlike propositions about social behavior.
3. See Jack S. Levy, “Explaining Events and Testing Theories: History, Political Science, and the Analysis of International Relations”, in Bridges and Boundaries, ed. Colin Elman and Miriam Fendius Elman (Cambridge: MIT Press, 2001), 39-83.
4. For example, Brecher, Steinberg, and Stein argued that most empirically oriented foreign policy analyses were single country in orientation and “usually devoid of theoretical value”.
5. Michael Brecher, Blema Steinberg, and Janice Stein, “A Framework for Research on Foreign Policy Behavior”, Journal of Conflict Resolution 13, no. 1 (1969): 75.
6. Sidney Verba, “Some Dilemmas in Comparative Research”, World Politics 20, no. 1 (1967): 111-27.
7. Adam Przeworski and Henry Teune, The Logic of Comparative Social Inquiry (New York: Wiley, 1970).
8. Arend Lijphart, “Comparative Politics and the Comparative Method”, American Political Science Review 65, no. 3 (1971): 682-93.
9. Arend Lijphart, “The Comparable Cases Strategy in Comparative Research”, Comparative Political Studies 8, no. 2 (1975): 133-77.
10. Harry Eckstein, “Case Study and Theory in Political Science”, in Handbook of Political Science, vol. 7, ed. Fred I. Greenstein and Nelson W. Polsby (Reading, Mass.: Addison-Wesley, 1975), 79-138.
11. Alexander L. George, “Case Studies and Theory Development”, in Diplomacy: New Approaches in Theory, History, and Policy, ed. Paul Lauren (New York: Free Press, 1979), 43-68.
12. Arthur Stinchcombe, Constructing Social Theories (New York: Harcourt, Brace and World, 1968)
13. Neil Smelser, “The Methodology of Comparative Analysis”, in Comparative Research Methods, ed. Donald P. Warwick and Samuel Osherson (Englewood Cliffs, N.J.: Prentice Hall, 1973), 42-86.
14. Theda Skocpol and Margaret Somers, “The Uses of Comparative History in Macrosocial Inquiry”, Comparative Studies in Society and History 22, no. 2 (1980): 156-73.
15. Barrington Moore Jr., Social Origins of Dictatorship and Democracy: Lord and Peasant in the Making of the Modern World (Boston: Beacon Press, 1966).
16. Charles Tilly, ed., The Formation of National States in Western Europe (Princeton, N.J.: Princeton University Press, 1975).
17. Theda M. Skocpol, States and Social Revolutions (New York: Cambridge University Press, 1979).
18. Charles Tilly, Big Structures, Large Processes, Huge Comparisons (New York: Russell Sage Foundation, 1984)
19. Alexander George and Timothy McKeown, “Case Studies and Theories of Organizational Decision Making”, in Advances in Information Processing in Organizations, ed. Robert Coulam and Richard Smith (Greenwich, Conn.: JAI Press, 1985), 43-68.
20. Charles C. Ragin, The Comparative Method (Berkeley: University of California Press, 1987)
21. Charles C. Ragin, “Introduction to Qualitative Comparative Analysis”, in The Comparative Economy of the Welfare State, ed. Thomas Janoski and Alexander M. Hicks (New York: Cambridge University Press, 1994), 299-319.
22. Barbara Geddes, “How the Cases You Choose Affect the Answers You Get: Selection Bias in Comparative Politics”, in Political Analysis, vol. 2, ed. James A. Stimson (Ann Arbor: University of Michigan Press, 1990), 131-50
23. Charles C. Ragin and Howard Becker, eds., What Is a Case? (New York: Cambridge University Press, 1992)
24. David Collier, “The Comparative Method”, in Political Science: The State of the Discipline II, ed. Ada Finifter (Washington, D.C.: American Political Science Association, 1993), 105-19
25. David Collier and James Mahoney, “Conceptual Stretching Revisited: Adapting Categories in Comparative Analysis”, American Political Science Review 87, no. 4 (1993): 845-55.
26. Gary King, Robert Keohane, and Sidney Verba, Designing Social Inquiry (Princeton, N.J.: Princeton University Press, 1994).
27. David Collier and James Mahoney, “Insights and Pitfalls: Selection Bias in Qualitative Research”, World Politics 49, no. 1 (1996): 56-91.
28. Douglas Dion, “Evidence and Inference in the Comparative Case Study”, Comparative Politics 30, no. 2 (1998): 127-46.
29. Timothy J. McKeown, “Case Studies and the Statistical World View”, International Organization 53, no. 1 (1999): 161-90.
30. Charles C. Ragin, Fuzzy-Set Social Science (Chicago: University of Chicago Press, 2000)
31. Alexander L. George and Andrew Bennett, Case Studies and Theory Development (Cambridge: MIT Press, forthcoming).
32. For further information about the consortium, and for syllabi on qualitative methods, see http://www.asu.edu/clas/polisci/cqrm.
33. There has been an interesting reversal in the “balance” between the literature on qualitative methodology and applications of qualitative methods over the last three decades. In the early 1970s, the methodological literature on comparative and case study methods failed to adequately reflect the theoretical and methodological sophistication of some of the best applied research in the field, including work by George and Brecher. Today, applied qualitative research probably lags behind developments in the literature on the methodology of qualitative analysis.
34. Alexander L. George and Richard Smoke, Deterrence in American Foreign Policy (New York: Columbia University Press, 1974)
35. Michael Brecher, The Foreign Policy System of Israel: Setting, Images, Process (New Haven, Conn.: Yale University Press, 1972).
36. Lijphart, “Comparative Politics and the Comparative Method”, 682.
37. Collier, “The Comparative Method”, 105.
38. Andrew Bennett, “Case Study: Methods and Analysis”, in International Encyclopedia of the Social and Behavioral Sciences, ed. Neil J. Smelser and Paul B. Baltes (New York: Pergamon, forthcoming).
39. Ragin and Becker, What Is a Case?.
40. See George, “Case Studies and Theory Development”.
41. If we think of cases as instances of broader theoretical categories, and if we acknowledge that historians are more idiographic in orientation than are political scientists, it should not be surprising that historians rarely use the term case.
42. Levy, “Explaining Events and Testing Theories”.
43. John Walton, “Making the Theoretical Case”, in Ragin and Becker, What Is a Case? 122.
44. For further discussion of whether cases are made or found see Ragin and Becker, What Is a Case?
45. See King, Keohane, and Verba, Designing Social Inquiry.
46. Eckstein, “Case Study and Theory in Political Science”, 85.
47. Lijphart, “Comparative Politics and the Comparative Method”, 691.
48. Eckstein, “Case Study and Theory in Political Science”, 96-123.
49. The first two in the list come from Verba, “Some Dilemmas in Comparative Research”.
50. Eric Hobsbawm, On History (New York: New Press, 1997), 109.
51. Because of their “total” orientation toward the subject matter, atheoretical case studies do not fit the definition of a “case” as an instance of a broader class of phenomena.
52. Stephen Van Evera, Guide to Methods for Students of Political Science (Ithaca, N.Y.: Cornell University Press, 1997)
53. Levy, “Explaining Events and Testing Theories”.
54. One example is Allison’s use of three alternative theoretical frameworks to interpret the Cuban missile crisis (though this study had theoretical aims as well).
55. Graham Allison, Essence of Decision: Explaining the Cuban Missile Crisis (New York: Little, Brown, 1971).
56. Christopher Achen and Duncan Snidal, “Rational Deterrence Theory and Comparative Case Studies”, World Politics 41, no. 2 (1989): 145.
57. Christopher Achen and Duncan Snidal, “Rational Deterrence Theory and Comparative Case Studies”, World Politics 41, no. 2 (1989), 169.
58. The widely accepted injunction against testing a theory or hypothesis against the same data from which it was generated does not preclude us from testing hypotheses generated in a case study against different data from the same case. If a case study of the origins of the cold war leads a researcher to formulate hypotheses about enduring rivalries, and if those hypotheses have implications for the termination of enduring or militarized rivalries, then those same hypotheses can be tested against evidence from the end of the cold war. Indeed, one of the most useful ways of validating a historical interpretation is to derive additional implications of that interpretation for other aspects of that particular historical episode.
59. George and Smoke, Deterrence in American Foreign Policy.
60. Robert Bates, Avner Greif, Margaret Levi, Jean-Laurent Rosenthal, and Barry Weingast, Analytic Narratives (Princeton, N.J.: Princeton University Press, 1998).
61. George, “Case Studies and Theory Development”.
62. See James Lee Ray, Democracies and International Conflict (Columbia: University of South Carolina Press, 1995)
63. John Owen IV, Liberal Peace, Liberal War: American Politics and International Security (Ithaca, N.Y.: Cornell University Press, 1997).
64. Noted in Howard S. Becker, “Cases, Causes, Conjunctures, Stories, and Imagery”, in Ragin and Becker, What Is a Case? 209.
65. George, “Case Studies and Theory Development”.
66. George and Bennett, Case Studies and Theory Development.
67. David Collier and Steven Levitsky, “Democracy with Adjectives: Conceptual Innovation in Comparative Research”, World Politics 49, no. 3 (1997): 430-51.
68. One caveat is worth noting. If the proliferation of subtypes and of the number of causal paths leading to them is taken to an extreme, the theory might explain all the anomalies but only at the cost of a loss of analytic power and parsimony. This problem is minimized if the variety of causal paths is linked to the conditions under which each is most likely to occur.
69. Arthur L. Stinchcombe, Theoretical Methods in Social History (New York: Academic Press, 1978), 5.
70. Lijphart, “The Comparable Cases Strategy in Comparative Research”, 159.
71. Hugh Stretton, The Political Sciences (London: Routledge and Kegan Paul, 1969), 245-47.
72. Lijphart, “Comparative Politics and the Comparative Method”, 692.
73. The many variables/small-N problem affects the use of case studies for theory testing but not for describing and explaining individual historical episodes or for generating or refining hypotheses to be tested by other means. If the universe of cases is relatively small (hegemonic wars, for example), and if the aim of case study analysis is to generate hypotheses, there may be advantages of using a small number of cases. This leaves a maximum number of cases for hypothesis testing, given the need to test hypotheses on data that are independent of the data from which the hypotheses are generated.
74. See also King, Keohane, and Verba, Designing Social Inquiry, 217-18.
75. Lijphart, “Comparative Politics and the Comparative Method”, 685.
76. Lijphart, “The Comparable Cases Strategy in Comparative Research”, 163.
77. Campbell also emphasized the utility of a single case, but only after retracting his earlier argument to the contrary.
78. Donald Campbell, “Degrees of Freedom and the Case Study”, Comparative Political Studies 8, no. 2 (1975): 178-93.
79. See also McKeown, “Case Studies and the Statistical World View”.
80. John Frendreis, “Explanation of Variation and Detection of Covariation: The Purpose and Logic of Comparative Analysis”, Comparative Political Studies 16, no. 2 (1983): 255.
81. John Stuart Mill, A System of Logic (1875 reprint, London: Longman, 1970).
82. Przeworski and Teune, The Logic of Comparative Social Inquiry
83. Theodore Meckstroth, “‘Most Different Systems’ and ‘Most Similar Systems’: A Study in the Logic of Comparative Inquiry”, Comparative Political Studies 8, no. 2 (1975): 133-77.
84. This terminology has generated some confusion. Mill defines agreement or difference in terms of the dependent variable, whereas most similar and most different are defined in terms of explanatory variables. Thus Mill’s method of agreement is equivalent to a most-different systems design, and Mill’s method of difference is equivalent to a most-similar systems design.
85. Lijphart, “The Comparable Cases Strategy in Comparative Research”.
86. Smelser, “The Methodology of Comparative Analysis”.
87. Przeworski and Teune, The Logic of Comparative Social Inquiry.
88. Mill, A System of Logic, bk. 6, ch. 7.
89. Lijphart, “Comparative Politics and the Comparative Method”, 688.
90. George, “Case Studies and Theory Development”.
91. George and Bennett, Case Studies and Theory Development.
92. Stephen Peter Rosen, Societies and Military Power: India and Its Armies (Ithaca, N.Y.: Cornell University Press, 1996).
93. Jack Snyder, Myths of Empire: Domestic Politics and International Ambition (Ithaca, N.Y.: Cornell University Press, 1991).
94. See King, Keohane, and Verba, Designing Social Inquiry, 126.
95. Collier and Mahoney, “Insights and Pitfalls”.
96. King, Keohane, and Verba, Designing Social Inquiry, ch. 4.
97. King, Keohane, and Verba, Designing Social Inquiry, ch. 4.
98. Collier and Mahoney, “Insights and Pitfalls”.
99. An example involving large-N designs is Bueno de Mesquita’s empirical analysis of his core hypothesis that a positive expected utility for war is a necessary condition for a state to initiate war.
100. Bruce Bueno de Mesquita, The War Trap (New Haven, Conn.: Yale University Press, 1981).
101. Dion, “Evidence and Inference in the Comparative Case Study”.
102. The proposition that democracies rarely if ever go to war with each other posits that joint democracy is a sufficient condition for peace, and for the analysis of this proposition scholars focus exclusively on cases of democratic dyads.
103. Ragin, The Comparative Method.
104. Ragin argues that because the number of interaction effects necessary to capture combinatorial effects increases rapidly with the number of variables, serious degrees-of-freedom problems result.
105. Bruce Bueno de Mesquita, The War Trap (New Haven, Conn.: Yale University Press, 1981).
106. For a new statistical approach to modeling multiple causal paths see Bear F. Braumoeller, “Modeling Multiple Causal Paths: Logic, Derivation, and Implementation” (manuscript, Harvard University, 2000).
107. For an application of Boolean models to the study of deterrence see Frank P. Harvey, “Practicing Coercion: Revisiting Successes and Failures Using Boolean Logic and Comparative Methods”, Journal of Conflict Resolution 43, no. 6 (1999): 840-71.
108. Eckstein, “Case Study and Theory in Political Science”, 113-23.
109. This assumes that measurement error is low.
110. Arend Lijphart, The Politics of Accommodation (Berkeley: University of California Press, 1968).
111. See David Bicknell Truman, The Governmental Process (New York: Knopf, 1951).
112. Ronald Rogowski, “The Role of Theory and Anomaly in Social-Scientific Inference”, American Political Science Review 89, no. 2 (1995): 467-70.
113. Lijphart also contributed to theory construction by suggesting hypotheses on why the theory of cross-cutting cleavages broke down in the Netherlands.
114. See Lijphart, The Politics of Accommodation.
115. Imre Lakatos, “Falsification and the Methodology of Scientific Research Programmes”, in Criticism and the Growth of Knowledge, ed. Imre Lakatos and Alan Musgrave (Cambridge: Cambridge University Press, 1970), 91-196.
116. Allison, Essence of Decision.
117. Similarly, if the rational-unitary model cannot explain state behavior in an international crisis as acute as the one in 1962, we would have little confidence that it could explain behavior in situations of noncrisis decision making.
118. George, “Case Studies and Theory Development”.
119. George and Bennett, Case Studies and Theory Development.
120. George and Bennett, Case Studies and Theory Development.
121. Ernest Nagel, The Structure of Science (Indianapolis: Hackett, 1979), 564-68.
122. W. B. Gallie, “The Historical Understanding”, History and Theory 3, no. 2 (1963): 149-202.
123. Carl G. Hempel, “The Function of General Laws in History”, Journal of Philosophy 39 (1942): 35-48.
124. Experimental methods may be superior for testing many of these hypotheses because of their ability to control for extraneous variables. It is often difficult, however, to generalize from laboratory settings that do not incorporate the relevant political variables and that cannot fully replicate the stakes and emotions inherent in the contexts of foreign policy decision making.
125. Sidney Tarrow, “Bridging the Quantitative-Qualitative Divide in Political Science”, American Political Science Review 89, no. 2 (1995): 474.
126. Paul Pierson, “Increasing Returns, Path Dependence, and the Study of Politics”, American Political Science Review 94, no. 2 (2000): 251-67.
127. Ira Katznelson, “Periodization and Preferences: Contributions of Comparative Historical Social Science” (manuscript, Columbia University, 2000).
128. Andrew Bennett and Alexander L. George, “The Alliance of Statistical and Case Study Methods: Research on the Interdemocratic Peace”, APSA-CP Newsletter 9, no. 1 (1998): 6-9.
129. A good example of a large-N study that empirically distinguishes between alternative causal mechanisms within the black box of decision making is Schultz’s statistical test of competing institutional constraint and informational signaling models of the democratic peace proposition.
130. Kenneth A. Schultz, “Do Democratic Institutions Constrain or Inform? Contrasting Two Perspectives on Democracy and War”, International Organization 53, no. 2 (1999): 233-66.
131. Many of those committed to large-N analysis exaggerate in the opposite directions.
132. Clayton Roberts, The Logic of Historical Explanation (University Park: Pennsylvania State University Press, 1996).
133. Thomas D. Cook and Donald T. Campbell, Quasi-Experimentation (Chicago: Rand McNally, 1979), ch. 1.
134. If theories have no observable consequences, they cannot explain the variation in outcomes that is the primary task of social science, and hence those theories have no explanatory power. This does not imply that all theories are easy to test.
135. For an argument that hypotheses involving necessary conditions are more common, see Gary Goertz and Harvey Starr, eds., Necessary Conditions: Theory, Methodology, and Applications (Lanham, Md.: Rowman and Littlefield, forthcoming).
136. These are very demanding criteria. They lead Lieberson to conclude that Mill’s methods cannot be applied if the number of cases is small because they do “not allow for probabilistic theories, interaction effects, measurement errors, or even the presence of more than one cause”.
137. Stanley Lieberson, “Small N’s and Big Conclusions”, in Ragin and Becker, What Is a Case? 105-18.
138. Stanley Lieberson, “More on the Uneasy Case for Using Mill-Type Methods in Small-N Comparative Studies”, Social Forces 72 (June 1994): 1225-37.
139. Multiple causes and interaction effects can probably be dealt with, but only by increasing the number of cases in order to encompass all possible interaction effects. This creates a practical problem for intensive case study methods. Some case study researchers acknowledge this problem and supplement Mill’s methods with process tracing.
140. (George and Bennett, Case Studies and Theory Development), While others have adopted a Boolean-based “qualitative comparative analysis”.
141. (Ragin, The Comparative Method).
142. The large-N researcher makes the opposite trade-off.
143. Whether social science theories can have causal mechanisms that are truly probabilistic—as opposed to those that reflect measurement error, the misspecification of functional forms, or omitted variables that have yet to be understood and specified—is quite problematic.
144. See Wesley C. Salmon, Four Decades of Scientific Explanation (Minneapolis: University of Minnesota Press, 1990).
145. and George and Bennett, Case Studies and Theory Development.
146. I prefer to speak of a theory’s probabilistic testable implications rather than probabilistic causal mechanisms. My thinking (admittedly provisional) on this complex issue has benefited from conversations with Andrew Bennett.
147. Ragin and Zaret, 1983, 744.
148. Ragin, The Comparative Method, 15.
149. Ragin’s exploration of the relevance of the logic of fuzzy sets for social science analysis is an attempt to deal with the problem of measurement error and probabilistic testable implications.
150. Ragin, Fuzzy-Set Social Science.
151. Becker, “Cases, Causes, Conjunctures, Stories, and Imagery”, 212.
152. Dorothy Vaughan, “Theory Elaboration: The Heuristics of Case Analysis”, in Ragin and Becker, What Is a Case? 184.
153. Proponents of “offense/defense theory”, for example, theorize about whether military technology favors the offense or defense but fail to establish the magnitude of this effect relative to that of the balance of material capabilities and related variables.
154. Sean M. Lynn-Jones, “Offense-Defense Theory and Its Critics”, Security Studies 4, no. 4 (1995): 660-91.
155. Sartori, “Concept Misinformation in Comparative Politics”.
156. Collier and Mahoney, “Conceptual Stretching Revisited”.
157. Skocpol and Somers, “The Uses of Comparative History in Macrosocial Inquiry”.
158. Paul Huth, Extended Deterrence and the Prevention of War (New Haven, Conn.: Yale University Press, 1988)
159. Ray, Democracies and International Conflict.
160. Lisa L. Martin, Coercive Cooperation (Princeton, N.J.: Princeton University Press, 1992)
161. Beth A. Simmons, Who Adjusts? (Princeton, N.J.: Princeton University Press, 1994).
162. Brecher, The Foreign Policy System of Israel.
163. Michael Brecher, with Benjamin Geist, Decisions in Crisis: Israel, 1967 and 1973 (Berkeley: University of California Press, 1980)
164. Michael Brecher and Jonathan Wilkenfeld, A Study of Crisis (Ann Arbor: University of Michigan Press, 1997).
165. Michael Doyle, “Liberalism and World Politics”, American Political Science Review 80, no. 4 (1986): 1151-70.
166. Michael Doyle, Ways of War and Peace (New York: Norton, 1997).
167. Edward D. Mansfield and Jack Snyder, “Democratization and the Danger of War”, International Security 20, no. 1 (1995): 5-38.
168. Jack Snyder, From Voting to Violence: Democratization and Nationalist Conflict (New York: Norton, 2000).
169. Bruce M. Russett, “International Behavior Research: Case Studies and Cumulation”, in Approaches to the Study of Political Science, ed. Michael Haas and Henry S. Kariel (Scranton, Penn.: Chandler, 1970), 425-43.
170. Lijphart, “Comparative Politics and the Comparative Method”.
171. Bennett and George, “The Alliance of Statistical and Case Study Methods”.
172. Jack S. Levy, “Preferences, Constraints, and Choices in July 1914”, International Security 15, no. 3 (1990-91): 151-86.
173. Bates et al., Analytic Narratives.
174. Bruce Bueno de Mesquita, “Popes, Kings, and Endogenous Institutions: The Concordat of Worms and the Origins of Sovereignty”, International Studies Review 2, no. 2 (2000): 93-118.