How good is good enough? a multidimensional, best-possible standard for research design

Political Research Quarterly

Volumn 64, Issue 3, 2011, Pages 625-636

  Gerring, John ^a  

^a Boston University   (United States)

Author keywords

natural experiment; qualitative methods; quasi experiment; research design

Indexed keywords

EID: 80052197173     PISSN: 10659129     EISSN: None     Source Type: Journal    
DOI: 10.1177/1065912910361221     Document Type: Article

References (66)

1. Barrett, Christopher, and Jeffery Cason. 1997. Overseas research. Baltimore: Johns Hopkins University Press.
2. Bennett, Andrew. 1999. Causal inference in case studies: From Mill's methods to causal mechanisms. Paper presented at the annual meetings of the American Political Science Association, Atlanta, GA, September.
3. Berk, Richard A. 2005. Randomized experiments as the bronze standard. Unpublished manuscript.
4. Berk, Richard A., A. Campbell, R. Klapp, and Bruce Western. 1992. The differential deterrent effects of an arrest in incidents of domestic violence: A Bayesian analysis of four randomized field experiments. American Sociological Review 5:689-708.
5. Blalock, Hubert M., Jr. 1979. Social statistics. 2nd ed. New York: McGraw-Hill.
6. Bloom, H. S., C. J. Hill, and J. A. Riccio. 2002. Linking program implementation and effectiveness: Lessons from a pooled sample of welfare-to-work experiments. Journal of Policy Analysis and Management 22:551-75.
7. Brady, Henry E., and David Collier, eds. 2004. Rethinking social inquiry: Diverse tools, shared standards. Lanham, MD: Rowman & Littlefield.
8. Breslow, N. E. 1996. Statistics in epidemiology: The casecontrol study. Journal of the American Statistical Association 91:14-28.
9. Campbell, Donald T. 1966. Pattern matching as an essential in distal knowing. In The psychology of Egon Brunswick, ed. K. R. Hammond. New York: Holt, Rinehart and Winston.
10. Campbell, Donald T. 1988. Methodology and epistemology for social science. Edited by E. Samuel Overman. Chicago: University of Chicago Press.
11. Campbell, Donald T., and Julian Stanley. 1963. Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.
12. Dion, Douglas. 1998. Evidence and inference in the comparative case study. Comparative Politics 30 (2): 127-45.
13. Dunning, Thad. 2008. Improving causal inference: Strengths and limitations of natural experiments. Political Research Quarterly 61:282-93.
14. Eckstein, Harry. 1975. Case studies and theory in political science. In Handbook of political science, vol. 7, Political science: Scope and theory, ed. Fred I. Greenstein and Nelson W. Polsby, 94-137. Reading, MA: Addison-Wesley.
15. Eckstein, Harry. 1992. Regarding politics: Essays on political theory, stability, and change. Berkeley: University of California Press.
16. Elman, Miriam Fendius. 1997. Paths to peace: Is democracy the answer? Cambridge, MA: MIT Press.
17. Fearon, James. 1991. Counter factuals and hypothesis testing in political science. World Politics 43 (January): 169-95.
18. Feyerabend, Paul. 1963. How to be a good empiricist: A plea for tolerance in matters epistemological. Philosophy of Science: The Delaware Seminar 2:3-39.
19. Friedman, Milton. 1953. The methodology of positive economics. In Essays in positive economics, 3-43. Chicago: University of Chicago Press.
20. Geddes, Barbara. 2007. What causes democratization? In The Oxford handbook of comparative politics, ed. Carles Boix and Susan Stokes, 317-39. Oxford: Oxford University Press.
21. George, Alexander L., and Andrew Bennett. 2005. Case studies and theory development. Cambridge, MA: MIT Press.
22. George, Alexander L., and Timothy J. McKeown. 1985. Case studies and theories of organizational decision making. Vol. 2 of Advances in information processing in organizations. Santa Barbara, CA: JAI.
23. Gerber, Alan S., Donald P. Green, and Edward H. Kaplan. 2004. The illusion of learning from observational research. In Problems and methods in the study of politics, ed. IanShapiro, Rogers M. Smith, and Tarek E. Masoud, 251-73. Cambridge: Cambridge University Press.
24. Gerber, Alan S., Donald P. Green, and David Nickerson. 2001. Testing for publication bias in political science. Political Analysis 9 (4): 385-92.
25. Gerring, John. 2007. Case study research: Principles and practices. Cambridge: Cambridge University Press.
26. Gerring, John. 2011. Social science methodology: Tasks, strategies, and criteria. Cambridge: Cambridge University Press.
27. Glynn, Adam N., and Kevin M. Quinn. 2009. Why process matters for causal inference. Unpublished manuscript, Department of Government, Harvard University, Cambridge, MA.
28. Goldstone, Jack A. 1997. Methodological issues in comparative macrosociology. Comparative Social Research 16:107-20.
29. Goldthorpe, John H. 1997. Current issues in comparative macrosociology: A debate on methodological issues [with response to commentaries]. Comparative Social Research 16:121-32.
30. Goodin, Robert, and Charles Tilly, eds. 2006. The Oxford handbook of contextual analysis. Oxford: Oxford University Press.
31. Harman, Gilbert. 1965. The inference to the best explanation. Philosophical Review 74:88-95.
32. Holland, Paul W. 1986. Statistics and causal inference. Journa of the American Statistical Association 81 (396): 945-60.
33. Hume, David. 1960. The idea of necessary connexion [from An enquiry concerning human understanding, sec. 7]. In The structure of scientific thought: An introduction to philosophy of science, ed. Edward H. Madden. London: Routledge & Kegan Paul.
34. Imbens, Guido W., and Jeffrey M. Wooldridge. 2009. Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47 (1): 5-86.
35. Kelman, Herbert C. 1982. Ethical issues in different social science methods. In Ethical issues in social science research, ed. Tom L. Beauchamp, Ruth R. Faden, R. Jay Wallace Jr., and LeRoy Walters, 40-98. Baltimore: Johns Hopkins University Press.
36. Kenny, Charles, and David Williams. 2000. What do we know about economic growth? Or, why don't we know very much? World Development 29 (1): 1-22.
37. King, Gary. 1995. Replication, replication. PS: Political Science and Politics 28 (3): 443-99.
38. King, Gary, Robert O. Keohane, and Sidney Verba. 1994. Designing social inquiry: Scientific inference in qualitative research. Princeton, NJ: Princeton University Press.
39. Lebow, Richard Ned. 2007. Counterfactual thought experiments: A necessary teaching tool. History Teacher 40 (2): 153-76.
40. Lieberman, Evan S., Marc Howard, and Julia Lynch. 2004. Symposium: Field research. Qualitative Methods 2 (1): 2-15.
41. Lieberson, Stanley. 1985. Making it count: The improvement of social research and theory. Berkeley: University of California Press.
42. Manski, Charles F. 1995. Identification problems in the social sciences. Cambridge, MA: Harvard University Press.
43. Marini, Margaret, and Burton Singer. 1988. Causality in the social sciences. Sociological Methodology 18:347-409.
44. Mazur, Dennis J. 2007. Evaluating the science and ethics of research on humans: A guide for IRB members. Baltimore: Johns Hopkins University Press.
45. Miguel, Edward, and Michael Kremer. 2004. Worms: Identifying impacts on education and health in the presence of treatment externalities. Econometrica 72 (1): 193-217.
46. Mill, John Stuart. 1843/1872. System of logic. 8th ed. London: Longmans, Green.
47. Miller, Richard W. 1987. Fact and method: Explanation, confirmation and reality in the natural and the social sciences. Princeton, NJ: Princeton University Press.
48. Morgan, Stephen L., and Christopher Winship. 2007. Counterfactuals and causal inference: Methods and principles for social research. Cambridge: Cambridge University Press.
49. Neuman, W. Lawrence. 1997. Social research methods: Qualitative and quantitative approaches. 2nd ed. Boston: Allyn & Bacon.
50. Paluck, Elizabeth Levy. 2008. Methods and ethics with research teams and NGOs: Comparing experiences across the border of Rwanda and Democratic Republic of Congo. Unpublished manuscript, Princeton University, Princeton, NJ.
51. Pearl, Judea. 2009. Causal inference in statistics: An overview. Statistics Surveys 3: 96-146.
52. Popper, Karl. 1934/1968. The logic of scientific discovery. New York: Harper & Row.
53. Rindskopf, David. 2000. Plausible rival hypotheses in measurement, design, and scientific theory. In Research design: Donald Campbell's legacy, vol. II, ed. Leonard Bickman, 1-12. Thousand Oaks, CA: Sage.
54. Robinson, Gregory, John E. McNulty, and Jonathan S. Krasno. 2009. Observing the counterfactual? The search for political experiments in nature. Political Analysis 17 (4): 341-57.
55. Rosenbaum, Paul R. 1984. From association to causation in observational studies: The role of strongly ignorable treatment assignment. Journal of the American Statistical Association 79 (385): 41-48.
56. Rosenbaum, Paul R. 1999. Choice as an alternative to control in observational studies. Statistical Science 14 (3): 259-304.
57. Rosenbaum, Paul R. 2007. Interference between units in randomized experiments. Journal of the American Statistical Association 102 (477): 191-200.
58. Rubin, Donald B. 2008. For objective causal inference, design trumps analysis. Annals of Applied Statistics 2 (3): 808-40.
59. Sales, Bruce Dennis, and Susan Folkman, eds. 2000. Ethics in research with human participants. Washington, DC: American Psychological Association.
60. Samuelson, Paul A. 1959. What economists know. In The human meaning of the social sciences, ed. Daniel Lerner. New York: Meridian Books.
61. Sekhon, Jasjeet S. 2009. Opiates for the matches: Matching methods for causal inference. Annual Review of Political Science 12:487-508.
62. Shadish, William R., and Thomas D. Cook. 1999. Design rules: More steps toward a complete theory of quasi-experimentation. Statistical Science 14 (3): 294-300.
63. Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
64. Trochim, William M. K. 1989. Outcome pattern matching and program theory. Evaluation and Program Planning 12:355-66.
65. Van Evera, Stephen. 1997. Guide to methods for students of political science. Ithaca, NY: Cornell University Press.
66. Wood, Elisabeth. 2006. The ethical challenges of field research in conflict zones. Qualitative Sociology 29 (3): 373-86.