Are survey experiments externally valid?

American Political Science Review

Volumn 104, Issue 2, 2010, Pages 226-242

  Barabas, Jason ^a   Jerit, Jennifer ^a  

^a FLORIDA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77958505865     PISSN: 00030554     EISSN: 15375943     Source Type: Journal    
DOI: 10.1017/S0003055410000092     Document Type: Article

References (85)

1. Abrajano, Marisa A. 2005. "Who Evaluates a Presidential Candidate by Using Non-policy Campaign Messages?" Political Research Quarterly 58 (March): 55-67.
2. Achen, Chris. 1982. Interpreting and Using Regression. Thousand Oaks, CA: Sage.
3. Arceneaux, Kevin, and Martin Johnson. 2007. "Channel Surfing: Does Choice Reduce Videolmalaise." Presented at the 2007 Annual Meeting of the Midwest Political Science Association, Chicago.
4. Arceneaux, Kevin, and Robin Kolodny. 2009. "Educating the Least Informed: Group Endorsements in a Grassroots Campaign." American Journal of Political Science 53 (October): 755-70.
5. Barabas, Jason. 2004. "How Deliberation Affects Policy Opinions." American Political Science Review 98 (November): 687-701.
6. Barabas, Jason, and Jennifer Jerit. 2009. "Estimating the Causal Effects of Media Coverage on Policy-specific Knowledge." American Journal of Political Science 53 (January): 73-89.
7. Bennett, W. Lance. 2006. News: The Politics of Illusion. 7th ed. New York: Longman.
8. Benz, Matthias, and Stephan Meier. 2008. "Do People Behave in Experiments as in the Field? Evidence from Donations." Experimental Economics 11: 268-81.
9. Berinsky, Adam J. 2007. "Assuming the Costs of War: Events, Elites, and American Public Support for Military Conflict." Journal of Politics 69 (November): 975-97.
10. Blalock, H.M., Jr. 1979. Social Statistics. 2nd ed. New York: McGraw-Hill.
11. Brader, Ted. 2005. "Striking a Responsive Chord: How Political Ads Motivate and Persuade Voters by Appealing to Emotions." American Journal of Political Science 49 (April): 388-405.
12. Brehm, John. 1993. The Phantom Respondents: Opinion Surveys and Political Representation. Ann Arbor: University of Michigan Press.
13. Cameron, A. Colin, and Pravin K. Trivedi. 2005. Microeconomet-rics: Methods and Applications. New York: Cambridge University Press.
14. Campbell, Donald T., and Julian C. Stanley. 1963. Experimental and Quasi-experimental Designs for Research. Chicago: Rand McNally.
15. Chong, Dennis, and James N. Druckman. 2007. "Framing Public Opinion in Competitive Democracies." American Political Science Review 101 (November): 637-56.
16. Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum.
17. Converse, Philip E. 1964. "The Nature of Belief Systems in Mass Publics." In Ideology and Discontent, ed. David E. Apter. New York: Free Press.
18. Cook, Fay Lomax, Lawrence R. Jacobs, and Dukhong Kim. 2010. "Trusting What You Know: Information, Knowledge, and Confidence in Social Security." Journal of Politics 72 (April): 397-412.
19. Cook, Thomas D., and Donald T. Campbell. 1979. Quasi-experimentation: Design & Analysis Issues for Field Settings. New York: Houghton Mifflin.
20. Cook, Thomas D., William R. Shadish, and Vivian C. Wong. 2008. "Three Conditions under Which Experimental and Observational Studies Produce Comparable Causal Estimates: New Findings from Within-study Comparisons." Journal of Policy Analysis and Management 27 (Autumn): 724-50.
21. Delli Carpini, Michael X. 2009. "The Psychology of Citizen Learning." In The Political Psychology of Democratic Citizenship, eds. Eugene Borgida, Christopher M. Federico, and John L. Sullivan. New York: Oxford University Press, 23-51.
22. Druckman, James N. 2004. "Political Preference Formation: Competition, Deliberation, and the (Ir)relevance of Framing Effects." American Political Science Review 98 (November): 671-86.
23. Druckman, James N., Donald P. Green, James H. Kuklinski, and Arthur Lupia. 2006. "The Growth and Development of Experimental Research in Political Science." American Political Science Review 100 (November): 627-35.
24. Druckman, James N., and Kjersten R. Nelson. 2003. "Framing and Deliberation: How Citizens' Conversations Limit Elite Influence." American Journal of Political Science 47 (October): 729-45.
25. Dunning, Thad. 2008. "Improving Causal Inference: Strengths and Limitations of Natural Experiments." Political Research Quarterly 61 (2): 282-93.
26. Eagly, Alice, and Shelly Chaiken. 1993. The Psychology of Attitudes. New York: Harcourt Brace.
27. Gaines, Brian J., and James H. Kuklinski. 2009. "Including Self-selection in Random Assignment Experiments." University of Illinois at Urbana-Champaign. Unpublished manuscript.
28. Gaines, Brian J., James H. Kuklinski, and Paul J. Quirk. 2007. "The Logic of the Survey Experiment Reexamined." Political Analysis 15 (Winter): 1-20.
29. Gelman, Andrew. 2007. "Struggles with Survey Weighting and Regression Modeling." Statistical Science 22 (2): 153-64.
30. Gerber, Alan, and Donald P. Green. 2000. "The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment." American Political Science Review 94 (September): 653-63.
31. Gibson, James. 1998. "A Sober Second Thought: An Experiment in Persuading Russians to Tolerate." American Journal of Political Science 42 (July): 819-50.
32. Gilens, Martin. 2001. "Political Ignorance and Collective Policy Preferences." American Political Science Review 95 (June): 379-96.
33. Gilens, Martin. 2002. "An Anatomy of Survey-Based Experiments." In Navigating Public Opinion. eds. J. Manza, F. L. Cook, and B. I. Page. New York: Oxford, 232-50.
34. Gramlich, Edward M. 1998. Is It Time to Reform Social Security? Ann Arbor: University of Michigan Press.
35. Grosser, Jens, and Arthur Schram. 2006. "Neighborhood Information Exchange and Voter Participation: An Experimental Study." American Political Science Review 100 (May): 235-48.
36. Hovland, Carl. 1959. "Reconciling Conflicting Results Derived from Experimental and Survey Studies of Attitude Change." American Psychologist 14: 8-17.
37. Huber, Gregory A., and Kevin Arceneaux. 2007. "Identifying the Persuasive Effects of Presidential Advertising." American Journal of Political Science 51 (October): 957-77.
38. Iyengar, Shanto, and Donald Kinder. 1987. News That Matters. Chicago: University of Chicago Press.
39. Jacobs, Lawrence, R., Fay Lomax Cook, and Michael X. Delli Carpini. 2009. Talking Together: Public Deliberation and Political Participation in America. Chicago: University of Chicago Press.
40. Jacobs, Lawrence R., and Robert Y. Shapiro. 1998. "Myths and Misunderstandings about Public Opinion toward Social Security." In Framing the Social Security Debate: Values, Politics, and Economics, eds. R. Douglas Arnold, Michael J. Graetz, and Alicia H. Munnell. Washington, DC: Brookings Institution Press, 355-88.
41. Jerit, Jennifer. 2009. "How Predictive Appeals Shape Policy Opinions." American Journal of Political Science 53 (April): 411-26.
42. Jerit, Jennifer, and Jason Barabas. 2006. "Bankrupt Rhetoric: How Misleading Information Affects Knowledge about Social Security." Public Opinion Quarterly 70 (Fall): 278-303.
43. Jerit, Jennifer, James H. Kuklinski, and Paul J. Quirk. 2009. "Strategic Rhetoric, Emotional Citizens, and the Rhetoric of Prediction." In The Political Psychology of Democratic Citizenship, eds. Eugene Borgida, John L. Sullivan, and Christopher M. Federico. New York: Oxford University Press, 100-24.
44. Kam, Cindy D., Jennifer R. Wilking, and Elizabeth J. Zechmeis-ter. 2007. "Beyond the 'Narrow Database': Another Convenience Sample for Experimental Research." Political Behavior 29 (December): 415-40.
45. Kellstedt, Paul M., and Guy D. Whitten. 2009. The Fundamentals of Political Science Research. New York: Cambridge University Press.
46. Kennedy, Peter. 2003. A Guide to Econometrics. 5th ed. Cambridge, MA: MIT Press.
47. Keppel, Geoffrey. 1982. Design and Analysis: A Researcher's Handbook. 2nd ed. Englewood Cliffs, NJ: Prentice-Hall.
48. Kinder, Donald. 2007. "Curmudgeonly Advice." Journal of Communication 57 (March): 155-62.
49. Kinder, Donald R., and Thomas R. Palfrey, eds. 1993. Experimental Foundations of Political Science. Ann Arbor: University of Michigan Press.
50. Kinder, Donald R., and Lynn M. Sanders. 1996. Divided by Color: Racial Politics and Democratic Ideals. Chicago: University of Chicago Press.
51. King, Gary, Robert O. Keohane, and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press.
52. Krosnick, Jon A., and Howard Schuman. 1988. "Attitude Intensity, Importance, and Certainty and Susceptibility to Response Effects." Journal of Personality and Social Psychology 54 (6): 940-52.
53. Kuklinski, James H., Paul J. Quirk, Jennifer Jerit, and Robert F. Rich. 2001. "The Political Environment and Citizen Competence." American Journal of Political Science 45 (April): 410-24.
54. Kuklinski, James H., Paul J. Quirk, Jennifer Jerit, David Schwieder, and Robert F. Rich. 2000. "Misinformation and the Currency of Citizenship." Journal of Politics 62 (August): 790-816.
55. Lassen, David Dreyer. 2005. "The Effect of Information on Voter Turnout: Evidence from a Natural Experiment." American Journal of Political Science 49 (January): 103-88.
56. Lavine, Howard. 2002. "On-line versus Memory Based Process Models of Political Evaluation." In Political Psychology, ed. Kristen Monroe. Mahwah, NJ: Lawrence Erlbaum Associates, 225-47.
57. Levitt, Steven D., and John A. List. 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal about the Real World?" Journal of Economic Perspectives 21 (Spring): 153-71.
58. Loughin, Thomas M. 2004. "A Systematic Comparison of Methods for Combining p-Values from Independent Tests." Computational Statistics and Data Analysis 47: 467-85.
59. Lupia, Arthur, Yanna Krupnikov, Adam Seth Levine, Casandra Grafstrom, William McMillian, and Erin McGovern. n.d. "How 'Point Blindness' Dilutes the Value of Stock Market Reports." Political Communication. Forthcoming.
60. Luskin, Robert C., James S. Fishkin, and Roger Jowell. 2002. "Considered Opinions: Deliberative Polling in Britain." British Journal of Political Science 32: 455-87.
61. Marmor, Theodore R. 2000. The Politics of Medicare. 2nd ed. NY: Aldine de Gruyter.
62. McClelland, Gary H., and Charles M. Judd. 1993. "Statistical Difficulties of Detecting Interactions and Moderator Effects." Psychological Bulletin 114 (2): 376-90.
63. McDermott, Rose. 2002. "Experimental Methods in Political Science."Annual Review of Political Science 5: 31-61.
64. Mondak, Jeffery J. 1995. "Newspapers and Political Awareness." American Journal of Political Science 39 (May): 513-27.
65. Mutz, Diana, and Byron Reeves. 2005. "The New Videomalaise: Effects of Televised Civility on Political Trust." American Political Science Review 99 (February): 1-15.
66. Nadeau, Richard, and Richard G. Niemi. 1995. "Educated Guesses: The Process of Answering Factual Knowledge Questions in Surveys." Public Opinion Quarterly 59 (Autumn): 323-46.
67. Norris, Pippa, and David Sanders. 2003. "Message or Medium? Learning during the 2001 British General Election." Political Communication 20: 233-62.
68. Oberlander, Jonathan. 2003. The Political Life of Medicare. Chicago: University of Chicago Press.
69. O'Keefe, Daniel J. 2007. "Post Hoc Power, Observed Power, A Priori Power, Retrospective Power, Prospective Power, Achieved Power: Sorting Out Appropriate Uses of Statistical Power Analyses." Communication Methods and Measures 1(4): 291-99.
70. Page, Benjamin I. 2000. "How Public Opinion Affects Reform: Is Social Security Reform Ready for the American Public?" In Social Security and Medicare: Individual versus Collective Risk and Responsibility, eds. Shelia Burke, Eric Kingson, and Uwe Rein-hardt. Washington, DC: National Academy of Social Insurance, 183-218.
71. Page, Benjamin I., and Robert Y. Shapiro. 1992. The Rational Public: Fifty Years of Trends in Americans' Policy Preferences. Chicago: University of Chicago Press.
72. Prior, Marcus, and Arthur Lupia. 2008. "Money, Time, and Political Knowledge: Distinguishing Quick Recall and Political Learning Skills." American Journal of Political Science 52 (January): 169-83.
73. Robinson, Gregory, John E. McNulty, and Jonathan S. Krasno. 2009. "Observing the Counterfactual? The Search for Political Experiments in Nature." Political Analysis 17: 341-57.
74. Roth, Alvin. 1995. "Introduction to Experimental Economics." In The Handbook of Experimental Economics, eds. John H. Kagel and Alvin E. Roth. Princeton, NJ: Princeton University Press, 3-109.
75. Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
76. Shaw, Greg M., and Sarah E. Mysiewicz. 2004. "Trends: Social Security and Medicare." Public Opinion Quarterly 68 (Fall): 394-423.
77. Sniderman, Paul M., and Thomas Piazza. 1993. The Scar of Race. Cambridge, MA: Harvard University Press.
78. Sniderman, Paul M., and Sean M. Theriault. 2004. "The Structure of Political Argument and the Logic of Issue Framing." In Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change, eds. Willem E. Saris and Paul M. Sniderman. Princeton, NJ: Princeton University Press, 133-64.
79. Turgeon, Mathieu. 2009. "'Just Thinkin': Attitude Development, Public Opinion, and Political Representation." Political Behavior 31 (September): 353-78.
80. Weisberg, Herbert, F. 2005. The Total Survey Error Approach: A Guide to the New Science of Survey Research. Chicago: University of Chicago Press.
81. Winship, Christopher, and Larry Radbill. 1994. "Sampling Weights and Regression Analysis." Sociological Methods and Research 23 (2): 230-57.
82. Wooldridge, Jeffrey M. 2009. Introductory Economics: A Modern Approach. 4th ed. Mason, OH: South-Western Cengage Learning.
83. Zaller, John R. 1992. The Nature and Origins of Mass Opinion. New York: Cambridge.
84. Zaller, John R. 2002. "The Statistical Power of Election Studies to Detect Media Exposure Effects in Political Campaigns." Electoral Studies 21: 297-329.
85. Zaller, John R., and Stanley Feldman. 1992. "A Simple Theory of the Survey Response: Answering Questions versus Revealing Preferences." American Journal of Political Science 36 (August): 579-616.