Nonresponse and mode effects in self- and interviewer-administered surveys

Political Analysis

Volumn 22, Issue 3, 2014, Pages 304-320

  Atkeson, Lonna Rae ^a   Adams, Alex N ^a   Alvarez, R Michael ^b  

^a UNIVERSITY OF NEW MEXICO   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84903948658     PISSN: 10471987     EISSN: 14764989     Source Type: Journal    
DOI: 10.1093/pan/mpt049     Document Type: Article

References (88)

1. Adams, Alex, Lonna Atkeson, and Jeffrey Karp. 2013. Measurement error in online survey panels: Panel conditioning and data quality. Presented at the American Panel Survey (TAPS) Workshop, Washington University, St. Louis, November 8-9.
2. American Association for Public Opinion Research. 2010. AAPOR report on online panels. http://www.aapor.org/ AAPOR_Releases_Report_on_Online_Survey_Panels/2263.htm (accessed January 10, 2014).
3. The American Association for Public Opinion Research. 2011. Standard definitions: Final dispositions of case codes and outcome rates for surveys, 7th edition. AAPOR. http://aapor.org/Content/NavigationMenu/AboutAAPOR/ StandardsampEthics/StandardDefinitions/StandardDefinitions2011.pdf (accessed January 10, 2014).
4. Alexander, C. H. Jr., and S. Wetrogan. 2000. Integrating the American Community Survey and the Intercensal Demographic Estimates Program. Proceedings of the American Statistical Association at www.amstat.org/sections/ srms/proceedings/papers/2000_046.pdf (accessed January 10, 2014).
5. Aquilino, W. 1994. Interview mode effects in surveys of drug and alcohol use. Public Opinion Quarterly 58:210-40.
6. Atkeson, Lonna Rae, Alex N. Adams, and R. Michael Alvarez. 2013. Replication data for: Nonresponse and mode effects in self- and interviewer-administered surveys. http://dx.doi.org/10.7910/DVN/23773 UNF:5:QH1dvPfRoJ+lbQkO32oeww== IQSS Dataverse Network [Distributor] V1 [Version].
7. Atkeson, Lonna Rae, and Alex N. Adams. 2010. Mixed mode (Internet and Mail) probability samples and survey representativeness: The case of New Mexico 2008. San Francisco, CA: Presented at the Western Political Science Association, April 1-4.
8. Atkeson, Lonna Rae, Alex N. Adams, Lisa A. Bryant, Luciana Zilberman, and Kyle L. Saunders. 2011. Considering mixed mode surveys for questions in political behavior: Using the Internet and mail to get quality data at reasonable costs. Political Behavior 33(1):161-78.
9. Atkeson, Lonna Rae, and Lori Tafoya. 2008. Surveying political activists: An examination of the effectiveness of a mixedmode (Internet and mail) survey design. Journal of Elections Public Opinion and Parties 18(4):367-86.
10. Atkeson, Lonna Rae, R. Michael Alvarez, Alex N. Adams, and Lisa Bryant. 2011. The 2010 New Mexico election administration report. Typescript, University of New Mexico. http://polisci.unm.edu/common/documents/c-sved/ papers/nm-2010-general-election.pdf (accessed January 10, 2014).
11. Atkeson, Lonna Rae, Lisa A. Bryant, and Alex N. Adams. 2013. The 2012 Bernalillo County election administration report. Typescript, University of New Mexico. http://polisci.unm.edu/common/documents/c-sved/papers/ BernCoFinalReport.pdf (accessed January 10, 2014).
12. Atkeson, Lonna Rae, R. Michael Alvarez, and Thad E. Hall. 2010. Assessing electoral performance in New Mexico using an eco-system approach: New Mexico 2008. Typescript, University of New Mexico. http://polisci.unm.edu/common/ documents/c-sved/papers/nm08pew.pdf (accessed January 10, 2014).
13. Balden, W. 2004. Multi-mode data collection: Benefits and downsides. Paper presented at the 2004 Conference of the Great Lakes. Cancun, Mexico: Chapter of the Marketing Research Association.
14. Belli, Robert F., Michael W. Traugott, and Matthew N. Beckmann. 2001. What leads to voting overreports? Contrasts of overreporters and admitted nonvoters in the American National Election Studies. Journal of Official Statistics 17(4):479-98.
15. Blair, Graeme, and Kosuke Imai. 2012. Statistical analysis of list experiments. Political Analysis 20:47-77.
16. Blumberg, Stephen J., and Julian V. Luke. 2009. Wireless substitution: Early release of estimates from the National Health Interview Survey. July-December 2008. http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless200905.htm.
17. Blumberg, Stephen J. 2013. Wireless substitution: Early release of estimates from the National Health Interview Survey. July-December 2012. http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201306.pdf.
18. Burden, Barry C. 2000. Voter turnout and the National Election Studies. Political Analysis 8(4):389-98.
19. Burden, Barry C. 2003. Internal and external effects on the accuracy of NES turnout: Reply. Political Analysis 11(2):193-95.
20. Campbell, Donald T. 1950. The indirect assessment of social attitudes. Psychological Bulletin 47):15-38.
21. Cannell, C. F., P. V. Miller, and L. Oksenberg. 1981. Research on interviewing techniques. In Sociological Methodology, ed. S. Leinhardt, 389-437. San Francisco, CA: Jossey-Bass.
22. Carley-Baxter, Lisa R., Andy Peytchev, and Michele C. Black. 2010. Comparison of cell phone and landline surveys: A design perspective. Field Methods 22(1):3-15.
23. Chang, LinChiat, and Jon Krosnick. 2009. National surveys via RDD telephone interviewing versus the Internet: Comparing sample representativeness and response quality. Public Opinion Quarterly 73(4):641-78.
24. Chang, LinChiat. 2010. Comparing oral interviewing with self-administered computerized questionnaires: An experiment. Public Opinion Quarterly 74(1):154-67.
25. Christian Leah M., and Don A. Dillman. 2004. The influence of graphical and symbolic language manipulations on responses to self-administered questions. Public Opinion Quarterly 68(1):57-80.
26. Curtin, Richard, Stanley Presser, and Eleanor Singer. 2000. The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly 64:413-28.
27. Davis, James Allan, Tom Smith, and Peter V. Marsden. 2009. General social surveys, 1972-2008: Cumulative codebook. Chicago: National Opinion Research Center.
28. Day, N. A., D. R. Dunt, and S. Day. 1995. Maximizing response to surveys in health program evaluation at minimum cost using multiple methods. Evaluation Review 19(4):436-50.
29. de Leeuw, Edith. 2005. To mix or not to mix: Data collection modes in surveys. Journal of Official Statistics 21:233-55.
30. de Leeuw, E., J. Hox, E. Korendijk, G. Lensvelt-Mulders, and M. Callegaro. 2004. The influence of advance letters on response in telephone surveys: A meta-analysis. Maastricht: Paper presented at the 15 International Workshop on Household Survey Nonresponse.
31. de Leeuw, Edith D., and W. de Heer. 2002. Trends in household survey nonresponse: A longitudinal and international comparison. In Survey nonresponse, eds. Robert M. Groves, Don A. Dillman, John L. Eltinge, and Roderick J. A. Little. New York: Wiley.
32. de Leeuw, Edith, and J. Van der Zouwen. 1988. Data quality in telephone and face-to-face surveys: A comparative metaanalysis. In Telephone survey methodology, eds. R. Groves, P. P. Bimer, L. Lyberg, I. T. Massey, W. L. Nicholls, and J. Waksberg. New York: John Wiley & Sons.
33. DeBell, Mathew, Catherine Wilson, Gary Segura, Simon Jackman, and Vincent Hutchings. 2011. Methodology report and user's guide for the ANES 2010-2012 Evaluations of Government and Society Study. Palo Alto, CA, and Ann Arbor: Stanford University and University of Michigan.
34. Diamond, Alexis, and Jasjeet S. Sekhon. 2012. Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. Review of Economics and Statistics. http://sekhon. berkeley.edu/papers/GenMatch.pdf.
35. Dillman, Don A. 2000. Mail and Internet surveys: The tailored design method. 2nd ed. New York: Wiley.
36. Dillman, Don A., R. Sangster, John Tarnai, and T. Rockwood. 1996. In Current issues in survey research: New directions for evaluation, #70. Chapter 4, eds. Mark T. Braverman, and Jana Kay Slater, 45-62. San Francisco, CA: Jossey-Bass.
37. Dillman, Don A., and Leah Malani Christian. 2005. Survey mode as a source of instability in responses across surveys. Field Methods 17(1):30-52.
38. Dillman, Don A., Jolene Smyth, and Leah Melani Christian. 2009. Internet, mail, and mixed-mode surveys: The tailored design method. New York: Wiley.
39. Fowler, Floyd Jackson., Jr, Anthony M. Roman, and Zhu Xiao Di. 1998. Mode effects in a survey of Medicare prostate surgery patients. Public Opinion Quarterly 62(1):29-46.
40. Fricker, S., M. Galesic, R. Tourangeau, and T. Yan. 2005. An experimental comparison of web and telephone surveys. Public Opinion Quarterly 69(3):370-92.
41. Gingerich, D. W. 2010. Understanding off-the-books politics: Conducting inference on the determinants of sensitive behavior with randomized response surveys. Political Analysis 18:349-80.
42. Groves, Robert M. 1989. Survey errors and survey costs. New York: John Wiley and Sons.
43. Groves, R. M., and R. L. Kahn. 1979. Surveys by telephone: A national comparison with personal interviews. New York: Academic Press.
44. Groves, R. M., and E. Peytcheva. 2008. The impact of nonresponse rates on nonresponse bias. Public Opinion Quarterly 72:1-23.
45. Groves, Robert M., Stanley Presser, and Sarah Dipko. 2004. The role of topic interest in survey participation decisions. Public Opinion Quarterly 68(1):2-31.
46. Groves, Robert, Eleanor Singer, and Amy Corning. 2000. Leverage-salience theory of survey participation. Public Opinion Quarterly 64:299-308.
47. Ho, Daniel, Kosuke Imai, Gary King, and Elizabeth Stuart. 2007a. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15(3):199-236.
48. Ho, Daniel. 2007b. Matchit: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software. http://gking.harvard.edu/matchit/.
49. Holbrook, A. L., M. C. Green, and J. A. Krosnick. 2003. Telephone versus face-to-face interviewing of national probability samples with long questionnaires. Public Opinion Quarterly 67:79-125.
50. Holbrook, Allyson L., and Jon A. Krosnick. 2010. Social desirability bias in voter turnout reports: Tests using the item count technique. Public Opinion Quarterly 74(1):37-67.
51. Hox, J. J., and E. D. de Leeuw. 1994. A comparison of nonresponse in mail, telephone, and face-to-face surveys: Applying multilevel models to meta-analysis. Quality and Quantity 28:329-44.
52. Iannacchione, Vincent. 2011. The changing role of address-based sampling in survey research. Public Opinion Quarterly 75(3):556-75.
53. Keeter, Scott, Carolyn Miller, Andrew Kohut, Robert M. Groves, and Stanley Presser. 2000. Consequences of reducing non-response in a national telephone survey. Public Opinion Quarterly 64:125-48.
54. Kreuter, F., S. Presser, and R. Tourangeau. 2008. Social desirability bias in CATI, IVR, and web surveys: The effects of mode and question sensitivity. Public Opinion Quarterly 72:847-65.
55. Krosnick, J. A. 1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5:213-36.
56. Krosnick, J. A. 1999. Maximizing questionnaire quality. In Measures of political attitudes, eds. J. P. Robinson, P. R. Shaver, and L. S. Wrightsman. New York: Academic Press.
57. London, K., and L. Williams. 1990. A comparison of abortion underreporting in an in-person interview and self-administered questionnaire. Toronto: Paper presented at the Annual Meeting of the Population Association of America.
58. Lupia, Arthur, Jon Krosnick, Pat Luevano, Matthew DeBell, and Darrell Donakowski. 2009. Users Guide to the ANES 2008 Time Series Study. Ann Arbor, MI, and Palo Alto, CA: University of Michigan and Stanford University. http:// www.electionstudies.org/studypages/2008prepost/2008prepost_UsersGuide.pdf.
59. Martinez, Michael D. 2003. Comment on voter turnout and the National Election Studies. Political Analysis 11(2):187-92.
60. McDonald, Michael P. 2003. On the overreport bias of the National Election Study turnout rate. Political Analysis 11(2):180-86.
61. McDonald, Michael P. 2007. The true electorate: A cross validation of voter registration files and election survey demographics. Public Opinion Quarterly 71(4):588-602.
62. Mebane, Walter R. Jr, and Jasjeet S. Sekhon. 1998. GENetic Optimization Using Derivatives (GENOUD). Software package. http://sekhon/berkeley.edu/rgenoud/.
63. Mensch, Barbara S., and Denise B. Kandel. 1988. Underreporting of substance use in a National Longitudinal Youth Cohort. Public Opinion Quarterly 52(Spring):100-24.
64. Merkle, Daniel M., and Murray Edelman. 2002. Non-response in exit polls: A comprehensive analysis. In Survey nonresponse, eds. Robert M. Groves, Don A. Dillman, John L. Eltinge, and Roderick J. A. Little, 243-58. New York: John Wiley & Sons Inc.
65. Mulligan, Kenneth, Jon A. Krosnick, Wendy Smith, Melanie Green, and George Bizer. 2001. Nondifferentiation on attitude rating scales: A test of survey satisficing theory. Unpublished manuscript.
66. O'Muircheartaigh, Colm, Ned English, Michael Latterner, Stephanie Eckman, and Katie Dekker. 2009. Modeling the need for traditional vs. commercially available address listings for in-person surveys: Results from a National Validation of Addresses JSM. Proceedings of the American Statistical Association, Section on Survey Research Methods 6193-202.
67. Peterson, Robert A., and Roger A. Kerin. 1981. The quality of self-report data: Review and synthesis. In Review of marketing, eds. Ben M. Enis and Kenneth J. Roering, 5-20. Chicago: American Marketing Association.
68. Pew Research Center for the People and the Press. 2004. Polls face growing resistance, but still representative: Survey experiment shows. http://people-press.org/reportsdisplay.php3?ReportID=211.
69. Presser, Stanley, Michael W. Traugott, and Santa Traugott. 1990. Vote "over" reporting in surveys: The records or the respondents. Technical Report No. 39. Ann Arbor, MI: National Election Studies.
70. Rivers, Douglas. N.d. Sample matching: Representative sampling from Internet polling. Polimetrix.
71. Rosen, R., and T. Gomes. 2004. Converting CES reporters from TDE to web data collection. Toronto, Canada: Paper presented at the Joint Statistical Meetings.
72. Schuman, Howard, and Stanley Presser. 1981. Questions and answers in attitude surveys: Experiments on question form, wording and context. New York: Academic Press.
73. Schuman, Howard. 1992. Context effects: State of the past/state of the art. In Context effects in social and psychological research, eds. N. Schwarz and S. Sudman, 5-20. New York: Springer.
74. Schwarz, N. 1996. Cognition and communication judgmental biases, research methods, and the logic of conversation. Mahwah, NJ: Lawrence Erlbaum.
75. Schwarz, N., and E. Strack. 1985. Cognitive and affective processes in judgments of subjective well-being: A preliminary model. In Economic psychology, eds. H. Brandstatter and E. Kirehler, 439-47. Linz: Austria: R. Trauner.
76. Sekhon, Jasjeet Singh, and Walter R. Mebane Jr. 1998. Genetic optimization using derivatives: Theory and application to nonlinear models. Political Analysis 7:189-203.
77. Sekhon, Jasjeet S. 2011. Multivariate and propensity score matching software with automated balance optimization: The matching package for R. Journal of Statistical Software 42(7):1-52.
78. Shettle, Carolyn, and Geraldine Mooney. 1999. Monetary incentives in government surveys. Journal of Official Statistics 15:231-50.
79. Smyth, Jolene D., Don Dillman, Leah Malani Christian, and Michael J. Stern. 2006. Effects of using visual design principles to group response options in web surveys. International Journal of Internet Science 1:6-16.
80. Stoop, I. A. L. 2005. The hunt for the last respondent. The Hague: Social and Cultural Planning Office (SCP).
81. Seymour, Sudman, Norman M. Bradburn, and Norbert Schwarz. 1996. Thinking about answers. San Francisco, CA: Josey-Bass.
82. Tourangeau, Roger, Mick Couper, and Frederick Conrad. 2004. Spacing, position and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly 68(3):368-93.
83. Tourangeau, R., and K. A. Rasinski. 1988. Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin 103:299-314.
84. Tourangeau, Roger, and Ting Yan. 2007. Sensitive questions in surveys. Psychological Bulletin 133:859-83.
85. Traugott, Santa. 1989. Validating self-reported vote: 1964-1988. ANES Technical Report Series, No. nes010152. Ann Arbor, MI: National Election Studies.
86. U.S Census Bureau. American community survey 2006-2008 three-year estimates. Data profile: New Mexico: Bernalillo County: Hispanics (All).
87. Visser, Penny S., Jon A. Krosnick, Jesse Marquette, and Michael Curtin. 1996. Mail surveys for election forecasting? An evaluation of the Columbus Dispatch Poll. Public Opinion Quarterly 60(2):181-227.
88. Zickhur, K., and A. Smith. 2012. Digital differences. Washington, DC: Pew Internet and American Life Project. http:// pewinternet.org/Reports/2012/Digital-differences.aspx (accessed August 7, 2012).