The effects of item saliency and question design on measurement error in a self-administered survey

Field Methods

Volumn 24, Issue 1, 2012, Pages 3-27

  Stern, Michael J ^a   Smyth, Jolene D ^b   Mendez, Jeanette ^c  

^a Department of Teacher Education and Department of Sociology and Anthropology   (United States)

^b UNIVERSITY OF NEBRASKA LINCOLN   (United States)

^c OKLAHOMA STATE UNIVERSITY   (United States)

Author keywords

Measurement error; Saliency; Survey; Visual design

Indexed keywords

EID: 84857696239     PISSN: 1525822X     EISSN: 15523969     Source Type: Journal    
DOI: 10.1177/1525822X11419478     Document Type: Article

References (36)

1. American Association for Public Opinion Research (AAPOR). 2008. Standard definitions: Final dispositions of case codes and outcome rates for surveys. 5th ed. Lenexa, KS: AAPOR.
2. Christian, L. 2003. The influence of visual layout on scalar questions in web surveys. Master's thesis, Washington State University.
3. Christian, L., and D. Dillman. 2004. The influence of graphical and symbolic language manipulations on responses to self-administered questions. Public Opinion Quarterly 68:58-81.
4. Christian, L., D. Dillman, and J. Smyth. 2007. Helping respondents get it right the first time: The relative influence of words, symbols, and graphics in web and telephone surveys. Public Opinion Quarterly 71:113-25.
5. Converse, P. 1964. The nature of belief systems in mass publics. In Ideology and discontent, ed. D. Apter, 206-61. New York: The Free Press.
6. Dillman, D., and L. Christian. 2005. Survey mode as a source of instability in responses across surveys. Field Methods 17:30-52.
7. Dillman, D., R. Sangster, J. Tarnai, and T. Rockwood. 1996. Understanding differences in people's answers to telephone and mail surveys. New Directions for Evaluation 70:45-61.
8. Dillman, D., J. Smyth, and L. Christian. 2009. Internet, mail and mixed-mode surveys: The tailored design method. Hoboken, NJ: John Wiley.
9. Eisenhower, D., N. Mathiowetz, and D. Morganstein. 1991. Recall error: Sources and bias reduction techniques. In Measurement errors in surveys, eds. P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, and S. Sudman, 127-44. New York: Wiley.
10. Gilljam, M., and D. Granberg. 1993. Should we take don't know for an answer? Public Opinion Quarterly 57:348-57.
11. Goyder, J. 1987. The silent minority: Nonrespondents on sample surveys. Boulder, CO: Westview.
12. Groves, R., S. Presser, and S. Dipko. 2004. The role of topic interest in survey participation decisions. Public Opinion Quarterly 68:2-31.
13. Groves, R., E. Singer, and A. Corning. 2000. Leverage-salience theory of survey participation: Description and an illustration. Public Opinion Quarterly 64: 299-308.
14. Holland, J., and L. Christian. 2009. The influence of topic interest and interactive probing on responses to open-ended questions in web surveys. Social Science Computer Review 27:196-212.
15. Jenkins, C., and D. Dillman. 1997. Towards a theory of self-administered questionnaire design. In Survey measurement and process quality, eds. L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin, 165-96. New York: John Wiley.
16. Knäuper, B. 1999. The impact of age and education on response order effects in attitude measurement. Public Opinion Quarterly 63:347-70.
17. Knauper, B., R. Belli, D. Hill, and A. Herzog. 1998. Question difficulty and respondents' characteristics: Effects on data quality. Journal of Official Statistics 13:181-99.
18. Krosnick, J., and D. Alwin. 1987. An evaluation of a cognitive theory of responseorder effects in survey measurement. Public Opinion Quarterly 51:201-19.
19. Krosnick, J., J. Narayan, and W. Smith. 1996. Satisficing in surveys: Initial evidence. New Directions for Evaluation 70:29-44.
20. Lam, T., K. Green, and C. Bordingnon. 2002. Effects of item grouping and position of the "don't know" questionnaire response. Field Methods 14:418-32.
21. Marcus, B., M. Bosjnak, S. Lindner, S. Pilischenko, and A. Shutz. 2007. Compensating for low topic interest and long surveys: A field experiment on nonresponse in web surveys. Social Science Computer Review 25:372-83.
22. Petty, R., and J. Cacioppo. 1986. Communication and persuasion: Central and peripheral routes to attitude change. New York: Springer-Verlag.
23. Rapoport, R. 1982. Sex differences in attitude expression: A generational explanation. Public Opinion Quarterly 46:86-96.
24. Schuman, H., and S. Presser. 1981. Questions and answers in attitude surveys experiments on question form, wording, and context. New York: Academic Press.
25. Schwarz, N. 1996. Cognition and communication: Judgmental biases, research methods, and the logic of conversation. Mahwah, NJ: Lawrence Erlbaum.
26. Smyth, J., D. Dillman, L. Christian, and M. McBride. 2009. Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality? Public Opinion Quarterly 73:325-37.
27. Smyth, J., D. Dillman, L. Christian, and M. Stern. 2006. Comparing check-all and forced-choice question formats in web surveys. Public Opinion Quarterly 70:66-77.
28. Stern, M. 2006. How use of the Internet impacts community participation and the maintenance of core social ties: An empirical study. PhD diss., Department of Sociology, Washington State University.
29. Stern, M. 2008. The use of client-side paradata in analyzing the effects of visual layout on changing responses in web surveys. Field Methods 20:377-98.
30. Stern, M., D. Dillman, and J. Smyth. 2007. Visual design, order effects and respondent characteristics in a self-administered survey. Survey Research Methods 1:121-38.
31. Sudman, S., and N. Bradburn. 1974. Response effects in surveys. Chicago: Aldine.
32. Thomas, R., and J. Klein. 2006. Merely incidental? Effects of response format on self-reported behavior. Journal of Official Statistics 22:221-44.
33. Toepoel, V. 2008. A closer look at web questionnaire design. Center for Economic Research Dissertation Series No. 220. Tillburg University, Tillburg, The Netherlands.
34. Toepoel, V., C. Vis, M. Das, and A. van Soest. 2009. Design of web questionnaires: An information-processing perspective for the effect of response categories. Sociological Methods & Research 37:371-92.
35. Tourangeau, R. 1984. Cognitive science and survey methods. In Cognitive aspect of survey methodology: Building a bridge between disciplines, eds. T. Jabine, M. Straf, J. Tanur, and R. Tourangeau, 73-100. Washington, DC: National Academy Press.
36. Tourangeau, R., M. Couper, and F. Conrad. 2004. Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly 68:368-93.