Cognitive burden of survey questions and response times: A psycholinguistic experiment

Applied Cognitive Psychology

Volumn 24, Issue 7, 2010, Pages 1003-1020

  Lenzner, Timo ^a   Kaczmirek, Lars ^a   Lenzner, Alwine ^b  

^a GESIS LEIBNIZ INSTITUTE FOR THE SOCIAL SCIENCES   (Germany)

^b UNIVERSITY OF MANNHEIM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COGNITION; COMPREHENSION; CONTROLLED STUDY; EMPIRICAL RESEARCH; HUMAN; INFORMATION PROCESSING; LINGUISTICS; MALE; PRIORITY JOURNAL; QUALITY CONTROL; QUESTIONNAIRE; REDUNDANCY ANALYSIS; SATISFACTION; THEORETICAL STUDY; WORKING MEMORY;

EID: 78149399179     PISSN: 08884080     EISSN: 10990720     Source Type: Journal    
DOI: 10.1002/acp.1602     Document Type: Article

References (58)

1. Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.
2. Baddeley, A. D., & Hitch, G. T. (1974). Working memory. In G. A. Bower (Ed.), The psychology of learning and motivation (Vol. 8, pp. 47-89). New York: Academic Press.
3. Balota, D. A., Yap, M. J., & Cortese, M. J. (2006). Visual word recognition: The journey from features to meaning (a travel update). In M. J. Traxler, & M. A. Gernsbacher (Eds.), Handbook of Psycholinguistics (Vol. 2, pp. 285-375). Amsterdam: Elsevier.
4. Bassili, J. N. (1996). The how and the why of response latency measurement in telephone surveys. In N. Schwarz, & S. Sudman (Eds.), Answering questions (pp. 319-346). San Francisco, CA: Jossey-Bass.
5. Bassili, J. N., & Scott, B. S. (1996). Response latency as a signal to question problems in survey research. Public Opinion Quarterly, 60, 390-399.
6. Belson, W. A. (1981). The design and understanding of survey questions. Aldershot: Gower.
7. Bless, H., Bohner, G., Hild, T., & Schwarz, N. (1992). Asking difficult questions: Task complexity increases the impact of response alternatives. European Journal of Social Psychology, 22, 309-312.
8. Bradburn, N., Sudman, S., & Wansink, B. (2004). Asking questions (2nd ed.). San Francisco, CA: Jossey-Bass.
9. Chomsky, N., & Miller, G. A. (1963). Introduction to the formal analysis of natural languages. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology (Vol. 2, pp. 269-321). New York: Wiley.
10. Coleman, E. B. (1964). The comprehensibility of several grammatical transformations. Journal of Applied Psychology, 48, 186-190.
11. Colombo, L., Pasini, M., & Balota, D. A. (2006). Dissociating the influence of familiarity and meaningfulness from word frequency in naming and lexical decision performance. Memory & Cognition, 34, 1312-1324.
12. Coltheart, M. (1981). The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology, 33A, 497-505.
13. Couper, M. P., Tourangeau, R., & Conrad, F. G. (2004). What they see is what we get. Social Science Computer Review, 22, 111-127.
14. Draisma, S., & Dijkstra, W. (2004). Response latency and (para)linguistic expressions as indicators of response error. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 131-147). New York: Wiley.
15. Duffelmeyer, F. A. (1979). The effect of rewriting prose material on reading comprehension. Reading World, 19, 1-16.
16. Duffy, S. A., Morris, R. K., & Rayner, K. (1988). Lexical ambiguity and fixation times in reading. Journal of Memory and Language, 27, 429-446.
17. Ericsson, K. A., & Kintsch, W. A. (1995). Long-term working memory. Psychological Review, 102, 211-245.
18. Fazio, R. H. (1990). A practical guide to the use of response latency in social psychological research. In C. Hendrick, & M. S. Clark (Eds.), Research methods in personality and social psychology (review of personality and social psychology) (Vol. 11, pp. 74-97). Newbury Park, CA: Sage Publications.
19. Fillmore, C. J. (1999). A linguistic look at survey research. In M. G. Sirken, D. J. Herrmann, S. Schechter, N. Schwarz, J. M. Tanur, & R. Tourangeau (Eds.), Cognition and survey research (pp. 183-198). New York: Wiley.
20. Fink, A. (1995). How to ask survey questions. Thousand Oaks, CA: Sage.
21. Foddy, W. (1993). Constructing questions for interviews and questionnaires: Theory and practice in social research. Cambridge: Cambridge University Press.
22. Forster, K. I. (1970). Visual perception on rapidly presented word sequences of varying complexity. Perception and Psychophysics, 8, 197-202.
23. Forster, K. I., & Chambers, S. M. (1973). Lexical access and naming time. Journal of Verbal Learning and Verbal Behavior, 12, 627-635.
24. Forster, K. I., & Olbrei, I. (1974). Semantic heuristics and syntactic analysis. Cognition, 2, 319-347.
25. Foss, D. J. (1969). Decision processes during sentence comprehension: Effects of lexical item and position upon decision times. Journal of Verbal Learning and Verbal Behavior, 8, 457-462.
26. Fowler, F. J. (1995). Improving survey questions. Thousand Oaks: Sage.
27. Garrod, S., Freudenthal, S., & Boyle, E. (1994). The role of different types of anaphor in the on-line resolution of sentences in a discourse. Journal of Memory and Language, 33, 39-68.
28. Ganassali, S. (2008). The influence of the design of web survey questionnaires on the quality of responses. Survey Research Methods, 2, 21-32.
29. Graesser, A. C., Cai, Z., Louwerse, M. M., & Daniel, F. (2006). Question understanding aid (QUAID). A web facility that tests question comprehensibility. Public Opinion Quarterly, 70, 3-22.
30. Graesser, A. C., Hoffman, N. L., & Clark, L. F. (1980). Structural components of reading time. Journal of Verbal Learning and Verbal Behavior, 19, 135-151.
31. Globalpark. (2007). EFS survey [computer software, pc]. Hürth: Author.
32. Haviland, S. E., & Clark, H. H. (1974). What's new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512-521.
33. Horning, A. S. (1979). On defining redundancy in language: Case notes. Journal of Reading, 22, 312-322.
34. Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329-354.
35. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122-149.
36. Kaczmirek, L., & Faaß, T. (2008). Data quality of paradata: A comparison of three response time measures in a randomized online experiment. Poster session presented at the annual meeting of the General Online Research (GOR), Hamburg, Germany, March.
37. Kintsch, W., & Keenan, J. M. (1973). Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5, 257-279.
38. Kimball, J. (1973). Seven principles of surface structure parsing in natural language. Cognition, 2, 15-47.
39. Knäuper, B., Belli, R. F., Hill, D. H., & Herzog, A. R. (1997). Question difficulty and respondents' cognitive ability: The effect on data quality. Journal of Official Statisticsm, 13, 181-199.
40. Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.
41. Lessler, J. T., & Forsyth, B. H. (1996). A coding system for appraising questionnaires. In N. Schwarz, & S. Sudman (Eds.), Answering questions (pp. 259-291). San Francisco, CA: Jossey-Bass.
42. MacDonald, M. C., & Christiansen, M. H. (2002). Reassessing working memory: Comment on Just and Carpenter (1992) and Waters and Caplan (1996). Psychological Review, 109, 35-54.
43. McCutchen, D., Dibble, E., & Blount, M. M. (1994). Phonemic effects in reading comprehension and text memory. Applied Cognitive Psychology, 8, 597-611.
44. Mitchell, D. C., & Green, D. W. (1978). The effects of content on immediate processing in reading. Quarterly Journal of Experimental Psychology, 30, 29-63.
45. Morgan, J. L., & Green, G. M. (1980). Pragmatics and reading comprehension. In R. J. Spiro, B. C. Bruce, & W. F. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 113-140). Hillsdale, NJ: Erlbaum.
46. Morton, J. (1969). The interaction of information in word recognition. Psychological Review, 76, 165-178.
47. Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510-532.
48. Rayner, K., & Duffy, S. A. (1986). Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & Cognition, 14, 191-201.
49. Rayner, K., Pacht, J. M., & Duffy, S. A. (1994). Effects of prior encounter and global discourse bias on the processing of lexically ambiguous words: Evidence from eye fixations. Journal of Memory and Language, 33, 527-544.
50. Rayner, K., & Pollatsek, A. (2006). Eye-movement control in reading. In M. J. Traxler, & M. A. Gernsbacher (Eds.), Handbook of Psycholinguistics (Vol. 2, pp. 613-658). Amsterdam: Elsevier.
51. Schober, M. F., & Conrad, F. G. (1997). Does conversational interviewing reduce survey measurement error? Public Opinion Quarterly, 61, 576-602.
52. Spyridakis, J. H., & Isakson, C. S. (1998). Nominalizations vs. denominalizations: Do they influence what readers recall? Journal of Technical Writing and Communication, 28, 163-188.
53. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
54. University of Memphis. (2008). Question understanding aid. Retrieved 16 May 2008 from
55. Velez, P., & Ashworth, S. D. (2007). The impact of item readability on the endorsement of the midpoint response in surveys. Survey Research Methods, 1, 69-74.
56. Vonk, W., & Noordman, L. G. M. (1990). On the control of inferences in text understanding. In D. A. Balota, G. B. F. d'Arcais, & K. Rayner (Eds.), Comprehension processes in reading (pp. 447-464). Hillsdale, NJ: Lawrence Erlbaum.
57. Whaley, C. P. (1978). Word-nonword classification time. Journal of Verbal Learning and Verbal Behavior, 17, 143-154.
58. Yan, T., & Tourangeau, R. (2008). Fast times and easy questions: The effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology, 22, 51-68.