The impact of question and respondent characteristics on comprehension and mapping difficulties

Public Opinion Quarterly

Volumn 70, Issue 4, 2006, Pages 565-595

  Holbrook, Allyson ^a   Cho, Young Ik ^a   Johnson, Timothy ^a  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33845292555     PISSN: 0033362X     EISSN: 15375331     Source Type: Journal    
DOI: 10.1093/poq/nfl027     Document Type: Article

References (73)

1. Andrews, Frank M. 1984. "Construct Validity and Error Components of Survey Measures: A Structural Modeling Approach." Public Opinion Quarterly 48:409-42.
2. Arbuckle, Tannis Y., and Delores P. Gold. 1993. "Aging, Inhibition, and Verbosity." Journal of Gerontology: Psychological Sciences 48:225-32.
3. Baddeley, Alan. 1986. Working Memory. Oxford, UK: Clarendon Press.
4. Bates, Nancy, and Cynthia Good. 1996. "An Evaluation of the 1995 Test Census Integrated Coverage Measurement (ICM) Interview: Results from Behavior Coding." Paper presented at the annual meeting of the American Statistical Association, Chicago.
5. Belli, Robert F., Eun Ha Lee, Frank P. Stafford, and Chia-Hung Chou. 2004. "Calendar and Question-List Survey Methods: Association between Interviewer Behaviors and Data Quality." Journal of Official Statistics 20(2):185-218.
6. Belli, Robert F., Paul S. Weiss, and James M. Lepkowski. 1999. "Dynamics of Survey Interviewing and the Quality of Survey Reports: Age Comparisons." In Cognition, Aging, and Self-Reports, ed. Norbert Schwarz, Denise Park, Bärbel Knäuper, and Seymour Sudman, pp. 303-26. Philadelphia: Psychology Press.
7. Berry, John W. 1969. "On Cross-Cultural Comparability." International Journal of Psychology 4:207-29.
8. Bishop, George, and Andrew Smith. 2001. "Response-Order Effects and the Early Gallup Split-Ballots." Public Opinion Quarterly 65:479-505.
9. Bradburn, Norman M., and Carrie Miles. 1979. "Vague Quantifiers." Public Opinion Quarterly 43:92-101.
10. Byrne, Barbara M. 1998. Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS. Mahwah, NJ: Lawrence Erlbaum Associates.
11. Cahalan, Margaret, Susan Mitchell, Lucinda Gray, Selma Chen, and John Tsapogas. 1997. "Recorded Interview Behavior Coding Study: National Survey of Recent College Graduates." Proceedings of the American Statistical Association Section on Survey Research Methods, pp. 846-51. Alexandria, VA: American Statistical Association.
12. Cannell, Charles F., Steven Blixt, and Lois Oksenberg. 1992. Characteristics of Questions That Cause Problems for Interviewers or Respondents. Ann Arbor, MI: Survey Research Center.
13. Cannell, Charles F., Floyd D. Fowler, and Kent H. Marquis. 1968. "The Influence of Interviewer and Respondent Psychological and Behavioral Variables on the Reporting in Household Interviews." Vital and Health Statistics, Series 2: Data Evaluation and Methods Research, no. 26. Washington, DC: U.S. Department of Health, Education, and Welfare.
14. Cannell, Charles F., Kent H. Marquis, and André Laurent. 1977. "A Summary of Studies of Interviewing Methodology." Vital and Health Statistics, Series 2: Data Evaluation and Methods Research, no. 69. Washington, DC: U.S. Department of Health, Education, and Welfare.
15. Carpenter, Patricia A., and Marcel A. Just. 1983. "What Your Eyes Do While Your Mind Is Reading." In Eye Movements in Reading: Perceptual and Language Processes, ed. Keith Rayner, pp. 275-307. San Diego, CA: Academic Press.
16. Ceci, Stephen J. 1991. "How Much Does Schooling Influence General Intelligence and Its Cognitive Components? A Reassessment of the Evidence." Developmental Psychology 27:703-22.
17. Craik, Ferbus I. M., and M. Byrd. 1982. "Aging and Cognitive Deficits: The Role of Attentional Resources." In Aging and Cognitive Processes, ed. Ferbus I. M. Craik and Sandra Trehub, pp. 191-211. New York: Plenum.
18. Dykema, Jennifer, James M. Lepkowski, and Steven Blixt. 1997. "The Effect of Interviewer and Respondent Behavior on Data Quality: Analysis of Interaction Coding in a Validation Study." In Survey Measurement and Process Quality, ed. Lars Lyberg, Paul Biemer, Martin Collins, Edith de Leeuw, Cathryn Dippo, Norbert Schwarz, and Dennis Trewin, pp. 287-310. New York: Wiley.
19. Esposito, James L. 2004. "Iterative, Multiple-Method Questionnaire Evaluation Research: A Case Study." Journal of Official Statistics 20(2):143-83.
20. Flesch, Rudolf. 1948. "A New Readability Yardstick." Journal of Applied Psychology 32:221-33.
21. _. 1962. The Art of Readable Writing. New York: Macmillan.
22. _. 1979. How to Write Plain English. New York: Harper and Row.
23. Fowler, Floyd Jackson, Jr. 1992. "How Unclear Terms Affect Survey Data." Public Opinion Quarterly 56(2):218-31.
24. Fowler, Floyd Jackson, Jr., and Charles F. Cannell. 1996. "Using Behavioral Coding to Identify Cognitive Problems with Survey Questions." In Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, ed. Norbert Schwarz and Seymour Sudman, pp. 15-36. San Francisco: Jossey-Bass.
25. Groves, Robert M. 1996. "How Do We Know What We Think They Think Is Really What They Think?" In Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, ed. Norbert Schwarz and Seymour Sudman, pp. 389-402. San Francisco: Jossey-Bass.
26. Harkness, Janet A., Fons J. R. van de Vijver, and Peter Ph. Mohler. 2003. Cross-Cultural Survey Methods. Hoboken, NJ: Wiley.
27. Hedeker, Donald, and Robert D. Gibbons. 1994. "A Random-Effects Ordinal Regression Model for Multilevel Analysis." Biometrics 50(4):933-44.
28. Hess, Jennifer, Eleanor Singer, and John Bushery. 1999. "Predicting Test-Retest Reliability from Behavior Coding." International Journal of Public Opinion 11(4):346-60.
29. Hofstede, Geert. 2001. Culture's Consequences. 2d ed. Thousand Oaks, CA: Sage.
30. Holbrook, Allyson L., Jon A. Krosnick, Richard T. Carson, and Robert Cameron Mitchell. 2000. "Violating Conversational Conventions Disrupts Cognitive Processing of Attitude Questions." Journal of Experimental Social Psychology 36:465-94.
31. Ji, Li-Jun, Norbert Schwarz, and Richard E. Nisbett. 2000. "Culture, Autobiographical Memory, and Behavioral Frequency Reports: Measurement Issues in Cross-Cultural Studies." Personality and Social Psychology Bulletin 26 (May): 586-94.
32. Johnson, Timothy P., Diane O'Rourke, Noel Chávez, Seymour Sudman, Richard Warnecke, Loretta Lacey, and John Horm. 1996. "Cultural Variations in the Interpretation of Health Survey Questions." In Health Survey Research Methods Conference Proceedings, ed. Richard B. Warnecke, pp. 57-62. DHHS Publication no. (PHS) 96-1013. Hyattsville, MD: National Center for Health Statistics.
33. _. 1997. "Social Cognition and Responses to Survey Questions among Culturally Diverse Populations." In Survey Measurement and Process Quality, ed. Lars Lyberg, Paul Biemer, Martin Collins, Edith de Leeuw, Cathryn Dippo, Norbert Schwarz, and Dennis Trewin, pp. 87-113. New York: Wiley.
34. Jöreskog, Karl G., and Dag Sörbom. 1993a. PRELIS 2 User's Reference Guide. Chicago: Scientific Software International.
35. _. 1993b. LISREL 8 User's Reference Guide. Chicago: Scientific Software International.
36. Just, Marcel A., and Patricia A. Carpenter. 1980. "A Theory of Reading: From Eye Fixations to Comprehension." Psychological Review 87:329-54.
37. Kemper, Susan, and Karen Kemtes. 1999. "Aging and Message Production and Comprehension." In Cognition, Aging, and Self-Reports, ed. Norbert Schwarz, Denise Park, Bärbel Knäuper, and Seymour Sudman, pp. 229-44. Philadelphia: Psychology Press.
38. Klare, George R. 1984. "Readability." In Handbook of Reading Research, ed. P. David Pearson, Rebecca Barr, Michael L. Kamil, and Peter B. Mosenthal, pp. 681-744. New York: Longman.
39. Knäuper, Bärbel. 1999. "Age and Response Order Effects." Public Opinion Quarterly 63:347-70.
40. Krosnick, Jon A. 1991. "Response strategies for coping with the Cognitive Demands of Attitude Measurement in Surveys." Applied Cognitive Psychology 5:213-36.
41. Krosnick, Jon A., and Duane F. Alwin. 1987. "An Evaluation of a Cognitive Theory of Response Order Effects in Survey Measurement." Public Opinion Quarterly 51:201-19.
42. Krosnick, Jon A., and Leandre R. Fabrigar. In press. Designing Great Questionnaires: Insights from Psychology. New York: Oxford University Press.
43. Krosnick, Jon A., Sowmya Narayan, and Wendy R. Smith. 1996. "Satisficing in Surveys: Initial Evidence." New Directions for Evaluation 70:29-44.
44. Laurent, Andre. 1972. "Effects of Question Length on Reporting Behavior in the Survey Interview." Journal of the American Statistical Association 67:298-305.
45. Lepkowski, James M., Vivian Siu, and Justin Fisher. 2000. "Event History Analysis of Interviewer and Respondent Survey Behavior." In Developments in Survey Methodology, ed. Anuška Ferligoj and Andrej Mrvar, pp. 3-20. Metodološki zvezki 18. Ljubljana: FDV.
46. Mangione, Thomas W., Floyd J. Fowler, and Thomas A. Louis. 1992. "Question Characteristics and Interviewer Effects." Journal of Official Statistics 8:293-307.
47. McClendon, McKee J. 1986. "Response-Order Effects for Dichotomous Questions." Social Science Quarterly 67:205-11.
48. _. 1991. "Acquiescence and Recency Response-Order Effects in Interview Surveys." Sociological Methods and Research 20:60-103.
49. McGaw, Sue, and Peter Sturmey. 1989. "The Effects of Readability and Summary Exercises on Parental Knowledge of Behaviour Therapy: The Portage Parent Readings." Educational Psychology 9:128-32.
50. Morton-Williams, Jean, and Wendy Sykes. 1984. "The Use of Interaction Coding and Follow-Up Interviews to Investigate Comprehension of Survey Questions." Journal of Market Research Society 2:109-27.
51. Narayan, Sowmya, and Jon A. Krosnick. 1996. "Education Moderates Some Response Effects in Attitude Measurement." Public Opinion Quarterly 60:58-88.
52. Oksenberg, Lois, Charles Cannell, and Graham Kalton. 1991. "New Strategies for Pretesting Survey Questions." Journal of Official Statistics 7(3):439-65.
53. Olson, Arthur V. 1986. "A Question of Reading Validity." Journal of Research and Development in Education 19:33-40.
54. Park, Denise. C. 1999. "Cognitive Aging, Processing Resources, and Self-Report." In Cognition, Aging, and Self-Reports, ed. Norbert Schwarz, Denise Park, Bärbel Knäuper, and Seymour Sudman, pp. 285-302. Philadelphia: Psychology Press.
55. Payne, Stanley L. 1949. "Case Study in Question Complexity." Public Opinion Quarterly 13:653-58.
56. Petty, Richard E., Greg A. Rennier, and John T. Cacioppo. 1987. "Assertion versus Interrogation Format in Opinion Surveys: Questions Enhance Thoughtful Responding." Public Opinion Quarterly 51:481-94.
57. Pichora-Fuller, M. Kathleen, Bruce A. Schneider, and Meredyth Daneman. 1995. "How Young and Old Adults Listen to and Remember Speech in Noise." Journal of the Acoustical Society of America 97:593-608.
58. Presser, Stanley, and Johnny Blair. 1994. "Survey Pretesting: Do Different Methods Produce Different Results?" Sociological Methodology 24:73-104.
59. Raudenbush, Stephen W., Anthony S. Bryk, Uuk Fai Cheong, and Richard T. Congdon, Jr. 2004. HLM 6: Hierarchical Linear and Nonlinear Modeling. Lincolnwood, IL: Scientific Software International.
60. Salthouse, Timothy A. 1996. "The Processing-Speed Theory of Adult Age Differences in Cognition." Psychological Review 103:403-28.
61. Schwarz, Norbert, Denise Park, Bärbel Knäuper, and Seymour Sudman, eds. 1999. Cognition, Aging, and Self-Reports. Philadelphia: Psychology Press.
62. Stine, Elizabeth A. L., and Arthur Wingfield. 1988. "Memorability Functions as an Indicator of Qualitative Age Differences in Text Recall." Psychology and Aging 3:179-83.
63. Stine, Elizabeth A. L., Arthur Wingfield, and Leonard W. Poon. 1986. "How Much and How Fast: Rapid Processing of Spoken Language in Later Adulthood." Psychology and Aging 1:303-11.
64. Sykes, Wendy, and Martin Collins. 1992. "Anatomy of the Survey Interview." Journal of Official Statistics 8(3):277-91.
65. Sykes, Wendy and Jean Morton-Williams. 1987. "Evaluating Survey Questions." Journal of Official Statistics 3(2):191-207.
66. Tourangeau, Roger, and Kenneth A. Rasinski. 1988. "Cognitive Processes Underlying Context Effects in Measurement." Psychological Bulletin 103:299-314.
67. Tourangeau, Roger, Lance J. Rips, and Kenneth Rasinski. 2000. The Psychology of Survey Response. Cambridge, UK: Cambridge University Press.
68. Triandis, Harry C. 1972. The Analysis of Subjective Culture. New York: Wiley-Interscience.
69. van der Zouwen, Johannes. 2000. "An Assessment of the Difficulty of Questions Used in the ISSP-Questionnaires, the Clarity of Their Wording, and the Comparability of the Responses." ZA-Information 46:96-114.
70. _. 2002. "Why Study Interaction in a Survey Interview? Response from a Survey Researcher." In Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, ed. Douglas W. Maynard, Hanneke Houtkoop-Steenstra, Nora Cate Schaeffer, and Johannes van der Zouwen, pp. 47-66. New York: Wiley.
71. van der Zouwen, Johannes, and Wil Dijkstra. 2002. "Testing Questionnaires Using Interaction Coding." In Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, ed. Douglas W. Maynard, Hanneke Houtkoop-Steenstra, Nora Cate Schaeffer, and Johannes van der Zouwen, pp. 427-48. New York: Wiley.
72. van der Zouwen, Johannes, and Johannes Smit. 2004. "Evaluating Survey Questions by Analyzing Patterns of Behavior Codes and Question-Answer Sequences." In Methods for Testing and Evaluating Survey Questionnaires, ed. Stanley Presser, Jennifer M. Rothgeb, Mick P. Couper, Judith L. Lessler, Elizabeth Martin, Jean Martin, and Eleanor Singer, pp. 109-30. New York: Wiley.
73. Wingfield, Arthur, A. H. Alexander, and S. Cavigelli. 1994. "Does Memory Constrain Utilization of Top-Down Information in Spoken Word Recognition? Evidence from Normal Aging." Language and Speech 37:221-35.