Response 1 to Fowler's Chapter: Coding the Behavior of Interviewers and Respondents to Evaluate Survey Questions

Question Evaluation Methods: Contributing to the Science of Data Quality

Volumn , Issue , 2011, Pages 23-39

  Schaeffer, Nora Cate ^a   Dykema, Jennifer ^a  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

Coding the interaction, interviewers and respondents understanding interactional expressions, of cognitive processing and measurement error; Observing interviewer's behavior how standardized the respondent's behavior is, clues to their cognitive processing; Response 1 to Fowler's chapter, coding behavior of interviewers and respondents studying interaction in survey interview and why it is done

Indexed keywords

EID: 84055213949     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118037003.ch3     Document Type: Chapter

References (50)

1. Bassili JN, Scott BS (1996). Response latency as a signal to question problems. Survey research. Public Opinion Quarterly; 60:390-399.
2. Belli, RF, Lepkowski, JM (1996). Behavior of Survey Actors and the Accuracy of Response. In: edited by R. Warneke. Hyattsville, MD, Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics, pp. 69-74.
3. Belli RF, Lepkowski JM, Kabeto MU (2001). The respective roles of cognitive processing difficulty and conversational rapport on the accuracy of retrospective reports of doctors' office visits. In: Seventh Health Survey Research Methods Conference Proceedings (University of Illinois-Chicago, Chicago), pp. 197-203.
4. Belli RF, Lee EH, Stafford FP, & Chou C-H (2004). Calendar and question-list survey methods: Association between interviewer behaviors and data quality. Journal of Official Statistics; 20(2):143-184.
5. Brenner M (1982). Response effects of "role-restricted" characteristics of the interviewer. In: Dijkstra W, Van der Zouwen J, editors. Response Behavior in the Survey Interview. London: Academic; pp. 131-165.
6. Brill, E. (1995). Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics; 21: 543-566.
7. Collins M (1980). Interviewer variability: a review of the problem. Journal of the Market Research Society; 22 (2): 77-95.
8. Dijkstra W (1987). Interviewing style and respondent behavior: An experimental study of the survey interview. Sociological Methods and Research; 16(2): 309-334.
9. DijkstraW, Ongena Y (2006). Question-answer sequences in survey-interviews. Quality & Quantity; 40(6):983-1011.
10. Draisma S, Dijkstra W (2004). Response latency and (para) linguistic expression as indicators of response error. In: Presser S, Rothgeb JM, Couper MP, Lessler JT, Martin E, Martin J, Singer E, editors. Methods for Testing and Evaluating Survey Questionnaires. New York: Wiley; pp. 131-148.
11. Drew P (1984). Speakers' reportings in invitation sequences. In: Atkinson JM, Heritage J, editors. Structures of Social Action: Studies in Conversation Analysis. Cambridge: Cambridge University Press; pp. 129-151.
12. Dykema J, Schaeffer NC (2000). Events, instruments, and reporting errors. American Sociological Review; 65 (4): 619-629.
13. Dykema J, Schaeffer NC (2005). An investigation of the impact of departures from standardized interviewing on response errors in self-reports about child support and other family-related variables. Paper presented at the annual meeting of the American Association for Public Opinion Research, May, Miami Beach, FL.
14. Dykema J, Lepkowski JM, Blixt S (1997). The effect of interviewer and respondent behavior on data quality: analysis of interaction coding in a validation study. In: Lyberg L, Biemer P, Collins M, De Leeuw E, Dippo C, Schwarz N, Trewin D, editors. Survey Measurement and Process Quality. New York: Wiley-Interscience; pp. 287-310.
15. Ehlen P, Schober MF, Conrad FG (2007). Modeling speech disfluency to predict conceptual misalignment in speech survey interfaces. Discourse Processes; 44 (3): 245-265.
16. Forsyth B, Rothgeb JM, Willis GB (2004). Does pretesting make a difference? An experimental test. In: Presser S, Rothgeb JM, Couper MP, Lessler JT, Martin E, Martin J, Singer E, editors. Methods forTesting and Evaluating Survey Questionnaires. New York: Wiley; pp. 525-546.
17. Fowler FJ, Cosenza C (2008). Writing effective questions. In: De Leeuw ED, Hox JJ, Dillman DA, editors. International Handbook of Survey Methodology. pp. 136-160.
18. Garbarski D, Schaeffer NC, Dykema J (2011, forthcoming). Are interactional behaviors exhibited when the self-reported health question is asked associated with health status? Social Science Research.
19. Graesser, AC, Cai, Louwerse, M, Daniel, F (2006). Question understanding AID (QUAID): A web facility that tests question comprehensibility. Public Opinion Quarterly; 0 (1): 3-22.
20. Groves RM, Magilavy LJ (1986). Measuring and explaining interviewer effects in centralized telephone surveys. Public Opinion Quarterly; 50 (2): 251-266.
21. Hess J, Singer E, Bushery JM (1999). Predicting test-retest reliability from behavior coding. International Journal of Public Opinion Research; 11(4):346-360.
22. Holbrook A, Cho YI, & Johnson T (2006). The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opinion Quarterly; 70 (4): 565-595.
23. Holbrook AL, Krosnick JA, Carson RT, Mitchell RC (2000). Violating conversational conventions disrupts cognitive processing of attitude questions. Journal of Experimental Social Psychology; 36:465-494.
24. Knauper B, Belli RF, Hill DH, Herzog AR (1997). Question difficulty and respondents' cognitive ability: the effect on data quality. Journal of Official Statistics; 13(2): 181-199.
25. Mangione TW, Fowler FJ, Louis TA (1992). Question characteristics and interviewer effects. Journal of Official Statistics; 8(3):293-307.
26. Mathiowetz NA (1999). Expressions of respondent uncertainty as indicators of data quality. International Journal of Public Opinion Research; 11 (3): 289-296.
27. Oksenberg L, Cannell CF, Kalton G (1991). New strategies for pretesting survey questions. Journal of Official Statistics; 7(3):349-365.
28. O'Muircheartaigh C, Campanelli P (1998). The relative impact of interviewer effects and sample design effects on survey precision. Journal of the Royal Statistical Society, Series A; 161:63-77.
29. Ongena, YP, Dijkstra, W (2009). Preventing mismatch answers in standardized survey interviews. Quality & Quantity; 44(4):641-659.
30. Saris WE, Gallhofer IN (2007). Design, Evaluation, and Analysis of Questionnaires for Survey Research. New York:Wiley.
31. Schaeffer NC (1991). Conversation with a purpose-or conversation? Interaction in the standardized interview. In: Biemer PP, Groves RM, Lyberg LE, Mathiowetz NA, Sudman S, editors. Measurement Errors in Surveys. New York:John Wiley& Sons; pp. 367-392.
32. Schaeffer NC (1994). Errors of experience: response errors in reports about child support and their implications for questionnaire design. In: Schwarz N, Sudman S, editors. Autobiographical Memory and the Validity of Retrospective Reports. New York: Springer Verlag; pp. 142-160.
33. Schaeffer NC, Dykema J (2004). A multiple-method approach to improving the clarity of closely related concepts: distinguishing legal and physical custody of children. In: Presser S, Rothgeb JM, Couper MP, Lessler JT, Martin E, Martin J, Singer E, editors. Methods for Testing and Evaluating Survey Questionnaires. New York: Wiley; pp.475-502.
34. Schaeffer NC, Dykema J (2010). Characteristics of survey questions: a review. American Association for Public Opinion Research, May, Chicago, IL.
35. Schaeffer NC, Maynard DW (1996). From paradigm to prototype and back again: interactive aspects of cognitive processing in survey interviews. In: Schwarz N, Sudman S, editors. Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research. San Francisco: Jossey-Bass; pp. 65-88.
36. Schaeffer NC, Maynard DW (2002). Occasions for intervention: interactional resources for comprehension in standardized survey interviews. In: Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, Van der Zouwen J, editors. Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York: Wiley; pp. 261-280.
37. Schaeffer NC, Maynard DW (2008). The contemporary standardized survey interview for social research. In: Conrad FG, Schober MF, editors. Envisioning the Survey Interview of the Future. Hoboken, NJ: Wiley; pp. 31-57.
38. Schaeffer NC, Presser S (2003). The science of asking questions. Annual Review of Sociology; 29:65-88.
39. Schaeffer NC, Dykema J, Garbarski D, Maynard DW (2008). Verbal and paralinguistic behaviors in cognitive assessments in a survey interview. In: American Statistical Association 2008 Proceedings of the Section on Survey Research Methods. Washington, DC: American Statistical Association; pp. 4344-4351.
40. Schaeffer NC, Dykema J, Maynard DW (2010). Interviewers and interviewing. In: Marsden PV, Wright JD, editors. Handbook of Survey Research, 2nd ed. Biggleswade, UK: Emerald Group; pp. 437-470.
41. Schnell R, Kreuter F (2005). Separating interviewer and sampling-point effects. Journal of Official Statistics; 21(3):389-410.
42. Schober MF, Bloom JE (2004). Discourse cues that respondents have misunderstood survey questions. Discourse Processes; 38(3):287-308.
43. Smit JH, Dijkstra W, Van der Zouwen J (1997). Suggestive interviewer behaviour in surveys: An experimental study. Journal of Official Statistics; 13(1):19-28.
44. Uhrig SN, Sala E (2011, forthcoming). When change matters: An analysis of survey interaction in dependent interviewing on the British Household Panel Study. Sociological Methods & Research.
45. Van der Zouwen J (2002). Why study interaction in survey interviews? In: Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, Van der Zouwen J, editors. Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York: Wiley; pp.47-66.
46. Van der Zouwen J, Dijkstra W (2002). Testing questionnaires using interaction coding. In: Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, Van der Zouwen J, editors. Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York: Wiley; pp. 427-448.
47. Van der Zouwen J, Smit JH (2004). Evaluating survey questions by analyzing patterns of behavior codes and question-answer sequences: a diagnostic approach. In: Presser S, Rothgeb JM, Couper MP, Lessler JT, Martin E, Martin J, Singer E, editors. Methods for Testing and Evaluating Survey Questionnaires. New York: Wiley; pp. 109-130.
48. Viterna, JS, Maynard, DW (2002). How Uniform Is Standardization? Variation within and across survey research centers regarding protocols for interviewing. In: Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, Van der Zouwen J, editors. Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York: Wiley; p. 365-401.
49. Willis GB (2005). Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage.
50. Yan T, Tourangeau R (2008). Fast times and easy questions: the effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology; 22 (1): 51-68.