The effect of cati questions, respondents, and interviewers on response time

Journal of Survey Statistics and Methodology

Volumn 3, Issue 3, 2015, Pages 361-396

  Olson, Kristen ^a   Smyth, Jolene D ^a  

^a UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

CATI surveys; Questionnaire design; Response latency

Indexed keywords

EID: 85013862857     PISSN: 23250984     EISSN: 23250992     Source Type: Journal    
DOI: 10.1093/jssam/smv021     Document Type: Article

References (50)

1. Bassili, J. N. (1996), "The How and Why of Response Latency Measurement in Telephone Surveys," in Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, eds. N. Schwarz, and S. Sudman, pp. 319-346, San Francisco: Jossey-Bass.
2. Bassili, J. N., and J. F. Fletcher (1991), "Response Time Measurement in Survey Research: A Method for CATI and a New Look at Nonattitudes," Public Opinion Quarterly, 55, 331-346.
3. Beretvas, S. N. (2010), "Cross-Classified and Multiple-Membership Models," in Handbook of Advanced Multilevel Analysis, eds.J.J.Hox, andJ. K.Roberts, pp.313-334, New York: Routledge.
4. Blumberg, S. J., and J. V. Luke (2013), "Wireless Substitution: Early Release Estimates from the National Health Interview Survey, January-June 2013," retrieved from http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201312.pdf.
5. Bradburn, N., S. Sudman, and Associates (1979), Improving Interview Method and Questionnaire Design, San Francisco: Jossey-Bass.
6. Bradburn, N., S. Sudman, and B. Wansink (2004), Asking Questions: The Definitive Guide to Questionnaire Design for Market Research, Political Polls, and Social and Health Questionnaires, San Francisco: Jossey Bass.
7. Centers for Disease Control and Prevention (2013), 2014 Behavioral Risk Factor Surveillance System Questionnaire, Retrieved from http://www.cdc.gov/brfss/questionnaires/pdf-ques/2014_BRFSS.pdf.
8. Christian, L. M., and D. A. Dillman (2004), "The Influence of Graphical and Symbolic Language Manipulations on Responses to Self-Administered Questions," Public Opinion Quarterly, 68 (1), 58-81.
9. Chromy, J. R., J. Eyerman, D. Odom, M. E. Madeline, and A. Hughes (2005), "Association between Interviewer Experience and Substance-Use Prevalence Rates in NSDUH," in Evaluating and Improving Methods Used in the National Survey on Drug Use and Health, DHHS Publication No. SMA 05-4044, Methodology Series M-5, eds. Joel Kennet, and Joseph Gfroerer, pp. 59-87, Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
10. Cleary, P. D., D. Mechanic, and N. Weiss (1981), "The Effect of Interviewer Characteristics on Responses to a Mental Health Interview," Journal of Health and Social Behavior, 22, 183-93.
11. Couper, M. (1998), "Measuring Survey Quality in a CASIC Environment," Proceedings of the Section on Survey Research Methods of the American Statistical Association, pp. 41-49.
12. Couper, M. P., and F. Kreuter (2013), "Using Paradata to Explore Item-Level Response Times in Surveys," Journal of the Royal Statistical Society Series A: Statistics in Society, 176(Part 1), 271-286.
13. Couper, M. P., C. Kennedy, F. G. Conrad, and R. Tourangeau (2011), "Designing Input Fields for Non-Narrative Open-Ended Responses in Web Surveys," Journal of Official Statistics, 27 (1), 65-85.
14. de Leeuw, E. (1992), Data Quality in Mail, Face to Face, and Telephone Surveys, Amsterdam: TT-Publikates.
15. de Leeuw, E. (2005), "To Mix or Not to Mix Data Collection Modes in Surveys," Journal of Official Statistics, 21, 233-255.
16. Dillman, D. A. (1991), "The Design and Administration of Mail Surveys," Annual Review of Sociology, 17, 225-249.
17. Dillman, D. A., and J. Tarnai (1991), "Mode effects of Cognitively Designed Recall Questions: A Comparison of Answers to Telephone and Mail Surveys," in Measurement Errors in Surveys, eds. Paul Biemer et al., pp. 73-93, New York: John Wiley.
18. Dillman, Don A., J. Smyth, and L. M. Christian (2014), Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method, Hoboken, NJ: John Wiley and Sons, Inc.
19. Edwards, B., S. Schneider, and P. D. Brick (2008), "Visual Elements of Questionnaire Design: Experiments with a CATI Establishment Survey," in Advances in Telephone Survey Methodology, eds. J. M. Lepkowski, C. Tucker, and J. M. Brick, et al., pp. 276-296, Hoboken, NJ: John Wiley & Sons.
20. Fowler, F.J.(1995), Improving Survey Questions: Design and Evaluation, Thousand Oaks, CA: Sage.
21. Fowler, F. J., and T. W. Mangione (1990), Standardized Survey Interviewing: Minimizing Interviewer-Related Error, Newbury Park, CA: Sage.
22. Fu, H., J. E. Darroch, S. K. Henshaw, and E. Kolb (1998), "Measuring the Extent of Abortion Underreporting in the 1995 National Survey of Family Growth," Family Planning Perspectives, 30, 128-133 & 138.
23. Jenkins, C., and D. A. Dillman (1997), "Towards a Theory of Self-Administered Questionnaire Design," in Survey Measurement and Process Quality, eds. L. E. Lyberg, P. Biemer, M. Collins, E. D. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin, pp. 165-196, New York: Wiley-Inter-science.
24. Johnson, M. (2004), "Timepieces: Components of Survey Question Response Latencies," Political Psychology, 25, 679-702.
25. Knauper, B. (1999), "Age and Response Order Effects," Public Opinion Quarterly, 63, 347-370.
26. Kreuter, F., S. Presser, and R. Tourangeau (2008), "Social Desirability Bias in CATI, IVR and Web Surveys," Public Opinion Quarterly, 72, 847-865.
27. Krosnick, J. A. (1991), "Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys," Applied Cognitive Psychology, 5, 213-236.
28. Krosnick, J. A., and S. Presser (2010), "Question and Questionnaire Design," in Handbook of Survey Research (2nd ed.), pp. 263-314, Bingley, UK: Emerald House Publishing.
29. Malhotra, N. (2008), "Completion Time and Response Order Effects in Web Surveys," Public Opinion Quarterly, 72, 914-943.
30. Maynard, D. W., and N. C. Schaeffer (2002), "Standardization and Its Discontents," in Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, eds. D. W. Maynard, H. Houtkoop-Steenstra, N. C. Schaeffer, and J. van der Zouwen, pp. 3-45, New York: John Wiley & Sons, Inc.
31. Miller, J., and J. Baker-Prewitt (2009), "Beyond 'Trapping' the Undesirable Panelist: The Use of Red Herrings to Reduce Satisficing," paper presented at the 2009 CASRO Panel Quality Conference, retrieved from http://www.survey4.burke.com/Library/Conference/Beyond%20Trapping%20the%20Undesirable%20Panelist_FINAL.pdf.
32. Mulligan, K., J. T. Grant, S. T. Mockabee, and J. Q. Monson (2003), "Response Latency Methodology for Survey Research: Measurement and Modeling Strategies," Political Analysis, 11, 289-301.
33. Narayan, S., and J. A. Krosnick (1996), "Education Moderates Some Response Effects in Attitude Measurement," Public Opinion Quarterly, 60, 58-88.
34. Olson, K., and I. Bilgen (2011), "The Role of Interviewer Experience on Acquiescence," Public Opinion Quarterly, 75, 99-114.
35. Olson, K., and B. Parkhurst (2013), "Collecting Paradata for Measurement Error Evaluations," in Improving Surveys with Paradata: Analytic Uses of Process Information, ed. F. Kreuter, pp. 43-72, Hoboken, NJ: John Wiley & Sons.
36. Olson, K., and A. Peytchev (2007), "Effect of Interviewer Experience on Interview Pace and Interviewer Attitudes," Public Opinion Quarterly, 71, 273-286.
37. Rabe-Hesketh, S., and A. Skrondal (2012), Multilevel and Longitudinal Modeling Using Stata, Third Edition, Volume I: Continuous Responses, College Station, TX: Stata Press.
38. Raudenbush, S. W., and A. S. Bryk (2002), Hierarchical Linear Models: Applications and Data Analysis Methods (2nd ed.), Newbury Park, CA: Sage.
39. Rizzo, L., J. M. Brick, and I. Park (2004), "A Minimally Intrusive Method for Sampling Persons in Random Digit Dial Surveys," Public Opinion Quarterly, 68, 267-274.
40. Saris, W. E., and I. N. Gallhofer (eds.) (2014), Survey Items in Batteries, in Design, Evaluation, and Analysis of Questionnaires for Survey Research (2nd ed.), Hoboken, NJ: John Wiley & Sons, doi:10.1002/9781118634646.
41. Schuman, H., and S. Presser (1981), Questions and Answers In Attitude Surveys: Experiments on Question form, Wording, and Context, New York: Academic Press.
42. Smyth, J. D. (2008, May 15-18). "Unresolved Issues in Multiple-Answer Questions," paper presented at the American Association for Public Opinion Research, New Orleans, LA.
43. Smyth, J. D., D. A. Dillman, L. M. Christian, and M. McBride (2009), "Open-Ended Questions In Web Surveys: Can Increasing the Size of Answer Spaces and Providing Extra Verbal Instructions Improve Response Quality?" Public Opinion Quarterly, 73, 325-337.
44. Tourangeau, R., and T. W. Smith (1996), "Asking Sensitive Questions: The Impact of Data Collection Mode, Question Format, and Question Context," Public Opinion Quarterly, 60, 275-304.
45. Tourangeau, R., and T. Yan (2007), "Sensitive Questions in Surveys," Psychological Bulletin, 133, 859-883.
46. Tourangeau, R., K. Rasinski, and R. D'Andrade (1991), "Attitude Structure and Belief Accessibility," Journal of Experimental Social Psychology, 27, 48-75.
47. Tourangeau, R., L. J. Rips, and K. Rasinski (2000), The Psychology of Survey Response, New York: Cambridge University Press.
48. West, B., and K. Olson (2010), "How Much of Interviewer Variance is Really Nonresponse Error Variance?" Public Opinion Quarterly, 74, 1004-1026.
49. Yan, T., and K. Olson (2013), "Analyzing Paradata to Investigate Measurement Error," in Improving Surveys with Paradata: Analytic Uses of Process Information, ed. F. Kreuter, pp. 73-96, Hoboken, NJ: John Wiley & Sons.
50. Yan, T., and R. Tourangeau (2008), "Fast Times and Easy Questions: The Effects of Age, Experience, and Question Complexity on Web Survey Response Times," Applied Cognitive Psychology, 22, 51-68.