Using paradata to explore item level response times in surveys

Journal of the Royal Statistical Society. Series A: Statistics in Society

Volumn 176, Issue 1, 2013, Pages 271-286

  Couper, Mick P ^a,b   Kreuter, Frauke ^b,c,d  

^a UNIVERSITY OF MICHIGAN   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c INSTITUTE FOR EMPLOYMENT RESEARCH IAB   (Germany)

^d UNIVERSITY OF MUNICH   (Germany)

Author keywords

Item level times; Measurement error; Paradata

Indexed keywords

EID: 84871657426     PISSN: 09641998     EISSN: 1467985X     Source Type: Journal    
DOI: 10.1111/j.1467-985X.2012.01041.x     Document Type: Article

References (46)

1. Ajzen, I., Nichols, A. J. and Driver, B. C. (1995) Identifying salient beliefs about leisure activities: frequency of elicitation versus response latency. J. Appl. Socl Psychol., 25, 1391-1410.
2. Bassili, J. N. (1996) The how and the why of response latency measurement in telephone surveys. In Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research (eds N. Schwarz and S. Sudman ), pp. 319-346. San Francisco:Jossey-Bass.
3. Bassili, J. N. (2003) The minority slowness effect: subtle inhibitions in the expression of views not shared by others. J. Personlty Socl Psychol., 84, 261-276.
4. Bassili, J.N. and Krosnick, J. A. (2000) Do strength-related attitude properties determine susceptibility to response effects?: new evidence from response latency, attitude extremity, and aggregate indices. Polit. Psychol., 21, 107-132.
5. Bassili, J. N. and Scott, B. S. (1996) Response latency as a signal to question problems in survey research. Publ. Opin. Q., 60, 390-399.
6. Cannell, C. F., Miller, P. V. and Oksenberg, L. (1981) Research on interviewing techniques. In Sociological Methodology(ed. S. Leinhardt ), pp. 389-437. San Francisco: Jossey-Bass.
7. Caspar, R. A. and Couper, M. P. (1997) Using keystroke files to assess respondent difficulties with an ACASI instrument. Proc. Surv. Res. Meth. Sect. Am. Statist. Ass., 239-244.
8. Conrad, F., Broome, J., Benki, J., Groves, R., Kreuter, F. and Vannette, D. (2010) To agree or not to agree?: impact of interview speech and survey participation decisions. In Proc. Jt Statist. Meet. American Statistical Association, pp. 5979-5993. Alexandria: American Statistical Association.
9. Conrad, F., Couper, M., Tourangeau, R. and Galesic, M. (2005) Interactive feedback can improve the quality of responses in web surveys. Proc. Surv. Res. Meth. Sect. Am. Statist. Ass., 3835-3840.
10. Couper, M. P. (1998) Measuring survey quality in a CASIC environment. Jt Statist. Meet. American Statistical Association, Dallas, Aug.
11. Couper, M. P., Hansen, S. E. and Sadosky, S. A. (1997a) Evaluating interviewer performance in a CAPI survey. In Survey Measurement and Process Quality (eds L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz and D. Trewin ), pp. 267-285. New York: Wiley.
12. Couper, M. P., Horm, J. and Schlegel, J. (1997b) Using trace files to evaluate the National Health Interview Survey CAPI instrument. Proc. Surv. Res. Meth. Sect. Am. Statist. Ass., 825-829.
13. Couper, M. P., Tourangeau, R. and Marvin, T. (2009) Taking the audio out of audio-CASI. Publ. Opin. Q., 73, 281-303.
14. Draisma, S. and Dijkstra, W. (2004) Response latency and (para) linguistic expressions as indicators of response error. In Methods for Testing and Evaluating Survey Questionnaires (eds S. Presser, J. Rogthgeb, M. Couper, J. Lessler, E. Martin, J. Martin and E. Singer ), pp. 131-147. Hoboken: Wiley.
15. Faust, M. E., Balota, D. A., Spieler, D. H. and Ferraro, F. R. (1999) Individual differences in informationprocessing rate and amount: implications for group differences in response latency. Psychol. Bull., 126, 777-799.
16. Fazio, R. H. (1986) How do attitudes guide behavior? In The Handbook of Motivation and Cognition: Foundation of Social Behavior (eds R. M. Sorrentino and E. T. Higgins ), pp. 204-243. New York: Guilford.
17. Fazio, R. H. (1990) A practical guide to the use of response latency in social psychological research. In Review of Personality and Social Psychology, vol. 11, Research Methods in Personality and Social Researcheds C. Hendrick and M. S. Clark ), pp. 74-97. Newbury Park: Sage.
18. Fazio, R. H. and Williams, C. J. (1986) Attitude accessibility as a moderator of the attitude-perception and attitude behavior relations: an investigation of the 1984 Presidential Election. J. Personlty Socl Psychol., 51, 505-514.
19. Fisher, D. L. and Glaser, R. A. (1996) Molar and latent models of cognitive slowing: implications for aging, dementia, depression, development, and intelligence. Psychonom. Bull. Rev., 3, 458-480.
20. Groves, R. M. and Heeringa, S. G. (2006) Responsive design for household surveys: tools for actively controlling survey errors and costs. J. R. Statist. Soc. A, 169, 439-457.
21. Hansen, S. E. and Marvin, T. (2001) Reporting on item times and keystroke from Blaise audit trails. Int. Blaise User Group Meet., Washington DC. (Available from http://www.blaiseusers.org/2001/papers/Hansen_Marvin-IBUC_Paper_FINAL.pdf
22. Heerwegh, D. (2003) Explaining response latencies and changing answers using client-side paradata from a web survey. Socl Sci. Comput. Rev., 21, 360-373.
23. Huckfeldt, R., Levine, J., Morgan, W. and Sprague, J. (1998) Election campaigns, social communication, and the accessibility of perceived discussant preference. Polit. Behav., 20, 263-294.
24. Knowles, E. S. and Condon, C. A. (1999) Why people say 'yes': a dual-process theory of acquiescence. J. Personlty Socl Psychol., 77, 379-386.
25. Kreuter, F., Couper, M. P. and Lyberg, L. E. (2010) The use of paradata to monitor and manage survey data collection. In Proc. Jt Statist. Meet. American Statistical Association, pp. 282-296. Alexandria: American Statistical Association.
26. Kreuter, F. and Kohler, U. (2009) Analyzing contact sequences in call record data: potential and limitation of sequence indicators for nonresponse adjustment in the European Social Survey. J. Off. Statist., 25, 203-226.
27. LaFlamme, F. (2009) Data collection research using paradata at Statistics Canada. Modernisation of Statistics Production Conf., Stockholm, Nov.
28. Lepkowski, J. M., Mosher, W. D., Davis, K. E., Groves, R. M., Van Hoewyk, J. and Willem, J. (2006) National Survey of Family Growth, Cycle 6: sample design, weighting, imputation, and variance estimation. Vitl Hlth Statist., 2, no. 142.
29. Mayerl, J., Sellke, P. and Urban, D. (2005) Analyzing cognitive processes in CATI surveys with response latencies: an empirical evaluation of the consequences of using different baseline speed measures. In Schriftenreihe des Instituts fuer Sozialwissenschaften der Universität Stuttgart, no. 2/20005.
30. Mayerl, J. and Urban, D. (2008) Antwortreaktionszeiten in Survey-Analysen. Wiesbaden: Verlag für Sozialwissenschaften.
31. McLachlan, G. and Peel, D. (2000) Finite Mixture Models. New York: Wiley.
32. Olson, K. and Peytchev, A. (2007) Effect of interviewer experience on interview pace and interviewer attitudes. Publ. Opin. Q., 71, 273-286.
33. Salthouse, T. A. (1991) Theoretical Perspectives on Cognitive Aging. Hillsdale: Erlbaum.
34. Schaeffer, N. C. and Maynard, D. M. (2006) Sounds of silence: response latency in cognitive assessments. A. Meet. American Association for Public Opinion Research, Montreal, May 18th-21st.
35. Schwarz, N., Park, D., Knäuper, B. and Sudman, S. (1999) Cognition, Aging, and Self-reports. Washington DC: Psychology Press.
36. Smith, E. E. (1968) Choice reaction time: an analysis of major theoretical positions. Psychol. Bull., 69, 77-110.
37. Smith, E. R., Fazio, R. H. and Ceijka, M. A. (1996) Accessible attitudes influence categorization of multiply categorizable objects. J. Personlty Socl Psychol., 71, 888-898.
38. Smith, E. R., and Lerner, M. (1986) Development of automatism of social judgments. J. Personlty Socl Psychol., 50, 246-259.
39. Statistics Netherlands (2002) Blaise Developer's Guide. Heerlen: Statistics Netherlands.
40. Stern, M. J. (2008) The use of client-side paradata in analyzing the effects of visual layout on changing responses in web surveys. Fld Meth., 20, 377-398.
41. Tourangeau, R. (1992) Context effects on responses to attitude questions: attitudes as memory structures. In Context Effects in Social and Psychological Research (eds N. Schwarz and S. Sudman ), pp. 35-47. New York: Springer.
42. Tourangeau, R., Rips, L. and Rasinski, K. (2000) The Psychology of Survey Response. Cambridge: Cambridge University Press.
43. Wagner-Menghin, M. M. (2002) Towards the identification of non-scalable personality questionnaire respondents: taking response time into account. Psychol. Beitr., 44, 62-77.
44. Wells, T., Vidalon, M. and DiSogra, C. (2010) Differences in length of survey administration between Spanish-language and English-language survey respondents. Proc. Surv. Res. Meth. Sec. Am. Statist. Ass., 6186-6191.
45. West, B. T., Welch, K. B. and Galecki, A. T. (2007) Linear Mixed Models; a Practical Guide using Statistical Software. Boca Raton: Chapman and Hall.
46. Yan, T. and Tourangeau, R. (2008) Fast times and easy questions: the effects of age, experience and question complexity on web survey response times. Appl. Cogn. Psychol., 22, 51-68.