Can conversational interviewing improve survey response quality without increasing interviewer effects?

Journal of the Royal Statistical Society. Series A: Statistics in Society

Volumn 181, Issue 1, 2018, Pages 181-203

  West, Brady T ^a   Conrad, Frederick G ^a   Kreuter, Frauke ^b,c   Mittereder, Felicitas ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c INSTITUTE FOR EMPLOYMENT RESEARCH IAB   (Germany)

Author keywords

Conversational interviewing; Interviewer effects; Intra interviewer correlation; Multilevel modelling; Standardized interviewing; Survey paradata

Indexed keywords

EID: 85008258893     PISSN: 09641998     EISSN: 1467985X     Source Type: Journal    
DOI: 10.1111/rssa.12255     Document Type: Article

References (48)

1. Ackermann-Piek, D. and Massing, N. (2014) Interviewer behavior and interviewer characteristics in PIAAC Germany. Meth. Data Anal., 8, 199–222.
2. Bell, K., Fahmy, E. and Gordon, D. (2016) Quantitative conversations: the importance of developing rapport in standardised interviewing. Qual. Quant., 50, 193–212.
3. Belli, R. F. and Lepkowski, J. M. (1996) Behavior of survey actors and the accuracy of response. 5th Health Survey Research Methods Conf., Breckenridge, June 1995.
4. Brick, J. M., McGuinness, R., Lapham, S. J., Cahalan, M., Owens, D. and Gray, L. (1995) Interviewer variance in two telephone surveys. Proc. Surv. Res. Meth. Sect. Am. Statist. Ass., 447–452.
5. Bruckmeier, K., Müller, G. and Riphahn, R. T. (2015) Survey misreporting of welfare receipt—respondent, interviewer, and interview characteristics. Econ. Lett., 129, 103–107.
6. Brunton-Smith, I., Sturgis, P. and Leckie, G. (2017) Detecting and understanding interviewer effects on survey data by using a cross-classified mixed effects location–scale model. J. R. Statist. Soc. A, 180, 551–568.
7. Cannell, C. F. and Axelrod, M. (1956) The respondent reports on the interview. Am. J. Sociol., 62, 177–181.
8. Cannell, C. F., Lawson, S. A. and Hausser, D. L. (1975) A technique for evaluating interviewer performance: a manual for coding and analyzing interviewer behavior from tape recordings of household interviews. Survey Research Center, Institute for Social Research, University of Michigan, Ann Arbor.
9. Clark, H. H. and Brennan, S. A. (1991) Grounding in communication. In Perspectives on Socially Shared Cognition. (eds L. B. Resnick, J. M. Levine and S. D. Teasley). Washington: American Psychological Association Books.
10. Conrad, F. G. and Schober, M. F. (2000) Clarifying question meaning in a household telephone survey. Publ. Opin. Q., 64, 1–28.
11. Conrad, F. G., Schober, M. F., Jans, M., Orlowski, R. A., Nielsen, D. and Levenstein, R. (2015) Comprehension and engagement in survey interviews with virtual agents. Front. Psychol., 6, article 1578.
12. Dijkstra, W. (1987) Interviewing style and respondent behavior: an experimental study of the survey-interview. Sociol. Meth. Res., 16, 309–334.
13. Eckman, S., Sinibaldi, J. and Möntmann-Hertz, A. (2013) Can interviewers effectively rate the likelihood of cases to cooperate? Publ. Opin. Q., 77, 561–573.
14. Fowler, F. J. and Mangione, T. W. (1990) Standardized Survey Interviewing: Minimizing Interviewer-related Error. New York: Sage.
15. Gelman, A. and Hill, J. (2007) Data Analysis using Regression and Multilevel/Hierarchical Models. New York: Cambridge University Press.
16. Groves, R. M. (2004) The interviewer as a source of survey measurement error. In Survey Errors and Survey Costs, 2nd edn, ch. 8. New York: Wiley-Interscience.
17. Groves, R. M. and Magilavy, L. J. (1986) Measuring and explaining interviewer effects in centralized telephone surveys. Publ. Opin. Q., 50, 251–266.
18. Haan, M., Ongena, Y. and Huiskes, M. (2013) Interviewers' questions: rewording not always a bad thing. In Interviewers' Deviations in Surveys: Impact, Reasons, Detection and Prevention (eds P. Winker, N. Menold and R. Porst), pp. 173–193. Frankfurt: Peter Lang Academic Research.
19. Henson, R., Cannell, C. F. and Lawson, S. (1976) Effects of interviewer style on quality of reporting in a survey interview. J. Psychol., 93, 221–227.
20. Houtkoop-Steenstra, H. (1995) Meeting both ends: between standardization and recipient design in telephone survey interviews. In Situated Order: Studies in the Social Organization of Talk and Embodied Activities (edsP. ten Have and G. Psathas), pp. 91–107. Washington DC: University Press of America.
21. Hubbard, F., Antoun, C. and Conrad, F. G. (2012) Conversational interviewing, the comprehension of opinion questions and nonverbal sensitivity. A. Conf. American Association for Public Opinion Research, Orlando.
22. Jacobebbinghaus, P. and Seth, S. (2007) The German integrated employment biographies sample (IEBS). Zeits. Wirts. Sozlwissensch., 127, 335–342.
23. Kahn, R. L. and Cannell, C. F. (1957) The Dynamics of Interviewing; Theory, Technique, and Cases. New York: Wiley.
24. Kaminska, O., McCutcheon, A. L. and Billiet, J. (2010) Satisficing among reluctant respondents in a cross-national context. Publ. Opin. Q., 74, 956–984.
25. Kim, Y., Choi, Y.-K. and Emery, S. (2013) Logistic regression with multiple random effects: a simulation study of estimation methods and statistical packages. Am. Statistn, 67, 171–182.
26. Kreuter, F., Müller, G. and Trappmann, M. (2010) Nonresponse and measurement error in employment research: making use of administrative data. Publ. Opin. Q., 74, 880–906.
27. Loosveldt, G. and Beullens, K. (2013) The impact of respondents and interviewers on interview speed in face-to-face interviews. Socl Sci. Res., 42, 1422–1430.
28. Mangione, T. W., Fowler, F. J. and Louis, T. A. (1992) Question characteristics and interviewer effects. J. Off. Statist., 8, 293–307.
29. Maynard, D. W., Freese, J. and Schaeffer, N. C. (2010) Calling for participation: requests, blocking moves, and rational (inter)action in survey introductions. Am. Sociol. Rev., 75, 791–814.
30. Mittereder, F., Durow, J., West, B. T., Kreuter, F. and Conrad, F. G.(2017) Interviewer-respondent interactions in conversational and standardized interviewing. Fld Meth., to be published.
31. O'Muircheartaigh, C. and Campanelli, P. (1998) The relative impact of interviewer effects and sample design effects on survey precision. J. R. Statist. Soc. A, 161, 63–77.
32. Peneff, J. (1988) The observers observed: French survey researchers at work. Socl Prob., 35, 520–535.
33. Sayles, H., Belli, R. F. and Serrano, E. (2010) Interviewer variance between event history calendar and conventional questionnaire interviews. Publ. Opin. Q., 74, 140–153.
34. Schaeffer, N. C., Dykema, J. and Maynard, D. W. (2010) Interviewers and interviewing. In Handbook of Survey Research, 2nd edn (eds J. D. Wright and P. V. Marsden). Bingley: Emerald.
35. Schnell, R. and Kreuter, F. (2005) Separating interviewer and sampling-point effects. J. Off. Statist., 21, 389–410.
36. Schober, M. F. and Conrad, F. G. (1997) Does conversational interviewing reduce survey measurement error? Publ. Opin. Q., 61, 576–602.
37. Schober, M. F., Conrad, F. G., Dijkstra, W. and Ongena, Y. P. (2012) Disfluencies and gaze aversion in unreliable responses to survey questions. J. Off. Statist., 28, 555–582.
38. Schober, M. F., Conrad, F. G. and Fricker, S. S. (2004) Misunderstanding standardized language in research interviews. Appl. Cogn. Psychol., 18, 169–188.
39. Suchman, L. and Jordan, B. (1990) Interactional troubles in face-to-face survey interviews. J. Am. Statist. Ass., 85, 232–241.
40. Trappmann, M., Beste, J., Bethmann, A. and Müller, G. (2013) The PASS panel survey after six waves. J. Lab. Markt Res., 46, 275–281.
41. Trappmann, M., Krumpal, I., Kirchner, A. and Jann, B. (2014) Item sum: a new technique for asking quantitative sensitive questions. J. Surv. Statist. Methodol., 2, 58–77.
42. West, B. T. and Elliott, M. R. (2014) Frequentist and Bayesian approaches for comparing interviewer variance components in two groups of survey interviewers. Surv. Methodol., 40, 163–188.
43. West, B. T. and Kreuter, F. (2013) Factors affecting the accuracy of interviewer observations: evidence from the National Survey of Family Growth (NSFG). Publ. Opin. Q., 77, 522–548.
44. West, B. T., Kreuter, F. and Jaenichen, U. (2013) Interviewer effects in face-to-face surveys: a function of sampling, measurement error or nonresponse? J. Off. Statist., 29, 277–297.
45. West, B. T. and Olson, K. (2010) How much of interviewer variance is really nonresponse error variance? Publ. Opin. Q., 74, 1004–1026.
46. West, B. T. and Peytcheva, E. (2014) Can interviewer behaviors during ACASI affect data quality? Surv. Pract., 7, no. 5
47. West, B. T., Welch, K. B. and Galecki, A. T. (2014) Linear Mixed Models: a Practical Guide using Statistical Software, 2nd edn. Boca Raton: Chapman and Hall–CRC.
48. van der Zouwen, J., Dijkstra, W. and Smit, J. H. (1991) Studying respondent-interviewer interaction: the relationship between interviewing style, interviewer behavior, and response behavior. InMeasurement Errors in Surveys (eds P. P. Biemer, R. M. Groves, L. Lyberg, N. A. Mathiowetz and S. Sudman), pp. 419–438. New York: Wiley.