Assessing the effect of data collection mode on measurement

International Statistical Review

Volumn 78, Issue 1, 2010, Pages 3-20

  Jäckle, Annette ^a   Roberts, Caroline ^b   Lynn, Peter ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

^b UNIVERSITY OF LAUSANNE   (Switzerland)

Author keywords

European Social Survey; Experimental design; Mixed modes; Proportional odds; Response process

Indexed keywords

EID: 77955779890     PISSN: 03067734     EISSN: 17515823     Source Type: Journal    
DOI: 10.1111/j.1751-5823.2010.00102.x     Document Type: Article

References (35)

1. Biemer PP. Measuring data quality. Telephone Survey Methodology 1988, 273-282. Groves RMBiemer PPLyberg LEMassey JTNicholls WLWaksberg J. Eds., New York, Wiley
2. Billiet J, Welkenhusen-Gybels H. Assessing cross-national construct equivalence on the ESS: the case of six immigration items. Recent Developments and Applications in Social Research Methodology 2004, Van Dijkum J Blasius, Durand C. Amsterdam, 2004/(CD-Rom), Eds, Opladen (Germany), Barbara Budrick Publishers
3. Braun M. Errors in comparative survey research: an overview. Cross-Cultural Survey Methods 2003, 137-155. Harkness JAVan der Vijver FJRMohler PP. Eds., Hoboken, NJ, John Wiley
4. Cannell C, Miller P, Oksenberg L. Research on interviewing techniques. Sociological Methodology 1981 1981, 389-437. Leinhardt S, Ed., San Francisco, Jossey-Bass
5. de Leeuw E. Data Quality in Mail, Telephone, and Face-to-Face Surveys 1992, Amsterdam, TT Publications
6. de Leeuw E. To mix or not to mix? Data collection modes in surveys. J. Off. Statist. 2005, 21:1-23.
7. de Leeuw E, van der Zouwen J. Data quality in telephone and face-to-face surveys: a comparative analysis. Telephone Survey Methodology 1988, 283-299. Groves RM, Biemer PPLyberg LEMassey JTNicholls WLWaksberg J. Eds., New York, Wiley
8. DeMaio TJ. Social desirability and survey measurement: a review. Surveying Subjective Phenomena 1984, 257-282. Turner CFMartin E. Eds., New York, Russell Sage Foundation
9. Deming WE. On errors in surveys. Amer. Soc. Rev. 1944, 9:359-369.
10. Dillman DA. Mail and Internet Surveys: The Tailored Design Method 2000, 2nd ed, New York, John Wiley
11. Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, Messer BL. Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response (IVR) and the internet. Soc. Sci. Res. 2009, 38:1-18.
12. Eva G, Roberts CE, Lynn P. Measuring the effect of interview length on telephone survey co-operation. 2008, May 15-18, New Orleans, LA
13. Fricker S, Galesic M, Tourangeau R, Yan T. An experimental comparison of web and telephone surveys. Publ. Opin. Quart. 2005, 69:370-392.
14. Greenfield TK, Midanik LT, Rogers JD. Effects of telephone versus face-to-face interview modes on reports of alcohol consumption. Addiction 2000, 95:277-284.
15. Groves RM, Kahn RL. Surveys by Telephone: A National Comparison with Personal Interviews 1979, New York, Academic Press
16. Hawkins DI, Albaum G, Best R. Stapel scale or semantic differential in marketing research?. J. Marketing Res. 1974, 11:318-322.
17. Hochstim JR. A critical comparison of three strategies of collecting data from households. J. Amer. Statist. Assoc. 1967, 62:976-989.
18. Holbrook AL, Green MC, Krosnick JA. Telephone vs. face-to-face interviewing of national probability samples with long questionnaires: comparisons of respondent satisficing and social desirability response bias. Pub. Opin. Quart. 2003, 67:79-125.
19. James JW. Comparing regression and correlation coefficients. Appl. Statist. 1964, 13:127-132.
20. Janssen B, Schouten B. Survey errors in mixed mode pilots that incorporate face-to-face, telephone, paper and web interviewing. 2007, Prague
21. Jordan LA, Marcus AC, Reeder LG. Response styles in telephone and household interviewing - a field experiment. Publ. Opin. Quart. 1980, 44:210-222.
22. Jäckle A, Roberts CE, Lynn P. Telephone versus face-to-face interviewing: mode effects on data quality and likely causes (Report on Phase II of the ESS-Gallup Mixed Mode Methodology Project). 2006, http://www.iser.essex.ac.uk/pubs/workpaps/pdf/2006-41.pdf), ISER Working Paper 2006-41, Colchester, UK, University of Essex, available online at
23. Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cog. Psy. 1991, 5:213-236.
24. Long JS. Regression Models for Categorical and Limited Dependent Variables 1997, Thousand Oaks, CA, Sage
25. Ludbrook J. Multiple comparison procedures updated. Clin. Exp. Pharmacol. Physiol. 1998, 25:1032-1037.
26. Mooney G, Giesbrecht L, Shettle C. To pay or not to pay: that is the question. 1993, St. Petersburg, FL
27. O'Connell AA. Logistic Regression Models for Ordinal Response Variables 2006, Thousand Oaks, CA, Sage
28. Peytcheva EA, Manchin R, Tortora R, Groves RM. Comparing face to face, telephone, paper self-administered and web survey measurement. 2004, Phoenix, AZ
29. Roberts CE. Mixing modes of data collection in surveys. 2007, http://www.ncrm.ac.uk/research/outputs/publications/methodsreview/MethodsReviewPaperNCRM-008.pdf), ESRC National Centre for Research Methods Methods Review Papers, NCRM/008, Southampton, UK, University of Southampton, available online at
30. Roberts CE, Eva G, Allum N, Lynn P. Does length matter? The effect of interview duration on data quality in telephone interviews. 2008a, May 15-18, New Orleans, LA
31. Roberts CE, Eva G, Widdop S. Assessing the demand and capacity for mixing modes of data collection on the European Social Survey: final report of the mapping exercise. 2008b, Unpublished manuscript, London, City University
32. Roberts CE, Jäckle A, Lynn P. Causes of mode effects: separating out interviewer and stimulus effects in comparisons of face-to-face and telephone surveys. 2006, 4221-4228. http://www.amstat.org/Sections/Srms/Proceedings/), American Statistical Association, available online at
33. Tourangeau R, Rips LJ, Rasinski KA. The Psychology of Survey Response 2000, Cambridge, Cambridge University Press
34. Voogt RJJ, Saris WE. Mixed mode designs: finding the balance between nonresponse bias and mode effects. J. Off. Statist. 2005, 21:367-387.
35. Williams R. Generalized ordered logit/partial proportional odds models for ordinal dependent variables. Stata J. 2006, 6:58-82.