Survey mode effects on data quality: Comparison of web and mail modes in a U.S. national panel survey

Social Science Computer Review

Volumn 30, Issue 2, 2012, Pages 212-228

  Shin, Eunjung ^a   Johnson, Timothy P ^a   Rao, Kumar ^b  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

^b ACNielsen   (New Zealand)

Author keywords

data quality; mixed mode survey; mode comparison; mode effects; web survey

Indexed keywords

EID: 84860124832     PISSN: 08944393     EISSN: 15528286     Source Type: Journal    
DOI: 10.1177/0894439311404508     Document Type: Article

References (41)

1. Akl E. A.,Maroun N.,Klocke R. A.,Montori V.,Schunemann H. J.Electronic mail was not better than postal mail for surveying residents and faculty.Journal of Clinical Epidemiology. 2005;58:425-429.
2. Standard definitions: Final disposition case codes and outcome rates for surveys. 2011). Standard definitions: Final disposition case codes and outcome rates for surveys. (7th ed.). AAPOR.AAPOR; 2011:.
3. Bachmann D.,Elfrink J.,Vazzana G.Tracking the progress of e-mail versus snail-mail.Marketing Research. 1996;8:31-35.
4. Bandillar W.,Bosnjak M.,Altdorfer P.Survey administration effects? A comparison of web-based and traditional written self-administrated surveys using the ISSP environment module.Social Science Computer Review. 2003;21:235-243.
5. Bech M.,Kristensen M. B.Differential response rates in postal and web-based surveys among older respondents.Survey Research Methods. 2009;3:1-6.
6. Beebe T. J.,Locke G. R. III, Barnes, S. A.,Davern M. E.,Anderson K. J.Mixing web and mail methods in a survey of physicians.Health Services Research. 2007;42:1219-1234.
7. Börkan B.The mode effect in mixed-mode surveys mail and web surveys.Social Science Computer Review. 2010;28:371-380.
8. Christian L. M.,Dillman D. A.The influence of graphical and symbolic language manipulations on responses to self-administered questions.Public Opinion Quarterly. 2004;68:57-80.
9. Converse P. D.,Wolfe E. W.,Huang X.,Oswald F. L.Response rates for mixed-mode surveys using mail and e-mail/web.American Journal of Evaluation. 2008;29:99-107.
10. Couper M. P.Web Surveys: A review of issues and approaches.Public Opinion Quarterly. 2000;64 (4): 464-494.
11. Couper M. P.Designing effective web surveys. New York, NY: Cambridge University Press; 2008:.
12. Couper M. P.,Blair J.,Triplett T.A comparison of mail and e-mail for a survey of employees in U.S. statistics agencies.Journal of Official Statistics. 1999;15:39-56.
13. Couper M. P.,Miller P. V.Web survey method: Introduction.Public Opinion Quarterly. 2008;72:831-835.
14. Das M.,Ester P.,Kaczmirek L.Social and behavioral research and the internet: Advances in applied methods and research strategies. New York, NY: Routledge; 2011:.
15. De Leeuw E. D.To mix or not to mix data collection modes in surveys.Journal of Official Statistics. 2005;21:233-255.
16. Denscombe M.Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes.Social Science Computer Review. 2006;24:246-254.
17. Denscombe M.The length of responses to open-ended questions: A comparison of online and paper questionnaires in terms of a mode effect.Social Science Computer Review. 2008;26:359-368.
18. Denscombe M.Item non-response rates: A comparison of online and paper questionnaires.International Journal of Social Research Methodology. 2009;12:281-291.
19. Dillman D. A.,Smyth J. D.,Christian M. E.Internet, mail and mixed-mode surveys: The tailored design method. Hoboken, NJ: Wiley; 2009a:.
20. Dillman D. A.,Phelps G.,Tortora R.,Swift K.,Kohrell J.,Berck J.,Messer B.Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet.Social Science Research. 2009b;38:1-18.
21. Greenlaw C.,Brown-Welty S.A comparison of web-based and paper-based survey methods: Testing assumptions of survey mode and response cost.Evaluation Review. 2008;33:464-480.
22. Grondin C.,Sun L.In 2008 Proceedings of the section on survey research methods. Alexandria, VA: American Statistical Association; 2009:1255-1262.
23. Heerwegh D.,Loosveldt G.Face-to-face versus web surveying in a high internet coverage population: Differences in response quality.Public Opinion Quarterly. 2008;72:836-846.
24. Kaplowitz M. D.,Hadlock T. D.,Levine R.A comparison of web and mail survey response rates.Public Opinion Quarterly. 2004;68:94-101.
25. Knapp H.,Kirk S. A.Using pencil and paper, internet and touch-tone phones for self-administered surveys: Does methodology matter?.Computers in Human Behavior. 2003;19:117-134.
26. Kwak N.,Radler B.A comparison between mail and web surveys: Response pattern, respondent profile, and data quality.Journal of Official Statistics. 2002;8:257-273.
27. Link M. W.,Mokdad A. H.Effects of survey mode on self-reports of adult alcohol consumption: Comparison of web, mail, and telephone approaches.Journal of Studies on Alcohol. 2005;66:239-245.
28. Link M. W.,Mokdad A. H.Can web and mail survey modes improve participation in an RDD-based national health surveillance?.Journal of Official Statistics. 2006;22:293-312.
29. LorencB. (2010). Item nonresponse analysis for a mixed-mode survey, in CarlsonM.NyquistH.VillaniM. (eds), Official Statistics - Methodology and Applications in Honour of Daniel Thorburn (pp. 117-136). Stockholm, Sweden: Stockholm University. Retrieved fromhttp://officialstatistics.wordpress.com.
30. Manfreda L.,Bosnjak K. M.,Berzelak J.,Haas I.,Vehovar V.Web surveys versus other survey modes: A meta-analysis comparing response rates.International Journal of Market Research. 2008;50:79-104.
31. McCabe S. E.Comparison of web and mail surveys in collecting illicit drug use data: A randomized experiment.Journal of Drug Education. 2004;34:61-72.
32. McCabe S. E.,Boyd C. J.,Couper M. P.,Crawford S.,d'Arcy H.Mode effects for collecting alcohol and other drug use data: Web and U.S. mail.Journal of Studies on Alcohol. 2002;63:755-761.
33. McCabe S. E.,Diez A.,Boyd C. J.,Nelson T. F.,Weitzman E. R.Comparing web and mail responses in a mixed mode survey in college alcohol use research.Addictive Behaviors. 2006;31:1619-1627.
34. Miller T. I.,Kobayashi M. M.,Caldwell E.,Thurston S.,Collett B.Citizen surveys on the web: General population surveys of community opinion.Social Science Computer Review. 2002;20:124-136.
35. Rao K.,Kaminska O.,McCutcheon A.Recruiting probability samples of a multi-mode research panel with internet and mail components.Public Opinion Quarterly. 2010;74:68-84.
36. Roster C. A.,Rogers R. D.,Albaurn G.,Klein J. D.A comparison of response characteristics from web and telephone surveys.International Journal of Market Research. 2004;46:359-373.
37. Schaefer D.,Dillman D.Development of a standard e-mail methodology: Results of an experiment.Public Opinion Quarterly. 1998;62:378-397.
38. Schonlau M.,Asch B.,Du C.Web surveys as part of a mixed-mode strategy for populations that cannot be contacted by e-mail.Social Science Computer Review. 2003;21:218-222.
39. Shih T.,Fan X.Comparing response rates from web and mail surveys: A meta-analysis.Field Methods. 2008;20:249-271.
40. Smyth J. D.,Dillman D. A.,Christian L. M.,McBride M.Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality?.Public Opinion Quarterly. 2009;73:325-337.
41. Wolfe E. W.,Converse P. D.,Oswald F. L.Item-level non-response rates in an attitudinal survey of teachers delivered via mail and web.Journal of Computer-Mediated Communication. 2008;14:35-66.