Web surveys versus other survey modes: A meta-analysis comparing response rates

International Journal of Market Research

Volumn 50, Issue 1, 2008, Pages 79-104

  Manfreda, Katja Lozar ^a   Bosnjak, Michael ^b   Berzelak, Jernej ^a   Haas, Iris ^c   Vehovar, Vasja ^a  

^a UNIVERSITY OF LJUBLJANA   (Slovenia)

^b FREE UNIVERSITY OF BOZEN BOLZANO   (Italy)

^c UNIVERSITY OF MANNHEIM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 39149083411     PISSN: 14707853     EISSN: None     Source Type: Journal    
DOI: 10.1177/147078530805000107     Document Type: Article

References (62)

1. Bason, J.J. (2000) Comparing results from telephone mail internet and interactive voice recognition surveys of drug and alcohol use among University of Georgia students. Paper presented at the American Association for Public Opinion Research 55th Annual Conference, Portland, USA, 17-20 May 2000.
2. Bates, N. (2001) Internet versus mail as data collection methodology from a high coverage population. Proceedings of the Annual Meeting of the American Statistical Association, 5-9 August 2001.
3. Birnholtz, J.P., Horn, D.B., Finholt, T.A. & Bae, S.J. (2004) The effects of cash, electronic, and paper gift certificates as respondent incentives for a web-based survey of a technologically sophisticated sample. Social Science Computer Review, 22, 3, pp. 377-384.
4. Bosnjak, M. & Tuten, T.L. (2003) Prepaid and promised incentives in web surveys - an experiment. Social Science Computer Review, 21, 2, pp. 208-217.
5. Braithwaite, D., Emery, J., de Lusignan, S. & Sutton, S. (2003) Using the internet to conduct surveys of health professionals: a valid alternative? Family Practice, 20, 5, pp. 545-551.
6. Chatt, C. & Dennis, J.M. (2003) Data collection mode effects controlling for sample origins in a panel survey: telephone versus internet. Paper presented at the 2003 Annual Meeting of the Midwest Chapter of the American Association for Public Opinion Research, Chicago, USA.
7. Chisholm, J. (1998) Using the internet to measure and increase customer satisfaction and loyalty. White paper by CustomerSat.com.
8. Cobanoglu, C., Warde, B. & Moreo, P.J. (2001) A comparison of mail, fax, and web-based survey methods. International Journal of Market Research, 43, 4, pp. 441-452.
9. Cooper, H. & Hedges, L.V. (1994) The Handbook of Research Synthesis. New York: Russel Sage Foundation.
10. Couper, M.P. (2000) Web surveys: a review of issues and approaches. Public Opinion Quarterly, 64, 4, pp. 464-494.
11. Crawford, S., McCabe, S., Couper, M. & Boyd, C. (2001) From mail to web: improving response rates and data collection efficiencies. Paper presented at the International Conference on Improving Surveys, Copenhagen, Denmark.
12. de Leeuw, E. & de Heer, W. (2002) Trends in household survey nonresponse: a longitudinal and international comparison. In: R.M. Groves, D.A. Dillman, J.L. Eltinge & R.J.A Little (eds) Survey Nonresponse. New York: Wiley, pp. 41-54.
13. Dillman, D.A. (2000) Mail and Internet Surveys. The Tailored Design Method. New York: Wiley.
14. Elder, A. & Incalcatera, T. (2000) Pushing the envelope: moving a major syndicated study to the web. Paper presented at the Net Effects 3 Conference, Dublin, Ireland.
15. Evans, J.R. & Mathur, A. (2005) The value of online surveys. Internet Research, 15, 2, pp. 195-219.
16. Fraze, S., Hardin, K., Brashears, T., Smith, J.H. & Lockaby, J. (2002) The effects of delivery mode upon survey response rate and perceived attitudes of Texas agriscience teachers. Paper presented at the National Agricultural Education Research Conference, Las Vegas, USA.
17. Frick, A., Bächtiger, M.T. & Reips, U.-D. (1999) Financial incentives, personal information and drop-out rate in online studies. In: U.-D. Reips, B. Batinic, W. Bandilla, M. Bosnjak, L. Graf, K. Moser & A.A. Werner (eds) Current internet science - trends, techniques, results. Aktuelle Online Forschung - Trends, Techniken, Ergebnisse. Zurich: Online Press. Retrieved from http://dgof.de/tband99/.
18. Fricker, S., Galešić, M., Tourangeau, R. & Yan, T. (2003) An experimental comparison of web and telephone surveys. Working paper.
19. Glass, G.V. (1976) Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 10, pp. 3-8.
20. Glass, G.V., McGaw, B. & Smith, M.L. (1981) Meta-Analysis in Social Research. London: Sage.
21. Grigorian, K.H., Sederstrom, S. & Hoffer, T.B. (2004) Web of intrigue? Evaluating effects on response rates of between web SAQ, CATI, and mail SAQ options in a national panel survey. Paper presented at the American Association for Public Opinion Research 59th Annual Conference, Phoenix, USA.
22. Groves, R.M. (1989) Survey Errors and Survey Costs. New York: Wiley.
23. Groves, R.M. (2006) Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70, 5, pp. 646-675.
24. Groves, R.M., Cialdini, R.B. & Couper, M.P. (1992) Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 4, pp. 475-495.
25. Hayslett, M.M. & Wildemuth, B.M. (2004) Pixels or pencils? The relative effectiveness of web-based versus paper surveys. Library & Information Science Research, 26, 1, pp. 73-93.
26. Hedges, L.V. & Olkin, I. (1985) Statistical Methods for Meta-Analysis. San Diego: Academic Press.
27. Hedges, L.V. & Vevea, J.L. (1998) Fixed- and random-effects models in metaanalysis. Psychological Methods, 3, 4, pp. 486-504.
28. Hunter, J.E. & Schmidt, F.L. (2004) Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Newbury Park: Sage Publications.
29. Jones, R. & Pitt, N. (1999) Health surveys in the workplace: comparison of postal, email and world wide web methods. Occupational Medicine, 49, 8, pp. 556-558.
30. Kaplowitz, M.D., Hadlock, T.D. & Levine, R. (2004) A comparison of web and mail survey response rates. Public Opinion Quarterly, 68, 1, pp. 94-101.
31. Keeter, S., Miller, K., Kohut, A., Groves, R.M. & Presser, S. (2000) Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly, 64, 2, pp. 125-148.
32. Kerwin, J., Brick, P.D., Levin, K., O'Brien, J., Cantor, D., Wang, A., Campbell, S. & Shipp, S. (2004) Web, mail, and mixed-mode data collection in a survey of advanced technology program applicants. Paper presented at the 2004 Joint Statistics Meetings, Toronto, Canada.
33. Kittleson, M.J. (1997) Determining effective follow-up of email surveys. American Journal of Health Behavior, 21, 3, pp. 193-196.
34. Knapp, F. & Heidingsfelder, M. (2001) Drop-out analysis: effects of the survey design. In: U.-D. Reips & M. Bosnjak (eds) Dimensions of Internet Science. Lengerich: Pabst Science Publishers, pp. 221-230.
35. Knapp, H. & Kirk, S.A. (2003) Using pencil and paper, internet and touch-tone phones for self-administered surveys: does methodology matter? Computers in Human Behaviour, 19, 1, pp. 117-134.
36. Kwak, N. & Radler, B.T. (1999) A comparison between mail and web surveys: response pattern, respondent profile, data quality, and construct association. Paper presented at the annual meeting of Midwest Association of Public Opinion Research, Chicago, USA.
37. Lesser, V.M. & Newton, L. (2001) Mail, email and web surveys: a cost and response rate comparison in a study of undergraduate research activity. Paper presented at the American Association for Public Opinion Research 56th Annual Conference, Montreal, Canada.
38. Lipsey, M.W. & Wilson, D.B. (2001) Practical Meta-Analysis. Thousand Oaks: Sage Publications.
39. Lozar Manfreda, K. & Vehovar, V. (2006) Internet surveys. In: J. Hox, E. de Leeuw & D.A. Dillman (eds) The International Handbook of Survey Methodology. Lawrence Erlbaum Associates, New Jersey, USA.
40. Lozar Manfreda, K., Vehovar, V. & Batagelj, Z. (2001) Web versus mail questionnaire for an institutional survey. Paper presented at the 2nd ASC International Conference on Survey Research Methods: The Challenge of the Internet, UK.
41. MacElroy, B. (2000) Variables influencing dropout rates in web-based surveys. Quirk's Marketing Research Review, July.
42. McNeish, J. (2001) Using the internet for data collection - just because we can, should we? Paper presented at 2001 AAPOR Annual Conference, Montreal, Canada, 17-20 May.
43. Miller, T.I., Miller Kobayashi, M., Caldwell, E., Thurston, S. & Collett, B. (2002) Citizen surveys on the web: general population surveys of community opinion. Social Science Computer Review, 20, 2, pp. 124-136.
44. Mullen, B., Muellerleile, P. & Bryant, B. (2001) Cumulative meta-analysis: a consideration of indicators of sufficiency and stability. Personality and Social Psychology Bulletin, 27, 11, pp. 1450-1462.
45. Pineau, V. & Slotwiner, D. (2004) Probability samples vs volunteer respondents in internet research: defining potential effects on data and decision-making in marketing applications. Retrieved from Knowledge Networks (www.knowledgenetworks.com).
46. Pötschke, M. (2004) Paper and pencil or online? Methodological experiences from an employee survey. Paper presented at the German Online Research Conference (GOR) 2004, Duisburg, Germany.
47. Rosenberg, M.S., Adams, D.C. & Gurevitch, J. (2000) MetaWin: Statistical Software for Meta-Analysis, Version 2.0. Sunderland: Sinauer Associates.
48. Rosenthal, R. (1979) The 'file drawer problem' and tolerance for null results. Psychological Bulletin, 86, 3, pp. 638-641.
49. Roster, C.A., Rogers, R.D., Albaurn, G. & Klein, J.D. (2004) A comparison of response characteristics from web and telephone surveys. International Journal of Market Research, 46, 3, pp. 359-373.
50. Sax, L.J., Gilmartin, S.K. & Bryant, A.N. (2003) Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44, 4, pp. 409-432.
51. Schonlau, M., Elliot, M.N. & Fricker, R.D. (2002) Conducting Research Surveys via E-mail and the Web. Rand: Santa Monica.
52. Smith, M.L. & Glass, G.V. (1977) Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 9, pp. 752-760.
53. Truell, A.D. (2003) Use of internet tools for survey research. Information Technology, Learning and Performance Journal, 21, 1, pp. 31-37.
54. Tuten, T.L. (1997) Getting a Foot in the Electronic Door: Understanding Why People Read or Delete Electronic Mail (Rep. No. 97/08). Mannheim: Zentrum für Umfragen, Methoden und Analysen.
55. Tuten, XL., Bosnjak, M. & Bandilla, W. (1999/2000) Banner-advertised web surveys. Marketing Research, 11, 4, pp. 16-21.
56. Tuten, T.L., Galešič, M. & Bosnjak, M. (2004) Effects of immediate versus delayed notification of prize draw results on response behavior in web surveys: an experiment. Social Science Computer Review, 22, 3, pp. 377-384.
57. Tuten, T.L., Urban, D.J. & Bosnjak, M. (2002) Internet surveys and data quality - a review. In: B. Batinic, U.-D. Reips & M. Bosnjak (eds) Online Social Sciences. Seattle, WA: Hogrefe & Huber Publishers, pp. 7-26.
58. Vehovar, V., Batagelj, Z., Lozar Manfreda, K. & Zaletel, M. (2002) Nonresponse in web surveys. In: R.M. Groves, D.A. Dillman, J.L. Eltinge & R.J.A. Little (eds) Survey Nonresponse. New York: Wiley, pp. 229-242.
59. Vehovar, V., Lozar Manfreda, K. & Batagelj, Z. (2001) Sensitivity of e-commerce measurement to the survey instrument. International Journal of Electronic Commerce, 6, 1, pp. 31-52.
60. Wang, M. C. & Bushman, B.J. (1998) Using the normal quantile plot to explore meta-analytic data sets. Psychological Methods, 3, 1, pp. 46-54.
61. Weible, R. & Wallace, J. (1998) Cyber research: the impact of the internet on data collection. Marketing Research, 10, 3, pp. 19-24.
62. Wygant, S. & Lindorf, R. (1999) Surveying collegiate net surfers. Quirk's Marketing Research Review, July.