Response Quality of Self-Administered Questionnaires: A Comparison Between Paper and Web Questionnaires

Social Science Computer Review

Volumn 32, Issue 2, 2014, Pages 256-269

  Rada, Vidal Díaz de ^a   Domínguez Álvarez, Juan Antonio ^b  

^a PUBLIC UNIVERSITY OF NAVARRE   (Spain)

^b bienestar subjetivo y comportamiento humano del Instituto de Estudios Sociales Avanzados Consejo Superior de Investigaciones Científicas   (Spain)

Author keywords

mail survey; response quality; self administered questionnaire; web survey

Indexed keywords

EID: 84898904131     PISSN: 08944393     EISSN: 15528286     Source Type: Journal    
DOI: 10.1177/0894439313508516     Document Type: Article

References (67)

1. American Association for Public Opinion Research. (2011). Standard Definitions: Final dispositions of case codes and outcome rates for surveys. (7th ed.). Retrieved fromhttp://www.aapor.org/Standard_Definitions2.htm.
2. Boyer K. K.,Olson J. R.,Calatone R. J.,Jackson E. C.Print versus electronic surveys: A comparison of two data collection methodologies.Journal of Operations Management. 2002;20:357-373.
3. Bradley N.Sampling for Internet surveys. An examination of respondent selection for Internet research.International Journal of Market Research. 1999;41:387-395.
4. Brecko B. N.,Carstens RProceedings of the 2nd IEA International Research Conference. Washington, DC:; 2006:.
5. Cobanoglu C.,Warde B.,Moreo P. J.A comparison of mail, fax and web-based survey methods.International Journal of Market Research. 2001;43:441-452.
6. Converse P. D.,Wolfe E. W.,Huang X.,Oswald F. L.Response rates for mixed-mode surveys using mail and e-mail/Web.American Journal of Evaluation. 2008;29 (1): 99-107.
7. Couper M.P.Technology trends in survey data collection.Social Science Computer Review. 2005;23:486-501.
8. Couper M. PDesigning effective web surveys. New York, NY: Cambridge University Press; 2008:.
9. Couper M. P.The future of modes of data collection.Public Opinion Quarterly. 2011;75:889-908.
10. Couper M.,Blair J.,Triplett T.Journal of Official Statistics. 1999:39-56.
11. Couper M. P.,Bosnjak M.Handbook of survey research. 2010). Internet surveys. In MarsdenP. V.WrightJ. D. (Eds.), Handbook of survey research (2nd ed., pp. 527-550). Bingley, England: Emerald Group Publishing Limited.Marsden P. V.Wright J. D., ed. Bingley, England: Emerald Group Publishing Limited; 2010:527-550.
12. Davidov E.,Depner F.Testing for measurement equivalence of human values across online and paper and pencil surveys.Quality & Quantity. 2011;45:375-390.
13. De Leeuw E. D.To mix or not to mix data collection modes in surveys.Journal of Official Statistics. 2005;21:233-255.
14. Denscombe M.Web based questionnaires and the mode effect. An evaluation based on item non-response rates and data contents of near-identical questionnaires delivered in different modes.Social Science Computer Review. 2006;24:246-254.
15. Denscombe M.Item non-response rates: A comparison of online and paper questionnaires.International Journal of Social Research Methodology. 2009;12:281-291.
16. DeRouvray C.,Couper M. P.Designing a strategy for capturing respondent uncertainty in web-based surveys.Social Science Computer Review. 2002;20:3-9.
17. Dillman D. A.Mail and telephone surveys. New York, NY: Wiley; 1978:.
18. Dillman D. A.International handbook of survey methodology. de Leeuw E. D.Hox J. J.Dillman D. A., ed. New York, NY: Lawrence Erlbaum Associates and European Association of Methodology; 2008:171-175.
19. Dillman D. A.,Kirsten K. W.,Clark J.R.Public Opinion Quarterly. 1994:557-68.
20. DillmanD. A.SmythJ. D.ChristianL. M.SternM. J. (2003). Multiple answer questions in self-administered surveys: The use of check-all-apply and forced-choice question formats. Retrieved fromhttp://www.websm.org/uploadi/editor/File/Dillman_2002_Multiple_Answer_Questions.pdf.
21. Dillman D. A.,Smyth J. D.,Christian L. M.Internet, mail, and mixed-mode surveys: The tailored design method. 2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). New York, NY: John Wiley.New York, NY: John Wiley; 2009:.
22. Diment K.,Garrett-Jones S.How demographic characteristics affect mode preference in a postal/web mixed mode survey of Australian researchers.Social Science Computer Review. 2007;25:410-417.
23. Fowler F. J.,Mangione T. W.Standardized survey interviewing: Minimizing interviewer-related error. Thousand Oaks, CA: Sage; 1990:.
24. Fricker R.,Schonlau M.Advantages and disadvantages of internet research surveys: Evidence tom the literature.Field Methods. 2002;14:347-367.
25. Fricker S.,Galesic M.,Tourangeau R.,Yan T.An experimental comparison of web and telephone surveys.Public Opinion Quarterly. 2005;69:370-392.
26. GreenM. C.KrosnickJ. A.HolbrookA. L. (2001). The survey response process in telephone and face-to-face surveys: Differences in respondent satisficing and social desirability response bias. Retrieved fromhttp://www.umich.edu/?nes/resources/techrpts/tech-abs/tech-ab62.htm.
27. Griffin D. H.,Fischer D. P.,Morgan M. T.Testing an Internet response option for the American Community Survey. Paper presented at the annual meeting of the American Association for Public Opinion Research, Montreal, Canada. 2001;:.
28. Groves R. M.Three eras of survey research.Public Opinion Quarterly. 2012;75:861-871.
29. Heerwegh D.Mode differences between face-to-face and web surveys: An experimental investigation of data quality and social desirability effects.International Journal of Public Opinion Research. 2009;21:111-121.
30. Heerwegh D.,Loosveldt G.Face to face versus web surveying in a high Internet coverage population.Public Opinion Quarterly. 2008;72:836-846.
31. Holbrook A. L.,Green M. C.,Krosnick J. A.Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias.Public Opinion Quarterly. 2003;67:79-125.
32. Holmberg A.,Lorenc B.Proceedings of Q2008 European Conference on Quality in Official Statistics.Understanding the decision to participate in a survey and the choice of the response mode. 2008;:.
33. Holmberg A.,Lorenc B.,Werner P.Contact strategies to improve participation via the Web in a mixed-mode mail and Web surveys.Journal of Official Statistics. 2010;26:465-480.
34. Ilieva J.,Baron S.,Healey N. M.Online surveys in marketing research: Pros and cons.International Journal of Market Research. 2002;44:362-380.
35. IsraelG.LammA. J. (2012). Item non-response in a client survey of the general public. Survey Practice, April. Retrieved fromhttp://surveypractice.wordpress.com/2012/4.
36. Johnson T. P.,Fendrich M. E.,Mackesy-Amiti M. E.An evaluation of the validity of the Crowne-Marlowe need for approval scale.Quality and Quantity. 2012;46:1883-1896.
37. Jones R.,Pitt N.Health surveys in the workplace.Occupational Medicine. 2000;48:556-558.
38. Kreuter F.,Presser S.,Tourangeau R.Social desirability bias in CATI, IVR, and web surveys: The effects of mode and question sensitivity.Public Opinion Quarterly. 2008;72:847-865.
39. Krosnick J. A.Response strategies fr coping with the cognitive demands of attitude measures in surveys.Applied cognitive psychology. 1991;5:213-236.
40. Krosnick J. A.Survey research.Annual Review of Psychology. 1999;50:537-567.
41. Kwak N.,Radler B.A comparison between mail and web surveys: Response pattern, respondent profile and data quality.Journal of Official Statistics. 2002;18:257-273.
42. Lozar Manfreda K.,Bosnjak M.,Berzelak J.,Haas I.,Vehovar V.Web surveys versus other survey modes: A meta-analysis comparing response rates.International Journal of Market Research. 2008;50:79-104.
43. Lozar Manfreda K.,Vehovar V.Development in social research methodology. Ferlijow A.Mrvar A., ed. Ljubljana, SI: Metodoloski Zvezki; 2002:149-169.
44. Mangione T. W.Mail surveys: Improving the quality. Thousand Oaks, CA: Sage; 1995:.
45. MesserB. L.MichelleL. E.DillmanD. A. (2012). Determinants of item nonresponse to web and mail respondents in three address-bases mixed-mode surveys of the general public, Survey Practice, 5. Retrieved fromhttp://surveypractice.wordpress.com/2012/4.
46. MillarM. M.DillmanD. A. (2012). Do Mail and Internet surveys produce different item nonresponse rates, Survey Practice, 5. Retrieved fromhttp://surveypractice.wordpress.com/2012/4.
47. Miller T.,Kobayashi M.,Caldwell E.,Thurston S.,Collett B.Citizen surveys on the web: General population surveys of community opinion.Social Science Computer Review. 2002;20:124-136.
48. MoscosoD.MoyanoE.SesmaD.RuizJ.DomínguezJ. A.TirantiS.PasadasS.SotomayorR.TrujilloM. (2010). Condiciones de Vida y Expectativas de Retorno de los Andaluces en el Exterior, IESA-CSIC / Consejería de Gobernación y Justicia. Junta de Andalucía. Retrieved fromhttp://www.iesa.csic.es/publicaciones/detallarpublicacion/id/57.
49. Muñoz Leiva F.,Sánchez Fernández J.,Montoro Ríos F.,Ibañez Zapata J. Á.Improving the response rate and quality in web-based surveys through the personalization and frequency of reminder mailings.Quality and Quantity. 2009;44:1037-1052.
50. Parker L.Collecting data the e-mail way.Training and Development. 1992;46:52-54.
51. Rasinski K. A.,Mingay D.,Bradburn N. M.Do respondents really mark al that aplly on self-administered questions?.Public Opinion Quarterly. 1994;58:400-408.
52. Reja U.,Lozar Manfreda K.,Hlebec V.,Vehovar V.Open-ended vs. close-ended questions in web questionnaires.Advances in Methodology and Statistics (Metodološki zvezki). 2003;19:159-177.
53. Rossi P. H.,Wright J. D.,Anderson A. B.Handbook of survey research. Rossi P. H.Wright J. D.Anderson A. B., ed. New York, NY: Academic Press; 1983:1-20.
54. Schaefer D.,Dillman D. A.Development of a standard e-mail methodology: Results of an experiment.Public Opinion Quarterly. 1998;62:378-397.
55. Smyth J. D.,Dillman D. A.,Christian L. M.,Mcbride M.Open-ended questions in web surveys can increasing the size of answer boxes and providing extra verbal instructions improve response quality?.Public Opinion Quarterly. 2009;73:325-337.
56. Smyth J. D.,Dillman D. A.,Christian L. M.,O'Neill A. C.Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21st century.American Behavioral Scientist. 2010;53:1423-1448.
57. SmythJ. D.DillmanD. A.ChristianL. M.SternM. J. (2005). Comparing check-all and forced-choice question formats in web survey: The role of satisficing, depth of processing, and acquiescence in explaining differences(SESRC Technical Report 05-029). Pullman, WA. Retrieved fromhttp://survey.sesrc.wsu.edu/dillman/papersweb/2005.htmldoi: 10.1093/poq/nfj007.
58. Stangl AMode effects on data quality: Benefits and drawbacks of mixed mode surveys. Paper presented at the 27th Centre for International Research on Economic Tendency Surveys Conference, Warsaw:; 2004:.
59. Stern M. J.,Smyth J. D.,Mendez J.The effects of item saliency and question design on measurement error in a self-administered survey.Field Methods. 2012;24:3-27.
60. TeclawR.OsatukeC.YanovskyB.MooreS.DyrenforthS. (2010, October). Comparison of paper, web, and IVR responses in the veterans health administration survey. Survey Practice. Retrieved fromhttp://www.surveypractice.org.
61. Thomas R. K.,Klein J. D.Merely incidental?. Effects of response format on self-reported behavior.Journal of Official Statistics. 2006;22:221-244.
62. Tourangeau R.Cognitive aspects of survey methodology: Building a bridge between disciplines. Jabine T.Straf M.Tanur J.Tourangeau R., ed. Washingron, DC: National Academic Press; 1984:73-100.
63. Tourangeau R.,Rasinski K.Cognitive processes underlying context effects in attitude measurement.Psychological Bulletin. 1988;103:299-314.
64. Tourangeau R.,Rips L. J.,Rasinski KThe psychology of survey response. Cambridge, UK: Cambridge University Press; 2000:.
65. Watson J. T.,Lissitz R. W.,Rudner L. M.The influence of web-based questionnaire presentation variations on survey cooperation and perception of survey quality.Journal of Official Statistics. 2006;22:271-291.
66. Werner P.Linköping Studies in Statistics, no. 6. Doctoral Thesis. Linköping University.On the cost-efficiency of probability sampling based mail surveys with web response option. 2005;:.
67. Wright J.,Marsden P. VHandbook of Survey Research. 2010). Survey research and social science: History, current practice and future prospects. In MarsdenP. V.WrightJ. D. (Eds.), Handbook of Survey Research (pp. 1-25). Bingley, UK: Emerald Group Publishing Limited. doi:10.1177/009430610503400458.Marsden P. V.Wright J. D., ed. Bingley, UK: Emerald Group Publishing Limited; 2010:1-25.