Social desirability is the same in offline, online, and paper surveys: A meta-analysis

Computers in Human Behavior

Volumn 36, Issue , 2014, Pages 487-495

  Dodou, D ^a   De Winter, J C F ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Decline effect; Meta analysis; Self disclosure; Social desirability

Indexed keywords

INTERNET; RANDOM PROCESSES;

ADMINISTRATION MODES; DECLINE EFFECT; INDEPENDENT SAMPLES; INTERNET CONNECTIVITY; META ANALYSIS; SAMPLING ERRORS; SELF-DISCLOSURE; SOCIAL DESIRABILITY;

SURVEYS;

EID: 84900413939     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2014.04.005     Document Type: Review

References (95)

1. Baydoun, R. B.; & Emperada, T. K. (1995). Equivalence of computer vs. paper-and-pencil administered attitude instruments. Paper presented at the conference of the society for industrial and organizational psychology, Orlando, FL.
2. A. Ben-Ze'ev Privacy, emotional closeness, and openness in cyberspace Computers in Human Behavior 19 2003 451 467
3. M.H. Birnbaum Psychological experiments on the Internet 2000 Elsevier
4. S. Booth-Kewley, J.E. Edwards, and P. Rosenfeld Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? Journal of Applied Psychology 77 1992 562 566
5. S. Booth-Kewley, G.E. Larson, and D.K. Miyoshi Social desirability effects on computerized and paper-and-pencil questionnaires Computers in Human Behavior 23 2007 463 477 (Pubitemid 44278503)
6. M. Borenstein, L.V. Hedges, J.P. Higgins, and H.R. Rothstein Introduction to meta-analysis 2011 John Wiley & Sons West Sussex, UK
7. T. Buchanan Potential of the Internet for personality research M.H. Birnbaum, Psychological experiments on the Internet 2000 Academic Press San Diego, CA 121 140
8. T. Buchanan, and J.L. Smith Using the Internet for psychological research: Personality testing on the World Wide Web British Journal of Psychology 90 1999 125 144
9. A.J. Campbell, S.R. Cumming, and I. Hughes Internet use by the socially fearful: Addiction or therapy? CyberPsychology & Behavior 9 2006 69 81
10. C. Cook, F. Heath, R.L. Thompson, and B. Thompson Score reliability in Web-or-Internet-based surveys: Unnumbered graphic rating scales versus Likert-type scales Educational and Psychological Measurement 61 2001 697 706 (Pubitemid 33403792)
11. S.D. Crawford, M.P. Couper, and M.J. Lamias Web surveys perceptions of burden Social Science Computer Review 19 2001 146 162
12. B.C. Cronk, and J.L. West Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings Behavior Research Methods, Instruments, & Computers 34 2002 177 180
13. Dahl, R. E. (1992). Response bias, interactive effects of interview method, social desirability, evaluation apprehension of self-reported data. Unpublished doctoral dissertation, Georgia State University.
14. R.N. Davis Web-based administration of a personality questionnaire: Comparison with traditional methods Behavior Research Methods, Instruments, & Computers 31 1999 572 577
15. C. Davis, and M. Cowles Automated psychological testing: Method of administration, need for approval, and measures of anxiety Educational and Psychological Measurement 49 1989 311 320
16. A. De Beuckelaer, and F. Lievens Measurement equivalence of paper-and-pencil and Internet organisational surveys: A large scale examination in 16 countries Applied Psychology 58 2009 336 361
17. J.C.F. De Winter, A.A. Zadpoor, and D. Dodou The expansion of Google Scholar versus Web of Science: A longitudinal study Scientometrics 98 2014 1547 1565
18. M. De Zwart, D. Lindsay, M. Henderson, and M. Phillips Teenagers, legal risks and social networking sites 2011 Monash University Faculty of Education
19. S.A. Dwight, and M.E. Feigelson A quantitative review of the effect of computerized testing on the measurement of social desirability Educational and Psychological Measurement 60 2000 340 360
20. W.M. Evan, and J.R. Miller Differential effects on response bias of computer vs. conventional administration of a social science questionnaire: An exploratory methodological experiment Behavioral Science 14 1969 216 227
21. J. Fang, C. Wen, and V. Prybutok The equivalence of Internet versus paper-based surveys in IT/IS adoption research in collectivistic cultures: The impact of satisficing Behaviour & Information Technology 32 2013 480 490
22. J. Fang, C. Wen, and V. Prybutok An assessment of equivalence between paper and social media surveys: The role of social desirability and satisficing Computers in Human Behavior 30 2014 335 343
23. J.E. Finegan, and N.J. Allen Computerized and written questionnaires: Are they equivalent? Computers in Human Behavior 10 1995 483 496 (Pubitemid 124010392)
24. M.S. Finger, and D.S. Ones Psychometric equivalence of the computer and booklet forms of the MMPI: A meta-analysis Psychological Assessment 11 1999 58 66 (Pubitemid 29124733)
25. R.J. Fisher Social desirability bias and the validity of indirect questioning Journal of Consumer Research 1993 303 315
26. S. Fox, and D. Schwartz Social desirability and controllability in computerized and paper-and-pencil personality questionnaires Computers in Human Behavior 18 2002 389 410 (Pubitemid 34295569)
27. C.P. Furner Socially desirable reporting in international computer based data collection: A review and research agenda Journal of International Management Studies 6 2011 185 201
28. J.-F. Gehanno, L. Rollin, and S. Darmoni Is the coverage of Google Scholar enough to be used alone for systematic reviews BMC Medical Informatics and Decision Making 13 2013 7
29. S.D. Gosling, S. Vazire, S. Srivastava, and O.P. John Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires American Psychologist 59 2004 93 104 (Pubitemid 38318025)
30. S.E. Griffis, T.J. Goldsby, and M. Cooper Web-based and mail surveys: A comparison of response, data, and cost Journal of Business Logistics 24 2003 237 258
31. D.R. Hancock, and C.P. Flowers Comparing social desirability responding on World Wide Web and paper-administered surveys Educational Technology Research and Development 49 2001 5 13 (Pubitemid 33399606)
32. L.V. Hedges, and J.L. Vevea Fixed-and random-effects models in meta-analysis Psychological Methods 3 1998 486 504
33. J.G. Helgeson, and M.L. Ursic The decision process equivalency of electronic versus pencil-and-paper data collection methods Social Science Computer Review 7 1989 296 310
34. J. Henrich, S.J. Heine, and A. Norenzayan The weirdest people in the world? Behavioral and Brain Sciences 33 2010 61 83
35. Higgins, J. P. T.; & Green, S. (2011). Cochrane handbook for systematic reviews of interventions version 5.1. 0 [updated March 2011]. The Cochrane Collaboration.
36. J.P. Higgins, S.G. Thompson, J.J. Deeks, and D.G. Altman Measuring inconsistency in meta-analyses BMJ. British Medical Journal 327 2003 557 560 (Pubitemid 37088507)
37. L.M. Honaker, T.H. Harrell, and J.D. Buffaloe Equivalency of microtest computer MMPI administration for standard and special scales Computers in Human Behavior 4 1988 323 337 (Pubitemid 19071401)
38. K. Hood, M. Robling, D. Ingledew, D. Gillespie, G. Greene, and R. Ivins et al. Mode of data elicitation, acquisition and response to surveys: A systematic review Health Technology Assessment 16 2012
39. J.E. Hunter, and F.L. Schmidt Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge International Journal of Selection and Assessment 8 2000 275 292 (Pubitemid 33203822)
40. A. Joinson Social desirability, anonymity, and Internet-based questionnaires Behavior Research Methods, Instruments, & Computers 31 1999 433 438
41. M.D. Kaplowitz, T.D. Hadlock, and R. Levine A comparison of web and mail survey response rates Public Opinion Quarterly 68 2004 94 101 (Pubitemid 38750629)
42. S. Kiesler, and L.S. Sproull Response effects in the electronic survey Public Opinion Quarterly 50 1986 402 413
43. W.C. King, and E.W. Miles A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: A test of measurement equivalence Journal of Applied Psychology 80 1995 643 651 (Pubitemid 126006443)
44. J.A. Krosnick Response strategies for coping with the cognitive demands of attitude measures in surveys Applied Cognitive Psychology 5 1991 213 236
45. I. Krumpal Determinants of social desirability bias in sensitive surveys: A literature review Quality & Quantity 47 2013 2025 2047
46. G.J. Lautenschlager, and V.L. Flaherty Computer administration of questions: More desirable or more social desirability? Journal of Applied Psychology 75 1990 310 314
47. C.L. Martin, and D.H. Nagao Some effects of computerized interviewing on job applicant responses Journal of Applied Psychology 74 1989 72 80
48. Mitchell, D. (1993). A multivariate analysis of the effects of gender and computer vs. paper/pencil modes of administration on survey results. Unpublished doctoral dissertation, Louisiana Tech University.
49. K.A. Pasveer, and J.H. Ellard The making of a personality inventory: Help from the WWW Behavior Research Methods, Instruments, & Computers 30 1998 309 313 (Pubitemid 128685481)
50. D.L. Paulhus Two-component models of socially desirable responding Journal of Personality and Social Psychology 46 1984 598 609
51. D.L. Paulhus Measurement and control of response bias J.P. Robinson, P.R. Shaver, L.S. Wrightsman, Measures of personality and social psychological attitudes 1991 Academic Press New York 17 59
52. F.A. Pettit A comparison of World-Wide Web and paper-and-pencil personality questionnaires Behavior Research Methods, Instruments, & Computers 34 2002 50 54
53. W.L. Richman, S. Kiesler, S. Weisband, and F. Drasgow A meta-analytic study of social desirability distortion in computer-administration questionnaires: Traditional questionnaires and interviews Journal of Applied Psychology 84 1999 754 775
54. P. Rosenfeld, S. Booth-Kewley, J.E. Edwards, and M.D. Thomas Responses on computer surveys: Impression management, social desirability, and the big brother syndrome Computers in Human Behavior 12 1996 263 274 (Pubitemid 126371877)
55. Rossiter, J. C. (2009). A comparison of social desirability bias among four widely used methods of data collection as measured by the impression management subscale of the balance inventory of desirable responding. Doctoral dissertation, Kent State University.
56. J. Schooler Unpublished results hide the decline effect Nature 470 2011 437
57. S.Z. Shariff, S.A. Bejaimal, J.M. Sontrop, A.V. Iansavichus, R.B. Haynes, and M.A. Weir et al. Retrieving clinical evidence: A comparison of PubMed and Google Scholar for quick clinical searches Journal of Medical Internet Research 15 2013
58. R.E. Smith Examination by computer Behavioral Science 8 1963 76 79
59. M.A. Smith, and B. Leigh Virtual subjects: Using the Internet as an alternative source of subjects and research environment Behavior Research Methods, Instruments, & Computers 29 1997 496 505 (Pubitemid 127710801)
60. L.S. Sproull Using electronic mail for data collection in organizational research Academy of Management Journal 29 1986 159 169
61. L. Sproull, and S. Kiesler Reducing social context cues: Electronic mail in organizational communication Management Science 32 1986 1492 1512 (Pubitemid 17482809)
62. L. Tan, and R.C. Grace Social desirability and sexual offenders: A review Sexual Abuse: A Journal of Research and Treatment 20 2008 61 87
63. Toppins, J. D. (1985). Automated administration of the MMPI with a psychiatric population, equivalence validity considerations. Unpublished doctoral dissertation, University of Kansas.
64. R. Tourangeau, and T. Yan Sensitive questions in surveys Psychological Bulletin 133 2007 859 883
65. Z.A. Uriell, and C.M. Dudley Sensitive topics: Are there modal differences? Computers in Human Behavior 25 2009 76 87
66. A. Weigold, I.K. Weigold, and E.J. Russell Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection methods Psychological Methods 18 2013 53 70
67. Weisband, S.; & Kiesler, S. (1996). Self disclosure on computer forms: Meta-analysis and implications. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 3-10).
68. Whelan, T. J. (2008). Effects of survey modality and access controls on perceived anonymity and socially desirable responding. North Carolina State University.
69. D.M. White, C.B. Clements, and R.D. Fowler A comparison of computer administration with standard administration of the MMPI Computers in Human Behavior 1 1985 153 162
70. E.M. Whitener, and H.J. Klein Equivalence of computerized and traditional research methods: The roles of scanning, social environment, and social desirability Computers in Human Behavior 11 1995 65 75
71. J.M. Wilkerson, D.H. Nagao, and C.L. Martin Socially desirable responding in computerized questionnaires: When questionnaire purpose matters more than the mode Journal of Applied Social Psychology 32 2002 544 559
72. Bigot, O. (2011). Passations papier/crayon et informatisées: quelle influence sur les stratégies d'autoprésentation? Doctoral dissertation, Université Rennes 2.
73. B.H. Biskin, and R.L. Kolotkin Effects of computerized administration on scores on the Minnesota Multiphasic Personality Inventory Applied Psychological Measurement 1 1977 543 549
74. Clough, S. J. (2008). Computerized versus paper-and-pencil assessment of socially desirable responding: Score congruence, completion time, and respondent preferences. Doctoral dissertation, University of Iowa.
75. Fastić-Pajk, I. (2005). Socijalno poželjno odgovaranje u različitim modalitetima prezentacije čestica i njegova povezanost s osobinama ličnosti. Ocjenski rad. Filozofski fakultet u Zagrebu, Department of Psychology.
76. R.R. Hart, and M.A. Goldstein Computer-assisted psychological assessment Computers in Human Services 1 1986 69 75 (Pubitemid 16128401)
77. S. Hays, and R.S. McCallum A comparison of the pencil-and-paper and computer-administered Minnesota Multiphasic Personality Inventory-Adolescent Psychology in the Schools 42 2005 605 613 (Pubitemid 41017730)
78. D. Koson, C. Kitchen, M. Kochen, and D. Stodolosky Psychological testing by computer: Effect on response bias Educational and Psychological Measurement 30 1970 803 810
79. M.E. Lambert, R.H. Andrews, K. Rylee, and J.R. Skinner Equivalence of computerized and traditional MMPI administration with substance abusers Computers in Human Behavior 3 1987 139 143
80. R.E. Lushene, H.F. O'Neil Jr.; and T. Dunn Equivalent validity of a completely computerized MMPI Journal of Personality Assessment 38 1974 353 361
81. A.W. Meade, L.C. Michels, and G.J. Lautenschlager Are Internet and paper-and-pencil personality tests truly comparable? An experimental design measurement invariance study Organizational Research Methods 10 2007 322 345
82. T. Merten, and K. Siebert A comparison of computerized and conventional administration of the EPQ-R and the CRS: Further data on the Merten and Ruch (1996) study Personality and Individual Differences 22 1997 283 286 (Pubitemid 127160488)
83. T.B. Pinsoneault Equivalency of computer-assisted and paper-and-pencil administered versions of the Minnesota Multiphasic Personality Inventory-2 Computers in Human Behavior 12 1996 291 300 (Pubitemid 126371879)
84. D. Potosky, and P. Bobko Computer versus paper-and-pencil administration mode and response distortion in noncognitive selection tests Journal of Applied Psychology 82 1997 293 299 (Pubitemid 127370119)
85. V. Rezmovic The effects of computerizing experimentation on response bias variance Research Methods and Instrumentation 9 1977 144 147
86. J. Ridgway, M.J. MacCulloch, and H.E. Mills Some experiences in administering a psychometric test with a light pen and microcomputer International Journal of Man-Machine Studies 17 1982 265 278 (Pubitemid 13206865)
87. E.F. Risko, L.C. Quilty, and J.M. Oakman Socially desirable responding on the web: Investigating the candor hypothesis Journal of Personality Assessment 87 2006 269 276 (Pubitemid 44955922)
88. B.L. Roper, Y.S. Ben-Porath, and J.N. Butcher Comparability and validity of computerized adaptive testing with the MMPI-2 Journal of Personality Assessment 65 1995 358 371
89. R. Sanitioso, and J.H. Reynolds Comparability of standard and computerized administration of two personality questionnaires Personality and Individual Differences 13 1992 899 907
90. D. Schuldberg The MMPI is less sensitive to the automated testing format than it is to repeated testing: Item and scale effects Computers in Human Behavior 4 1988 285 298 (Pubitemid 19071400)
91. Sloan, H. J. (1991). Respondent self-completion on computers in marketing research: Computers compared with alternative collection methods. Doctoral dissertation, The Ohio State University.
92. M. Sukigara Equivalence between computer and booklet administrations of the new Japanese version of the MMPI Educational and Psychological Measurement 56 1996 570 584
93. C.G. Watson, V. Manifold, W.G. Klett, J. Brown, D. Thomas, and D. Anderson Comparability of computer-and booklet-administered Minnesota Multiphasic Personality Inventories among primarily chemically dependent patients Psychological Assessment: A Journal of Consulting and Clinical Psychology 2 1990 276 280
94. Wicherts, J. M. (2002). Meetequivalentie tussen pen & papier en computerafname van vragenlijsten: Alexithymie, sociale wenselijkheid en angst [Measurement equivalence of paper-and pencil and computerized administration of questionnaires: Alexithymia, social desirability, and anxiety]. Unpublished master's thesis, University of Amsterdam.
95. J.M. Wicherts, and M. Bakker Publish (your data) or (let the data) perish! Why not publish your data too? Intelligence 40 2012 73 76