Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments

Psychonomic Bulletin and Review

Volumn 19, Issue 5, 2012, Pages 847-857

  Germine, Laura ^a,b   Nakayama, Ken ^a   Duchaine, Bradley C ^c   Chabris, Christopher F ^d   Chatterjee, Garga ^a   Wilmer, Jeremy B ^b  

^a HARVARD UNIVERSITY   (United States)

^b WELLESLEY COLLEGE   (United States)

^c Dartmouth College   (United States)

^d UNION COLLEGE   (United States)

Author keywords

Cognition; Face recognition; Visual perception; Web based testing

Indexed keywords

EID: 84866771790     PISSN: 10699384     EISSN: 15315320     Source Type: Journal    
DOI: 10.3758/s13423-012-0296-9     Document Type: Article

References (41)

1. Australian Twin Registry. (n. d.). Australian Twin Registry, publications. Retrieved April 20, 2011, from www. twins. org. au/index. php?page=52.
2. Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The "Reading the Mind in the Eyes" Test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, 42, 241-251.
3. Birnbaum, M. H. (2000). Decision making in the lab and on the Web. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 3-34). San Diego, CA: Academic Press.
4. Bowles, D. C., McKone, E., Dawel, A., Duchaine, B., Palermo, R., Schmalzl, L., et al. (2009). Diagnosing prosopagnosia: Effects of ageing, sex, and participant-stimulus ethnic match on the Cambridge Face Memory Test and Cambridge Face Perception Test. Cognitive Neuropsychology, 26, 423-455.
5. Buchanan, T. (2007). Personality testing on the Internet: What we know, and what we do not. In A. N. Joinson, K. Y. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford handbook of Internet psychology (pp. 447-460). New York, NY: Oxford University Press.
6. Buchanan, T., Ali, T., Heffernan, T. M., Ling, J., Parrott, A. C., Rodgers, J., et al. (2005a). Nonequivalence of online and paper-and-pencil psychological tests: The case of the Prospective Memory Questionnaire. Behavior Research Methods, 37, 148-154.
7. Buchanan, T., Johnson, J. A., & Goldberg, L. (2005b). Implementing a Five-Factor personality inventory for use on the Internet. European Journal of Psychological Assessment, 21, 115-127.
8. Buchanan, T., & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World Wide Web. British Journal of Psychology, 90, 125-144.
9. Buhrmester, M. D., Kwang, T., & Gosling, S. D. (2010). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5.
10. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
11. College Board. (2012). SAT percentile ranks for males, females and total group. Retrieved January 20, 2012, from http://professionals. collegeboard. com/profdownload/sat_percentile_ranks_composite_cr_m_w. pdf.
12. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
13. Duchaine, B., Germine, L., & Nakayama, K. (2007). Family resemblance: Ten family members with prosopagnosia and within-class object agnosia. Cognitive Neuropsychology, 24, 419-430.
14. Duchaine, B., & Nakayama, K. (2006). The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants. Neuropsychologia, 44, 576-585.
15. Frey, M. C., & Detterman, D. K. (2004). Scholastic assessment or g? The relationship between the Scholastic Assessment Test and general cognitive ability. Psychological Science, 15, 373-378.
16. Germine, L., Cashdollar, N., Düzel, E., & Duchaine, B. (2011a). A new selective developmental deficit: Impaired object recognition with normal face recognition. Cortex, 47, 598-607.
17. Germine, L. T., Duchaine, B., & Nakayama, K. (2011b). Where cognitive development and aging meet: Face learning ability peaks after age 30. Cognition, 118, 201-210.
18. Germine, L. T., & Hooker, C. I. (2011). Face emotion recognition is related to individual differences in psychosis-proneness. Psychological Medicine, 41, 937-948.
19. Gosling, S., Vazire, S., Srivastava, S., & John, O. (2004). Should we trust Web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93-104.
20. Haworth, C., Harlaar, N., Kovas, Y., Davis, O., Oliver, B., Hayiou-Thomas, M., et al. (2007). Internet cognitive testing of large samples needed in genetic research. Twin Research and Human Genetics, 10, 554-563.
21. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Beyond WEIRD: Towards a broad-based behavioral science. The Behavioral and Brain Sciences, 33, 111-135.
22. Herzmann, G., Danthiir, V., Schacht, A., Sommer, W., & Wilhelm, O. (2008). Toward a comprehensive test battery for face cognition: Assessment of the tasks. Behavior Research Methods, 40, 840-857. doi: 10. 3758/BRM. 40. 3. 840.
23. Kendler, K., Myers, J., Potter, J., & Opalesky, J. (2009). A Web-based study of personality, psychopathology and substance use in twin, other relative and relationship pairs. Twin Research and Human Genetics, 12, 137-141.
24. Krantz, J. H., Ballard, J., & Scher, J. (1997). Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness. Behavior Research Methods, Instruments, & Computers, 29, 264-269.
25. Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 35-60). San Diego, CA: Academic Press.
26. Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. American Psychologist, 59, 105-117.
27. McGraw, K. O., Tew, M. D., & Williams, J. E. (2000). The integrity of Web-delivered experiments: Can you trust the data? Psychological Science, 11, 502-506.
28. Nosek, B. A., Banaji, M., & Greenwald, A. G. (2002). Harvesting implicit group attitudes and beliefs from a demonstration Web site. Group Dynamics: Theory, Research, and Practice, 6, 101-115.
29. Pew Research Center. (2012a). Pew Internet, Internet adoption 1995-2011. Retrieved January 20, 2012, from www. pewinternet. org/Trend-Data/Internet-Adoption. aspx.
30. Pew Research Center. (2012b). Pew Internet, Online activities, 2000-2009. Retrieved January 20, 2012, from www. pewinternet. org/Trend-Data/Online-Activities-20002009. aspx.
31. Plomin, R., DeFries, J. C., McClearn, G. E., & McGuffin, P. (2008). Behavioral genetics (5th ed.). New York, NY: W. H. Freeman.
32. Reips, U.-D. (2000). The Web experiment method: Advantages, disadvantages, and solutions. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 89-117). San Diego, CA: Academic Press.
33. Reips, U.-D. (2002). Standards for Internet-based experimenting. Experimental Psychology, 49, 243-256.
34. Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. N. Joinson, K. Y. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford handbook of Internet psychology (pp. 373-390). New York, NY: Oxford University Press.
35. Ronalds, G. A., DeStavola, B. L., & Leon, D. A. (2005). The cognitive cost of being a twin: Evidence from comparisons within families in the Aberdeen children of the 1950s cohort study. British Medical Journal, 331, 1306.
36. Senior, C., Phillips, M. L., Barnes, J., & David, A. S. (1999). An investigation into the perception of dominance from schematic faces: A study using the World-Wide Web. Behavior Research Methods, 31, 341-346.
37. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology, 100, 330-348.
38. Uttl, B., Graf, P., & Richter, L. K. (2002). Verbal paired associates tests: Limits on validity and reliability. Archives of Clinical Neuropsychology, 17, 569-583.
39. Wechsler, D. (2008). Wechsler Adult Intelligence Scale (4th ed.). San Antonio, TX: Pearson.
40. Wellesley College. (n. d.). Wellesley College-Admission & financial aid-Testing. Retrieved January 20, 2012, from http://web. wellesley. edu/web/Admission/Apply/FAQ/testing. psml.
41. Wilmer, J. B., Germine, L., Chabris, C. F., Chatterjee, G., Williams, M., Loken, E., et al. (2010). Human face recognition ability is specific and highly heritable. Proceedings of the National Academy of Sciences, 107, 5238-5241.