Lie for a Dime: When Most Prescreening Responses Are Honest but Most Study Participants Are Impostors

Social Psychological and Personality Science

Volumn 8, Issue 5, 2017, Pages 500-508

  Chandler, Jesse J ^a   Paolacci, Gabriele ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b ERASMUS UNIVERSITY   (Netherlands)

Author keywords

consumer behavior; gender; individual differences; measurement; research methods; sexuality

Indexed keywords

EID: 85020091313     PISSN: 19485506     EISSN: 19485514     Source Type: Journal    
DOI: 10.1177/1948550617698203     Document Type: Article

References (48)

1. Arch J. J., (2014). Cognitive behavioral therapy and pharmacotherapy for anxiety: Treatment preferences and credibility among pregnant and non-pregnant women. Behaviour Research and Therapy, 52, 53–60.
2. Arch J. J., Carr A. L., (2016). Using mechanical Turk for research on cancer survivors. Psycho-Oncology. doi:10.1002/pon.4173
3. Beramendi P., Duch R. M., Matsuo A., (2014). When lab subjects meet real people: Comparing different modes of experiments. Paper presented at Asian Political Methodology Conference, Taipai, Taiwan. Retrieved from https://cess-web.nuff.ox.ac.uk/files/pdfs/working_papers/CESS_DP2014_003.pdf
4. Berg J., (2016). Income security in the on-demand economy: Findings and policy lessons from a survey of crowdworkers. Comparative Labor Law & Policy Journal, 37. Retrieved from SSRN: https://ssrn.com/abstract=2740940
5. Bernal D. R., (2014). Acculturation, acculturative stress, social status, and well-being among English language proficient immigrants (Doctoral dissertation). University of Miami. Retrieved from http://scholarlyrepository.miami.edu/oa_dissertations/1328/
6. Brown D. L., Segrist D., (2016). African American career aspirations examining the relative influence of internalized racism. Journal of Career Development, 43, 177–189.
7. Buhrmester M., Kwang T., Gosling S. D., (2011). Amazon’s Mechanical Turk a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5.
8. Casey L., Chandler J., Levine A. S., Proctor A., Strolovitch D., (2017). Demographic characteristics of a large sample of US workers. Retrieved from https://osf.io/preprints/psyarxiv/8352x/
9. Casscells W., Schoenberger A., Graboys T. B., (1978). Interpretation by physicians of clinical laboratory results. New England Journal of Medicine, 299, 999–1001.
10. Cavanagh T. M., (2014). Cheating on online assessment tests: Prevalence and impact on validity (Unpublished doctoral dissertation). Colorado State University. Retrieved from https://dspace.library.colostate.edu/handle/10217/83718
11. Chandler J., Mueller P., Paolacci G., (2014). Nonnaïveté among Amazon mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46, 112–130.
12. Chandler J., Paolacci G., Peer E., Mueller P., Ratliff K. A., (2015). Using nonnaive participants can reduce effect sizes. Psychological Science, 26, 1131–1139.
13. Chandler J., Shapiro D., (2016). Conducting clinical research using crowdsourced convenience samples. Clinical Psychology, 12, 53–81.
14. Connell P. M., Brucks M., Nielsen J. H., (2014). How childhood advertising exposure can create biased product evaluations that persist into adulthood. Journal of Consumer Research, 41, 119–134.
15. Cougle J. R., Hawkins K. A., Macatee R. J., Zvolensky M. J., Sarawgi S., (2014). Multiple facets of problematic anger in regular smokers: Exploring associations with smoking motives and cessation difficulties. Nicotine & Tobacco Research, 16, 881–885.
16. Devine E. G., Waters M. E., Putnam M., Surprise C., O’Malley K., Richambault C., …Ciraulo D. A., (2013). Concealment and fabrication by experienced research subjects. Clinical Trials, 10, 935–948.
17. Downes-Le Guin T., Mechling J., Baker R., (2006). Great results from ambiguous sources: Cleaning Internet panel data. In ESOMAR World Research Gonference: Panel Research 2006. Amsterdam, The Netherlands: ESOMAR.
18. Farrell A. M., Grenier J. H., Leiby J., (2017). Scoundrels or stars? Theory and evidence on the quality of workers in online labor markets. The Accounting Review, 92, 93–114.
19. Fergus T. A., Rowatt W. C., (2015). Uncertainty, god, and scrupulosity: Uncertainty salience and priming God concepts interact to cause greater fears of sin. Journal of Behavior Therapy and Experimental Psychiatry, 46, 93–98.
20. Gneezy U., Imas A., (in press). Lab in the field: Measuring preferences in the wild. In Banerjee A., Duflo E., (Eds.), Handbook of field experiments. doi:10.1016/bs.hefe.2016.08.003
21. Goodman J. K., Cryder C. E., Cheema A., (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26, 213–224.
22. Gosling S. D., Mason W., (2015). Internet research in psychology. Annual Review of Psychology, 66, 877–902.
23. Hamerman E. J., Johar G. V., (2013). Conditioned superstition: Desire for control and consumer brand preferences. Journal of Consumer Research, 40, 428–443.
24. Johnson P. S., Herrmann E. S., Johnson M. W., (2015). Opportunity costs of reward delays and the discounting of hypothetical money and cigarettes. Journal of the Experimental Analysis of Behavior, 103, 87–107.
25. Jones M. S., House L. A., Gao Z., (2015). Respondent screening and revealed preference axioms: Testing quarantining methods for enhanced data quality in Web panel surveys. Public Opinion Quarterly, 79, 687–709.
26. Konstam V., Tomek S., Celen-Demirtas S., Sweeney K., (2015). Volunteering and reemployment status in unemployed emerging adults a time-worthy investment? Journal of Career Assessment, 23, 152–165.
27. Litman L., Robinson J., Abberbock T., (2016). TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods. doi:10.3758/s13428-016-0727-z.
28. Lynn B. M. D., (2014). Shared sense of purpose and well-being among veterans and non-veterans (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 3690322)
29. Mason W., Watts D. J., (2010). Financial incentives and the performance of crowds. ACM SigKDD Explorations Newsletter, 11, 100–108.
30. Miller J., (2006). Research reveals alarming incidence of ‘undesirable’ online panelists (Research Conference Report synthesized from Presentation at Research Industry Summit: Improving Respondent Cooperation, Chicago, IL). Skokie, IL: RFL Communications. Retrieved from http://www.burke.com/Library/Articles/Jeff%20Miller%20RCR%20PDF.pdf
31. Paolacci G., Chandler J., (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184–188.
32. Papa A., Lancaster N. G., Kahler J., (2014). Commonalities in grief responding across bereavement and non-bereavement losses. Journal of Affective Disorders, 161, 136–143.
33. Parent J., McKee L. G., Rough J. N., Forehand R., (2016). The association of parent mindfulness with parenting and youth psychopathology across three developmental stages. Journal of Abnormal Child Psychology, 44, 191–202.
34. Pearl R. L., Puhl R. M., Dovidio J. F., (2015). Differential effects of weight bias experiences and internalization on exercise among women with overweight and obesity. Journal of Health Psychology, 20, 1626–1632.
35. Peer E., Paolacci G., Chandler J., Mueller P., (2012). Selectively recruiting participants from Amazon Mechanical Turk using Qualtrics. Retrieved from SSRN 21006.
36. Peer E., Vosgerau J., Acquisti A., (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46, 1023–1031.
37. Rand D. G., Peysakhovich A., Kraft-Todd G. T., Newman G. E., Wurzbacher O., Nowak M. A., Greene J. D., (2014). Social heuristics shape intuitive cooperation. Nature communications, 5, Article 3677.
38. Reece A. G., Danforth C. M., (2016, 8 10). Instagram photos reveal predictive markers of depression. Retrieved March 3, 2017, from the arXiv database. arXiv:1608.03282v2 [cs.SI].
39. Schleider J. L., Weisz J. R., (2015). Using Mechanical Turk to study family processes and youth mental health: A test of feasibility. Journal of Child and Family Studies, 24, 3235–3246.
40. Siegel J. T., Navarro M. A., Thomson A. L., (2015). The impact of overtly listing eligibility requirements on MTurk: An investigation involving organ donation, recruitment scripts, and feelings of elevation. Social Science & Medicine, 142, 256–260.
41. Springer V., Martini P., Lindsey S., Vezich I., (2016). Practice-based considerations for using multi-stage survey design to reach special populations on Amazon’s Mechanical Turk. Survey Practice, 9(5), 1–8.
42. Stewart N., Ungemach C., Harris A. J., Bartels D. M., Newell B. R., Paolacci G., Chandler J., (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10, 479–491.
43. Suri S., Goldstein D. G., Mason W. A., (2011). Honesty in an online labor market. Menlo Park, CA: Association for the advancement of Artificial Intelligence.
44. Taylor S. E., Lichtman R. R., Wood J. V., (1984). Attributions, beliefs about control, and adjustment to breast cancer. Journal of Personality and Social Psychology, 46, 489–502.
45. Wessling K. S., Huber J., Netzer O., (in press). Character misrepresentation by Amazon Turk workers: Assessment and solutions. Journal of Consumer Research.
46. Yang K., Friedman-Wheeler D. G., Pronin E., (2014). Thought acceleration boosts positive mood among individuals with minimal to moderate depressive symptoms. Cognitive Therapy and Research, 38, 261–269.
47. Zhou H., Fishbach A., (2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology. Advance online publication. doi:10.1037/pspa0000056
48. Zou C., Andersen J. P., Blosnich J. R., (2013). The association between bullying and physical health among gay, lesbian, and bisexual individuals. Journal of the American Psychiatric Nurses Association, 19, 356–365.