Seriousness checks are useful to improve data validity in online research

Behavior Research Methods

Volumn 45, Issue 2, 2013, Pages 527-535

  Aust, Frederik ^a   Diedenhofen, Birk ^a   Ullrich, Sebastian ^a   Musch, Jochen ^a  

^a HEINRICH HEINE UNIVERSITY   (Germany)

Author keywords

Incremental validity; Methodology; Online research; Participant screening; Seriousness; Voting survey

Indexed keywords

ADOLESCENT; ADULT; AGE; ARTICLE; ATTITUDE; EDUCATIONAL STATUS; FEMALE; GERMANY; HEALTH SURVEY; HUMAN; INFORMATION RETRIEVAL; MALE; METHODOLOGY; ONLINE SYSTEM; PATIENT SELECTION; POLITICS; QUESTIONNAIRE; REPRODUCIBILITY; SELF REPORT; STANDARD; STATISTICAL ANALYSIS; VALIDATION STUDY;

ADOLESCENT; ADULT; AGE FACTORS; ATTITUDE; DATA INTERPRETATION, STATISTICAL; EDUCATIONAL STATUS; FEMALE; GERMANY; HUMANS; INFORMATION STORAGE AND RETRIEVAL; MALE; ONLINE SYSTEMS; PATIENT SELECTION; POLITICS; POPULATION SURVEILLANCE; QUESTIONNAIRES; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SELF REPORT; YOUNG ADULT;

EID: 84878249870     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-012-0265-2     Document Type: Article

References (39)

1. Buchanan, T., Heffernan, T. M., Parrott, A. C., Ling, J., Rodgers, J., & Scholey, A. B. (2010). A short self-report measure of problems with executive function suitable for administration via the internet. Behavior Research Methods, 42, 709-714.
2. Buhrmester, M. D., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5.
3. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Mahwah: Erlbaum.
4. Couper, M. P. (2000). Web-based surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494.
5. Davis, R. N. (1999). Web-based administration of a personality questionnaire: Comparison with traditional methods. Behavior Research Methods, Instruments, & Computers, 31, 572-577.
6. Egermann, H., Nagel, F., Altenmüller, E., & Kopiez, R. (2009). Continuous measurement of musically-induced emotion: A web experiment. International Journal of Internet Science, 4, 4-20.
7. Federal Returning Officer (2009). Wahl zum 17. Deutschen Bundestag am 27. September 2009. Heft 4: Wahlbeteiligung und Stimmabgabe der Männer und Frauen nach Altersgruppen [Elections for the 17th German Bundestag on September 27, 2009. Volume 4: Electoral participation of and votes cast by men and women by age groups].
8. Fricker, R. D., & Schonlau, M. (2002). Advantages and disadvantages of internet research surveys: Evidence from the literature. Field Methods, 14, 347-367.
9. Ihme, J. M., Lemke, F., Lieder, K., Martin, F., Müller, J. C., & Schmidt, S. (2009). Comparison of ability tests administered online and in the laboratory. Behavior Research Methods, 41, 1183-1189.
10. Joinson, A. (1999). Social desirability, anonymity, and internet-based questionnaires. Behavior Research Methods, Instruments, & Computers, 31, 433-438.
11. Kalton, G., & Flores-Cervantes, I. (2003). Weighting methods. Journal of Official Statistics, 19, 81-97.
12. Keller, F., Gunasekharan, S., Mayo, N., & Corley, M. (2009). Timing accuracy of web experiments: A case study using the WebExp software package. Behavior Research Methods, 41, 1-12.
13. King, W. C., & Miles, E. W. (1995). A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: A test of measurement equivalence. Journal of Applied Psychology, 80, 643-651.
14. Klauer, K. C., Musch, J., & Naumer, B. (2000). On belief bias in syllogistic reasoning. Psychological Review, 107, 852-884.
15. Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research (pp. 35-60). San Diego, CA: Academic Press.
16. Lahl, O., Göritz, A. S., Pietrowsky, R., & Rosenberg, J. (2009). Using the World-Wide web to obtain large-scale word norms: 190, 212 ratings on a set of 2, 654 German nouns. Behavior Research Methods, 41, 13-19.
17. Li, M. F., & Olejnik, S. (1997). The power of rasch person-fit statistics in detecting unusual response patterns. Applied Psychological Measurement, 21, 215-231.
18. Madsen, H. S. (1987). Utilizing Rasch analysis to detect cheating on language examinations. Retrieved from ERIC database. (ED287284).
19. Malhotra, N. (2008). Completion time and response order effects in web surveys. Public Opinion Quarterly, 72, 914-934.
20. Mangan, M., & Reips, U.-D. (2007). Sleep, sex, and the web: Surveying the difficult-to-reach clinical population suffering from sexsomnia. Behavior Research Methods, 39, 233-236.
21. Musch, J., & Klauer, K. C. (2002). Psychological Experimenting on the World Wide Web: Investigating Content Effects in Syllogistic Reasoning (pp. 181-212). Seattle, WA: Hogrefe & Huber.
22. Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45, 867-872.
23. Pappi, F. (2009). Regierungsbildung im deutschen Fünf-Parteiensystem [Government formation in the German five party system]. Politische Vierteljahresschrift, 50, 187-202.
24. Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510-532.
25. Reips, U.-D. (2000). The Web Experiment Method: Advantages, Disadvantages, and Solutions (pp. 89-117). San Diego: Academic Press.
26. Reips, U.-D. (2002). Standards for internet-based experimenting. Experimental Psychology, 49, 243-256.
27. Reips, U.-D. (2008). How Internet-mediated research changes science (pp. 268-294). Cambridge, MA: Cambridge University Press.
28. Reips, U.-D. (2009). Internet experiments: Methods, guidelines, metadata. Human Vision and Electronic Imaging XIV, Proceedings of SPIE, 7240, 724008.
29. Reips, U.-D. (2011). Privacy and the disclosure of information on the Internet: Issues and measurement (pp. 71-104). Warsaw: UKSW Publishing House.
30. Reips, U.-D., & Buffardi, L. (2012). Studying migrants with the help of the internet: Methods from psychology. Journal of Ethnic and Migration Studies, 38, 1405-1424.
31. Reips, U.-D., & Neuhaus, C. (2002). WEXTOR: A web-based tool for generating and visualizing experimental designs and procedures. Behavior Research Methods, 34, 234-240.
32. Reise, S. P., & Due, A. M. (1991). The influence of test characteristics on the detection of aberrant response patterns. Applied Psychological Measurement, 15, 217-226.
33. Schmidt, W. C. (1997). World-wide web survey research: Benefits, potential problems, and solutions. Behavior Research Methods, Instruments, & Computers, 29, 274-279.
34. Schmitt, H., Sanz, A., & Braun, D. (2009). Motive individuellen Wahlverhaltens in Nebenwahlen: Eine theoretische Rekonstruktion und empirische Überprüfung [Motives of individual voting behavior in Nebenwahlen: A theoretical reconstruction and empirical revision] (pp. 585-605). Wiesbaden, Germany: Verlag für Sozialwissenschaften.
35. Schmitt, N., Cortina, J. M., & Whitney, D. J. (1993). Appropriateness fit and criterion-related validity. Applied Psychological Measurement, 17, 143-150.
36. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 20, 1-8.
37. Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867-873.
38. van den Wittenboer, G., Hox, J., & de Leeuw, E. (1997). Aberrant Response Patterns in Elderly Respondents: Latent Class Analysis of Respondent Scalability (pp. 155-162). Münster, Germany: Waxman.
39. Whitty, M. T., & Buchanan, T. (2010). What's in a screen name? attractiveness of different types of screen names used by online daters. International Journal of Internet Science, 5, 5-19.