Identifying careless responses in survey data

Psychological Methods

Volumn 17, Issue 3, 2012, Pages 437-455

  Meade, Adam W ^a   Craig, S Bartholomew ^a  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Careless response; Data cleaning; Data screening; Mixture modeling; Random response

Indexed keywords

ARTICLE; HUMAN; INFORMATION PROCESSING; INTERNET; METHODOLOGY; QUESTIONNAIRE; REACTION TIME; SELF REPORT; STATISTICAL MODEL;

DATA COLLECTION; HUMANS; INTERNET; LOGISTIC MODELS; QUESTIONNAIRES; REACTION TIME; RESEARCH DESIGN; SELF REPORT;

EID: 84870324292     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0028085     Document Type: Article

References (49)

1. Baer, R. A., Ballenger, J., Berry, D. T. R., & Wetter, M. W. (1997). Detection of random responding on the MMPI-A. Journal of Personality Assessment, 68, 139-151. doi:10.1207/s15327752jpa6801_11.
2. Beach, D. A. (1989). Identifying the random responder. Journal of Psychology: Interdisciplinary and Applied, 123, 101-103.
3. Ben-Porath, Y. S., & Tellegen, A. (2008). The Minnesota Multiphasic Personality Inventory-2 Restructured Form: Manual for administration, scoring, and interpretation. Minneapolis, MN: University of Minnesota Press.
4. Berry, D. T., Baer, R. A., & Harris, M. J. (1991). Detection of malingering on the MMPI: A meta-analysis. Clinical Psychology Review, 11, 585-598. doi:10.1016/0272-7358(91)90005-F.
5. Berry, D. T. R., Wetter, M. W., Baer, R. A., Larsen, L., Clark, C., & Monroe, K. (1992). MMPI-2 random responding indices: Validation using a self-report methodology. Psychological Assessment, 4, 340-345. doi:10.1037/1040-3590.4.3.340.
6. Berry, D. T. R., Wetter, M. W., Baer, R. A., Widiger, T. A., Sumpter, J. C., Reynolds, S. K., & Hallam, R. A. (1991). Detection of random responding on the MMPI-2: Utility of F, back F, and VRIN scales. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3, 418-423. doi:10.1037/1040-3590.3.3.418.
7. Buchanan, T. (2000). Potential of the Internet for personality research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 121-140). San Diego, CA: Academic Press.
8. Campbell, M. J., & Waters, W. E. (1990). Does anonymity increase response rate in postal questionnaire surveys about sensitive subjects? A randomised trial. Journal of Epidemiology and Community Health, 44, 75-76. doi:10.1136/jech.44.1.75.
9. Carrier, L. M., Cheever, N. A., Rosen, L. D., Benitez, S., & Chang, J. (2009). Multitasking across generations: Multitasking choices and difficulty ratings in three generations of Americans. Computers in Human Behavior, 25, 483-489. doi:10.1016/j.chb.2008.10.012.
10. Cattell, R. B., Cattell, A. K., & Cattell, H. E. P. (Eds.). (1993). 16PF Questionnaire (5th ed.). Champaign, IL: Institute for Personality and Ability Testing.
11. Clark, M. E., Gironda, R. J., & Young, R. W. (2003). Detection of back random responding: Effectiveness of MMPI-2 and personality assessment inventory validity indices. Psychological Assessment, 15, 223-234. doi:10.1037/1040-3590.15.2.223.
12. Clark, S. L., Muthén, B., Kaprio, J., D'Onofrio, B. M., Viken, R., Rose, R. J., & Smalley, S. L. (2009). Models and strategies for factor mixture analysis: Two examples concerning the structure underlying psychological disorders. Manuscript submitted for publication. Retrieved from http://www.statmodel.com/papers.shtml.
13. Costa, P. T., Jr., & McCrae, R. R. (2008). The Revised NEO Personality Inventory (NEO-PI-R). In D. H. Saklofske (Ed.), The SAGE handbook of personality theory and assessment. Vol. 2: Personality measurement and testing (pp. 179-198). Thousand Oaks, CA: Sage.
14. Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354. doi:10.1037/h0047358.
15. Curran, P. G., Kotrba, L., & Denison, D. (2010). Careless responding in surveys: Applying traditional techniques to organizational settings. Paper presented at the 25th annual conference of the Society for Industrial/Organizational Psychology, Atlanta, GA.
16. Dillman, D. A., Smyth, J. D., Christian, L. M., & Dillman, D. A. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken, NJ: Wiley.
17. Douglas, K. M., & McGarty, C. (2001). Identifiability and self presentation: Computer-mediated communication and intergroup interaction. British Journal of Social Psychology, 40, 399-416. doi:10.1348/014466601164894.
18. Ehlers, C., Greene-Shortridge, T. M., Weekley, J. A., & Zajack, M. D. (2009). The exploration of statistical methods in detecting random responding. Paper presented at the annual meeting of the Society for Industrial/Organizational Psychology, Atlanta, GA.
19. Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. D. Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe (pp. 7-28). Tilburg, the Netherlands: Tilburg University Press.
20. Gordon, M. E., Slade, L. A., & Schmitt, N. (1986). The science of the sophomore revisited: From conjecture to empiricism. Academy of Management Review, 11, 191-207.
21. Gough, H. G., & Bradley, P. (1996). California Psychological Inventory: Administrator's guide (3rd ed.). Palo Alto, CA: Consulting Psychologists Press.
22. Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics. Vol. 3: Speech acts (pp. 41-58). New York, NY: Academic Press.
23. Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104-121. doi:10.1177/109442819800100106.
24. Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27, 99-114.
25. Johnson, J. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39, 103-129. doi:10.1016/j.jrp.2004.09.009.
26. Kurtz, J. E., & Parrish, C. L. (2001). Semantic response consistency and protocol validity in structured personality assessment: The case of the NEO-PI-R. Journal of Personality Assessment, 76, 315-332. doi: 10.1207/S15327752JPA7602_12.
27. Lee, H. (2006). Privacy, publicity, and accountability of self-presentation in an on-line discussion group. Sociological Inquiry, 76, 1-22. doi: 10.1111/j.1475-682X.2006.00142.x.
28. Levenson, M. R., Kiehl, K. A., & Fitzpatrick, C. M. (1995). Assessing psychopathic attributes in a noninstitutionalized population. Journal of Personality and Social Psychology, 68, 151-158. doi:10.1037/0022-3514.68.1.151.
29. Lubke, G. H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21-39. doi: 10.1037/1082-989X.10.1.21.
30. Montgomery, K. C. (2007). Generation digital: Politics, commerce, and childhood in the age of the Internet. Cambridge, MA: MIT Press.
31. Moore, R. S., & Ames, G. M. (2002). Survey confidentiality vs. anonymity: Young men's self-reported substance use. Journal of Alcohol and Drug Education, 47, 32-41.
32. Nichols, D. S., Greene, R. L., & Schmolck, P. (1989). Criteria for assessing inconsistent patterns of item endorsement on the MMPI: Rationale, development, and empirical trials. Journal of Clinical Psychology, 45, 239-250. doi:10.1002/1097-4679(198903)45: 2<239::AID-JCLP2270450210>3.0.CO;2-1.
33. Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14, 535-569.
34. Ong, A. D., & Weiss, D. J. (2000). The impact of anonymity of responses to sensitive questions. Journal of Applied Social Psychology, 30, 1691-1708. doi:10.1111/j.1559-1816.2000.tb02462.x.
35. Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 598-609. doi:10.1037/0022-3514.46.3.598.
36. Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In H. I. Braun, D. N. Jackson, D. E. Wiley, H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 49-69). Mahwah, NJ: Erlbaum.
37. Raskin, R., & Terry, H. (1988). A principal-components analysis of the narcissistic personality inventory and further evidence of its construct validity. Journal of Personality and Social Psychology, 54, 890-902. doi:10.1037/0022-3514.54.5.890.
38. Rogers, R., Sewell, K. W., Martin, M. A., & Vitacco, M. J. (2003). Detection of feigned mental disorders: A meta-analysis of the MMPI-2 and malingering. Assessment, 10, 160-177. doi:10.1177/1073191103010002007.
39. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, 34(Suppl.), 100-114.
40. Schinka, J. A., Kinder, B. N., & Kremer, T. (1997). Research validity scales for the NEO-PI-R: Development and initial validation. Journal of Personality Assessment, 68, 127-138. doi:10.1207/s15327752jpa6801_10.
41. Schultz, D. P. (1969). The human subject in psychological research. Psychological Bulletin, 72, 214-228. doi:10.1037/h0027880.
42. Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93-105. doi:10.1037/0003-066X.54.2.93.
43. Spelke, E., Hirst, W., & Neisser, U. (1976). Skills of divided attention. Cognition, 4, 215-230. doi:10.1016/0010-0277(76)90018-4.
44. Stark, S., Chernyshenko, O. S., Drasgow, F., & Williams, B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring? Journal of Applied Psychology, 91, 25-39. doi:10.1037/0021-9010.91.1.25.
45. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Pearson/Allyn & Bacon.
46. Thissen, D. (1991). MULTILOG user's guide: Multiple categorical item analysis and test scoring using item response theory. Chicago, IL: Scientific Software International.
47. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge, England: Cambridge University Press.
48. Wallis, C. (2006). genM: The multitasking generation. TIME Magazine. Retrieved from http://www.time.com/time/magazine/article/0,9171,1174696,00.html.
49. Woods, C. M. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28, 186-191. doi:10.1007/s10862-005-9004-7.