How far does stereotype threat reach? The potential detriment of face validity in cognitive ability testing

Human Performance

Volumn 24, Issue 1, 2010, Pages 1-28

  Grand, James A ^a   Ryan, Ann Marie ^a   Schmitt, Neal ^a   Hmurovic, Jillian ^a  

^a MICHIGAN STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 78951472284     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1080/08959285.2010.518184     Document Type: Article

References (64)

1. American Educational Research Association. (1999). Standards for educational and psychological testing (3rd ed.). Washington, DC: American Psychological Association.
2. Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.
3. Arbuckle, J. L. (2003). Amos 5. 0. 1 [computer software]. Chicago: Small Waters Corp.
4. Arvey, R. D., Strickland, W., Drauden, G., & Martin, C. (1990). Motivational components of test-taking. Personnel Psychology, 43, 695-716.
5. Baggozzi, R. P., & Edwards, J. R. (1998). A general approach for representing constructs in organizational research. Organizational Research Methods, 1, 45-87.
6. Baggozzi, R. P., & Heatherton, T. F. (1994). A general approach to representing multifaceted personality constructs: Application to state self-esteem. Structural Equation Modeling, 1, 35-67.
7. Bennett, G. K. (2006). Bennett mechanical comprehension test, Form S. San Antonio, TX: Harcourt Assessment Incorporated.
8. Bennett, G., & Cruikshank, R. (1942). Sex differences in the understanding of mechanical problems. Journal of Applied Psychology, 26, 121-127.
9. Bornstein, R. F. (1996). Face validity in psychological assessment: Implications for a unified model of validity. American Psychologist, 51, 983-984.
10. Bornstein, R. F., Rossner, S. C., Hill, E. L., & Stepanian, M. L. (1994). Face validity and fakability of objective and projective measures of dependency. Journal of Personality Assessment, 63, 363-386.
11. Brutus, S., & Ryan, A. M. (1996). Individual characteristics as determinants of the perceived job relatedness of selection procedures. Unpublished manuscript.
12. Cascio, W. F. (1987). Applied psychology in personnel management (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
13. Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82(1), 143-159.
14. Chan, D., Schmitt, N., DeShon, R. P., Clause, C. S., & Delbridge, K. (1997). Reactions to cognitive ability tests: The relationships between race, performance, face validity perceptions, and test-taking motivation. Journal of Applied Psychology, 82, 300-310.
15. Davies, P. G., Spencer, S. J., Quinn, D. M., & Gerhardstein, R. (2002). Consuming images: How television commercials that elicit stereotype threat can restrain women academically and professionally. Personality and Social Psychology Bulletin, 28, 1615-1628.
16. Devine, P. G. (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56(1), 5-18.
17. Dwight, S. A., & Alliger, G. M. (1997). Reaction to overt integrity test items. Educational and Psychological Measurement, 57, 937-948.
18. Elkins, T. J., & Phillips, J. S. (2000). Job context, selection decision outcome, and perceived fairness of selection tests: Biodata as an illustrative case. Journal of Applied Psychology, 85, 479-484.
19. Feingold, A. (1988). Cognitive gender differences are disappearing. American Psychologist, 43, 95-103.
20. Gilliland, S. W. (1994). Effects of procedural and distributive justice on reactions to a selection system. Journal of Applied Psychology, 79, 691-701.
21. Halpern, D. F., Benbow, C. P., Geary, D. C., Gur, R. C., Hyde, J. S., & Gernsbacher, M. A. (2007). The science of sex differences in science and mathematics. Psychological Science in the Public Interest, 8(1), 1-51.
22. Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639-683.
23. Highhouse, S., Lievens, F., & Sinar, E. F. (2003). Measuring attraction to organizations. Educational and Psychological Measurement, 63, 986-1001.
24. Hmurovic, J., Ryan, A. M., Schmitt, N., & Grand, J. A. (2009, April). Sensitivity or stereotype threat? Effects of gendered test content. Poster presented at the meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.
25. Holtz, B. C., Ployhart, R. E., & Dominguez, A. (2005). Testing the rules of justice: The effects of frame-of-reference and pre-test validity information on personality test responses and test perceptions. International Journal of Selection and Assessment, 13(1), 75-86.
26. Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection, and amelioration of adverse impact in personnel selection procedures: Issues, evidence, and lessons learned. International Journal of Selection and Assessment, 9, 152-194.
27. Kray, L. J., Thompson, L., & Galinsky, A. (2001). Battle of the sexes: Gender stereotype confirmation and reactance in negotiations. Journal of Personality and Social Psychology, 80, 942-958.
28. Lepore, L., & Brown, R. (2000). Category and stereotype activation: Is prejudice inevitable? In C. Stangor (Ed.), Stereotypes and prejudice (pp. 119-137). Philadelphia: Psychology Press.
29. Levy, B. (1996). Improving memory in old age by implicit self-stereotyping. Journal of Personality & Social Psychology, 71, 1092-1107.
30. Levy, S. R., Stroessner, S. J., & Dweck, C. S. (1998). Stereotype formation and endorsement: The role of implicit theories. Journal of Personality and Social Psychology, 74, 1421-1436.
31. Lievens, F., De Corte, W., & Schollaert, E. (2008). A closer look at the frame-of-reference effect in personality scale scores and validity. Journal of Applied Psychology, 93, 268-279.
32. Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991). Complex, performance-based assessments: Expectations and validation criteria. Educational Researcher, 13, 15-31.
33. McFarland, L. A., Lev-Arey, D. M., Ziegert, J. C. (2003). An examination of stereotype threat in a motivational context. Human Performance, 16, 181-205.
34. McGrath, J. E. (1986). Dilematics: The study of research choices and dilemmas. In J. E. McGrath, J. Martin, & R. A. Kulka (Eds.), Judgment calls in research (pp. 69-101). Beverly Hills, CA: Sage.
35. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
36. Mosier, C. I. (1947). A critical examination of the concepts of face validity. Educational and Psychological Measurement, 7, 191-205.
37. Muchinsky, P. M. (2004). Mechanical aptitude and spatial ability testing. In J. C. Thomas (Ed.), Comprehensive handbook of psychological assessment: Vol. 4. Industrial and organizational assessment (pp. 21-33). Hoboken, NJ: Wiley.
38. Nevo, B. (1985). Face validity revisited. Journal of Educational Measurement, 22, 287-293.
39. Nguyen, H. -H., O'Neal, A., & Ryan, A. M. (2003). Relating test-taking attitudes and skills and stereotype threat effects to the racial gap in cognitive ability test performance. Human Performance, 16, 261-293.
40. Nguyen, H. -H., & Ryan, A. M. (2008). Does stereotype threat affect test performance of minorities and women? A meta-analysis of experimental evidence. Journal of Applied Psychology, 93, 1314-1334.
41. O*NET. (2004). Summary report for 49-9042. 00 - Maintenance and repair workers, general. Retrieved October 22, 2007, from http://online.onetcenter.org/link/summary/49-9042.00
42. Ployhart, R. E., Schneider, B., & Schmitt, N. (2006). Staffing organizations: Contemporary practice and theory (3rd ed.). Mahwah, NJ: Erlbaum.
43. Ployhart, R. E., Ziegert, J. C., & McFarland, L. A. (2003). Understanding racial differences on cognitive ability tests in selection contexts: An integration of stereotype threat and applicant reactions research. Human Performance, 16, 231-259.
44. Robie, C., Schmit, M. J., Ryan, A. M., & Zickar, M. J. (2000). Effects of item context specificity on the measurement equivalence of a personality inventory. Organizational Research Methods, 3, 348-365.
45. Sackett, P. R. (2003). Stereotype threat in applied selection settings: A commentary. Human Performance, 16, 295-309.
46. Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High stakes testing in employment, credentialing, and higher education. American Psychologist, 56, 302-318.
47. Sackett, P. R., Hardison, C. M., & Cullen, M. J. (2005). On interpreting research on stereotype threat and test performance. American Psychologist, 60, 271-272.
48. Seymour, R. T. (1988). Why plaintiffs' counsel challenge tests, and how they can successfully challenge the theory of "Validity Generalization. "Journal of Vocational Behavior, 33, 331-364.
49. Shotland, A., Alliger, G. M., & Sales, T. (1998). Face validity in the context of personnel selection: A multimedia approach. International Journal of Selection and Assessment, 6, 124-130.
50. Smither, J. W., Reilly, R. R., Millsap, R. E., Pearlman, K., & Stoffey, R. W. (1993). Applicant reactions to selection procedures. Personnel Psychology, 46(1), 49-76.
51. Spence, J. T., Helmreich, R., & Stapp, J. (1975). Ratings of self and peers on sex role attributes and their relation to self-esteem and conceptions of masculinity and femininity. Journal of Personality and Social Psychology, 32(1), 29-39.
52. Spencer, S. J., Steele, C. M., & Quinn, D. M. (1999). Stereotype threat and women's math performance. Journal of Experimental Social Psychology, 35(1), 4-28.
53. Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629.
54. Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69, 797-811.
55. Steele, C. M., & Davies, P. G. (2003). Stereotype threat and employment testing: A commentary. Human Performance, 16, 311-326.
56. Steele, C. M., Spencer, S. J., & Aronson, J. (2002). Contending with group image: The psychology of stereotype and social identity threat. Advances in Experimental Social Psychology, 34, 379-400.
57. Tett, R. P, Anderson, M. G., Ho, C., Yang, T. S., Huang, L., & Hanvongse, A. (2006). Seven nested questions about faking on personality tests. In R. Griffith & M. Peterson (Eds.), A closer examination of applicant faking behavior (pp. 43-83). Greenwich CT: Information Age.
58. Vandenberg, R. J. (2002). Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods, 5, 139-158.
59. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70.
60. Walton, G. M., & Cohen, G. L. (2003). Stereotype lift. Journal of Experimental Social Psychology, 39, 456-467.
61. Whitney, D. J., Diaz, J., Mineghino, M. E., & Powers, K. (1999). Perceptions of overt personality-based integrity tests. International Journal of Selection and Assessment, 7(1), 35-45.
62. Wicherts, J. M., Dolan, C. V., & Hessen, D. J. (2005). Stereotype threat and group differences in test performance: A question of measurement invariance. Journal of Personality and Social Psychology, 89, 696-716.
63. Wiechmann, D., & Ryan, A. M. (2003). Reactions to computerized testing in selection contexts. International Journal of Selection and Assessment, 11, 215-229.
64. Wiggins, J. S. (1973). Personality and prediction: Principles of personality assessment. Reading, MA: Addison-Wesley.