An examination of factors contributing to a reduction in subgroup differences on a constructed-response paper-and-pencil test of scholastic achievement

Journal of Applied Psychology

Volumn 92, Issue 3, 2007, Pages 794-801

  Edwards, Bryan D ^a   Arthur Jr , Winfred ^b  

^a AUBURN UNIVERSITY   (United States)

^b TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Adverse impact; African Americans; Constructed response tests; Multiple choice tests; Scholastic achievement; Subgroup differences; Test perceptions; Whites

Indexed keywords

ACHIEVEMENT; ADULT; ARTICLE; EDUCATIONAL STATUS; FEMALE; HUMAN; MALE; METHODOLOGY; MOTIVATION; PSYCHOLOGY; QUESTIONNAIRE; STATISTICS;

ACHIEVEMENT; ADULT; EDUCATIONAL STATUS; FEMALE; HUMANS; MALE; MOTIVATION; PSYCHOLOGY; QUESTIONNAIRES;

EID: 34248325737     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.92.3.794     Document Type: Article

References (38)

1. th ed.). New York: Worth Publishers.
2. Arthur, W., Jr., Bennett, W., Jr., Stanush, P. L., & McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11, 57-101.
3. Arthur, W., Jr., & Doverspike, D. (2005). Achieving diversity and reducing discrimination in the workplace through human resource management practices: Implications of research and theory for staffing, training, and rewarding performance. In R. L. Dipboye & A. Colella (Eds.), Discrimination at work: The psychological and organizational bases (pp. 305-327). Mahwah, NJ: Erlbaum.
4. Arthur, W., Jr., Edwards, B. D., & Barrett, G. V. (2002). Multiple-choice and constructed-response tests of ability: Race-based subgroup performance differences on alternative paper-and-pencil test formats. Personnel Psychology, 55, 985-1008.
5. Arvey, R. D., Strickland, W., Drauden, G., & Martin, C. (1990). Motivational components of test taking. Personnel Psychology, 43, 695-716.
6. Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1983). Effects of coaching programs on achievement test performance. Review of Educational Research, 53, 571-585.
7. Bauer, T. N., Truxillo, D. M., Sanchez, R. J., Craig, J., Ferrara, P., & Campion, M. A. (2001). Applicant reactions to selection: Development of the Selection Procedural Justice Scale (SPJS). Personnel Psychology, 54, 387-419.
8. Bell, B. S., Ryan, A. M., & Wiechmann, D. (2004). Justice expectations and applicant perceptions. International Journal of Selection and Assessment, 12, 24-38.
9. Bell, B. S., Wiechmann, D., & Ryan, A. M. (2006). Consequences of organizational justice expectations in a selection system. Journal of Applied Psychology, 91, 455-466.
10. Bennett, R. E. (1993). On the meanings of constructed-response. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement: Issues in constructed-response, performance testing, and portfolio assessment (pp. 1-27). Hillsdale, NJ: Erlbaum.
11. Bennett, R. E., Rock, D. A., & Wang, M. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28, 77-92.
12. Chan, D. (1997). Racial subgroup differences in predictive validity perceptions on personality and cognitive ability tests. Journal of Applied Psychology, 82, 311-320.
13. Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
14. Chan, D., Schmitt, N., DeShon, R. P., Clause, C. S., & Delbridge, K. (1997). Reactions to cognitive ability tests: The relationship between race, test performance, face validity perceptions, and test-taking motivation. Journal of Applied Psychology, 82, 300-310.
15. Chan, D., Schmitt, N., Sacco, J. M., & DeShon, R. P. (1998). Understanding pretest and posttest reactions to cognitive ability and personality tests. Journal of Applied Psychology, 83, 471-485.
16. Dovidio, J. F., & Gaertner, S. L. (1999). Reducing prejudice: Combating intergroup biases. Current Directions in Psychological Science, 8, 101-105.
17. Educational Testing Service. (2005a). The college board: The new SAT. Retrieved March 13, 2005, from http://www.collegeboard.com/student/ testing/sat/about/sat/writing.html
18. Educational Testing Service. (2005b). Graduate record examinations: The general test. Retrieved March 13, 2005, from http://www.gre.org/stuwrit. html
19. Gilliland, S. W. (1993). The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review, 18, 694-734.
20. Gilliland, S. W. (1994). Effects of procedural and distributive justice on reactions to a selection system. Journal of Applied Psychology, 79, 691-701.
21. Graduate Management Admission Council. (2005). Analytical writing assessment. Retrieved March 13, 2005, from http://www.gmac.com/gmac/TheGMAT/ WhatIstheGMAT/WhattheGMATMeasures.htm
22. Helms, J. E. (1992). Why is there no study of cultural equivalence in standardized cognitive ability testing? American Psychologist, 47, 1083-1101.
23. Huffcutt, A. I., & Roth, P. L. (1998). Racial group differences in employment interview evaluations. Journal of Applied Psychology, 83, 179-189.
24. Johns, M., Schmader, T., & Martens, A. (2005). Knowing is half the battle: Teaching stereotype threat as a means of improving women's math performance. Psychological Science, 16, 175-179.
25. Kuncel, N. R., Crede, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A metaanalysis and review of the literature. Review of Educational Research, 75, 63-82.
26. Oswald, F. L., Schmitt, N., Kirn, B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology, 89, 187-207.
27. Pennock-Román, M. (1999). English proficiency and differences among racial and ethnic groups in mean SAT and GRE scores: A longitudinal analysis (GRE Board Professionals Rep. No. 86-09cP, ETS Research Rep. 99-17). Princeton, NJ: Educational Testing Service.
28. Preacher, K. J., & Hayes, A. F. (2006). Asymptotic and resampling strategies for assessing and comparing indirect effects in simple and multiple mediator models. Retrieved February 14, 2006, from http://www.comm.ohio- state.edu/ahayes/indirect2.pdf
29. Ryan, A. M. (2001). Explaining the Black-White test score gap: The role of test perceptions. Human Performance, 14, 45-75.
30. Ryan, A. M., & Ployhart, R. E. (2000). Applicants' perceptions of selection procedures and decisions: A critical review and agenda for the future. Journal of Management, 26, 565-606.
31. Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist, 56, 302-318.
32. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-273.
33. Schmitt, N., & Mills, A. E. (2001). Traditional tests and job simulations: Minority and majority performance and test validities. Journal of Applied Psychology, 86, 451-458.
34. Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and non-experimental studies: New procedures and recommendations. Psychological Methods, 7, 422-445.
35. Smither, J. W., Reilly, R. R., Millsap, R. E., Pearlman, K., & Stoffey, R. W. (1993). Applicant reactions to selection procedures. Personnel Psychology, 46, 49-76.
36. Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69, 797-811.
37. Steele, C. M., Spencer, S. J., & Aronson, J. (2002). Contending with group image: The psychology of stereotype threat and social identity threat. Advances in Experimental Social Psychology, 34, 379-440.
38. Twenge, J., & Crocker, J. (2002). Race and self-esteem: Meta-analyses comparing Whites, Blacks, Hispanics, Asians, and American Indians and comment on Gray-Little and Hafdahl (2000). Psychological Bulletin, 128, 371-408.