Reductions in measured subgroup mean differences: What is possible?

Adverse Impact: Implications for Organizational Staffing and High Stakes Selection

Volumn , Issue , 2009, Pages 425-452

  Schmitt, Neal ^a,b   Quinn, Abigail ^a  

^a MICHIGAN STATE UNIVERSITY   (United States)

^b AMERICAN PSYCHOLOGICAL ASSOCIATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84887916867     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203848418     Document Type: Chapter

References (66)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
2. Bangert-Drowns , R. L. , Kulik , J. A. , & Kulik , C. C. (1983). Effects of coaching programs on achievement test performance. Review of Educational Research , 53, 571-585.
3. Barrick , M. R. , & Mount , M. K. (1991). The Big 5 personality dimensions and job performance. Personnel Psychology , 44, 1-25.
4. Bobko , P. , Roth , P. L. , & Potosky , D. (1999). Derivation and implications of a meta-analytic matrix incorporating cognitive ability, alternative predictors, and job performance. Personnel Psychology , 52, 561-590.
5. Borman , W. C. , & Motowidlo , S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 71-98). San Francisco: Jossey-Bass.
6. Borman , W. C. , & Motowidlo , S. J. (1997). Task performance and contextual performance: The meaning for personnel selection research. Human Performance , 10, 99-110.
7. Bridgeman , B. , Cline , F. , & Hessinger , J. (2004). Effect of extra time on verbal and quantitative GRE scores. Applied Measurement in Education , 17, 25-37.
8. Campbell , J. P. , McCloy , R. A. , Oppler , S. H. , & Sager , C. E. (1993). A theory of performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 35-70). San Francisco: Jossey-Bass.
9. Chan , D. (1997). Racial subgroup differences in predictive validity perceptions on personality and cognitive ability tests. Journal of Applied Psychology , 82, 311-320.
10. Chan , D. , & Schmitt , N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology , 82, 143-159.
11. Chan , D. , Schmitt , N. , DeShon , R. P. , Clause , C. , & Delbridge , K. (1997). Reactions to cognitive ability tests: The relationships between race, test performance, face validity perceptions, and test-taking motivation. Journal of Applied Psychology , 82, 302-310.
12. Cullen , M. J. , Hardison , C. M. , & Sackett , P. R. (2004). Using SAT-grade and ability-job performance relationships to test predictions from stereotype threat theory. Journal of Applied Psychology , 89, 220-230.
13. Cullen , M. J. , Waters , S. D. , & Sackett , P. R. (2006). Testing stereotype threat theory predictions for math-identified and non-math-identified students by gender. Human Performance , 19I, 421-440.
14. DeCorte , W. (1999). Weighing job performance predictors to both maximize the quality of the selected workforce and control the level of adverse impact. Journal of Applied Psychology , 84, 695-702.
15. DeShon , R. P. , Smith , M. R. , Chan , D. , & Schmitt , N. (1998). Can racial differences in cognitive test performance be reduced by presenting problems in a social context? Journal of Applied Psychology , 8, 438-451.
16. Dickens , W. T. , & Flynn , J. R. (2006). Black Americans reduce the racial IQ gap. Psychological Science , 17, 913-920.
17. Donlon , T. F. , Hicks , M. M. , & Wallmark , M. M. (1980). Sex differences in item responses on the Graduate Record Examination. Applied Psychological Measurement , 4 (1), 9-20.
18. Doudin , H. , & Al-Darabee , M. (2003). Gender-related differential item functioning on mathematics performance. Dirasat, Educational Sciences , 30 (2), 414-419.
19. Drasgow , F. (1987). Study of the measurement bias of two standardized psychological tests. Journal of Applied Psychology , 72, 19-29.
20. Dubois , C. L. Z. , Sackett , P. R. , Zedeck , S. , & Fogli , L. (1993). Further exploration of typical and maximum performance criteria: Definitional issues, prediction, and white-black differences. Journal of Applied Psychology , 78, 205-211.
21. Evans , F. R. , & Reilly , R. R. (1972). A study of speededness as a source of test bias. Journal of Educational Measurement , 9, 123-131.
22. Frierson , H. T. (1986). Enhancing minority college students' performance on educational tests. Journal of Negro Education , 55, 38-45.
23. Gallagher , A. , Bridgeman , B. , & Calahan , C. (2002). The effect of computer-based tests on racial-ethnic and gender groups. Journal of Educational Measurement , 39, 133-147.
24. Garner , M. , & Engelhard , G. (1999). Gender differences in performance on multiple-choice and constructed response mathematical items. Applied Measurement in Education , 12 (1), 29-51.
25. Guion , R. M. (1998). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Erlbaum.
26. Hattrup , K. , Rock , J. , & Scalia , C. (1997). The effects of varying conceptualizations of job performance on adverse impact, minority hiring, and predicted performance. Journal of Applied Psychology , 82, 656-664.
27. Hausknecht , J. P. , Halpert , J. A. , Di Paolo , N. T. , & Gerrard , M. O. M. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology , 92, 373-385.
28. Huselid , M. A. (1995). The impact of human resource management practices on turnover, productivity, and corporate financial performance. Academy of Management Journal , 38, 635-672.
29. Kim , B. H. , Schmitt , N ., Imus , A. L ., Oswald , F. L ., Drzakowski , S. M. , Friede , A. J . et al. Interpreting differential item functioning in a situational judgment test: A matter of differential access to opportunities? Unpublished manuscript.
30. Klein , S. P . (1983). An analysis of the relationship between trial practice skills and bar examination results. Unpublished manuscript.
31. Lane , S. , Wang , N. , & Magone , M. (1996). Gender-related differential item functioning on a middle-school mathematics performance assessment. Educational Measurement: Issues and Practice, 15(4), 21-27, 31.
32. McHenry , J. J. , Hough , L. M. , Toquam , J. L. , Hanson , M. A. , & Ashworth , S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology , 43, 335-354.
33. Medley , D. M. , & Quirk , T. J. (1974). The application of a factorial design to the study of cultural bias in general culture items on the National Teacher Examination. Journal of Educational Measurement , 11, 235-245.
34. Messick , S. , & Jungeblut , A. (1981). Time and method in coaching for the SAT. Psychological Bulletin , 89, 191-216.
35. Millman , J. , Mehrens , W. A ., & Sackett , P. R. (1993). An evaluation of the New York State Bar Examination . Unpublished manuscript.
36. Motowidlo , S. J. (2003). Job performance. In W. C. Borman , D. R. Ilgen , & R. J. Klimoski (Eds.), Handbook of psychology (Vol. 12, pp. 39-54).Hoboken, NJ: Wiley.
37. Motowidlo , S. J. , & Van Scooter , J. R. (1994). Evidence that task performance should be distinguished from contextual performance. Journal of Applied Psychology , 79, 475-480.
38. Munger , G. F. , & Lloyd , B. H. (1991). Effect of speededness on test performance of handicapped and nonhandicapped examinees. Journal of Educational Research , 85, 53-57.
39. Neisser , U. , Boodoo , G. , Bouchard , T. J. , Jr. , Boykin , A. W. , Brody , N. , Ceci , S. J. , et al. (1996). Intelligence: Knowns and unknowns. American Psychologist , 51, 77-101.
40. Nguyen , H. D. (2006). Does stereotype threat differentially affect cognitive ability test performance of minorities and women? A meta-analytic review of experimental evidence. Unpublished doctoral dissertation, Michigan State University, East Lansing.
41. Nguyen , N. T. , & McDaniel , M. A. (2003). Response instructions and racial differences in a situational judgment test. Applied Human Resource Management Research , 8, 33-44.
42. O'Neil , H. F. , & Brown , R. S. (1997). Differential effects of question formats in math assessment on metacognition and effect (Tech. Rep. No. 449). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
43. Ployhart , R. E. , Schneider , B. , & Schmitt , N. (2006). Staffing organizations: Contemporary practice and theory. Mahwah, NJ: Erlbaum.
44. Powers , D. E. (1987). Who benefits most from preparing for a "coachable" admissions test? Journal of Educational Measurement , 24, 247-262.
45. Powers , D. E. , & Fowles , M. E. (1996). Effects of applying different time limits to a proposed GRE writing test. Journal of Educational Measurement , 33, 433-452.
46. Pulakos , E. D. , & Schmitt , N. (1996). An evaluation of two strategies for reducing adverse impact and their effects on criterion-related validity. Human Performance , 9, 241-258.
47. Roth , P. L. , Bevier , C. A. , Bobko , P. , Switzer , F. S. , III , & Tyler , P. (2001). Ethnic group differences in cognitive ability in employment and educational settings. Personnel Psychology , 54, 297-330.
48. Rushton , J. P. , & Jensen , A. R. (2006). The totality of available evidence shows the race IQ gap still remains. Psychological Science , 17, 921-922.
49. Ryan , A. M. , Ployhart , R. E. , & Friedel , L. (1998). Using personality testing to reduce adverse impact: A cautionary note. Journal of Applied Psychology , 83, 298-307.
50. Ryan , A. M. , Ployhart , R. E. , Greguras , G. J. , & Schmit , M. J. (1998). Test preparation programs in selection contexts: Self-selection and program effectiveness. Personnel Psychology , 51, 599-621.
51. Sackett , P. R. (1998). Performance assessment in education and professional certification: Lessons for personnel selection. In M. D. Hakel (Ed.), Beyond multiple choice: Evaluating alternatives to traditional testing for selection (pp. 113-129). Mahwah, NJ: Erlbaum.
52. Sackett , P. R. , & Ellingson , J. E. (1997). The effects of forming multi-predictor composites on group differences and adverse impact. Personnel Psychology , 50, 707-721.
53. Sackett , P. R. , Schmitt , N. , Ellingson , J. E. , & Kabin , M. B. (2001). High-stakes testing in employment, credentialing, and higher education. American Psychologist , 56, 302-318.
54. Sackett , P. R. , & Wilk , S. L. (1994). Within-group norming and other forms of score adjustment in preemployment testing. American Psychologist , 49, 929-954.
55. Scheunemann , J. D. , & Gerritz , K. (1990). Using differential item functioning procedures to explore sources of item difficulty and group performance characteristics. Journal of Educational Measurement , 27 (2), 109-131.
56. Schmeiser , C. B. , & Ferguson , R. L. (1978). Performance of black and white students on test materials containing content based on black and white cultures. Journal of Educational Measurement , 15, 193-200.
57. Schmidt , F. L. , Mack , M. J. , & Hunter , J. E. (1984). Selection utility in the occupation of U.S. park ranger for three modes of test use. Journal of Applied Psychology , 69, 490-497.
58. Schmitt , A. P. (1988). Language and cultural characteristics that explain differential item functioning for Hispanic examinees on the Scholastic Aptitude Test. Journal of Educational Measurement , 25 (1), 1-13.
59. Schmitt , N. , & Mills , A. E. (2001). Traditional tests and job simulations: Minority and majority performance and test validities. Journal of Applied Psychology , 86, 451-458.
60. Schmitt , N. , Rogers , W. , Chan , D. , Sheppard , L. , & Jennings , D. (1997). Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology , 82, 719-730.
61. Stark , S. , Chernyshenko , O. S. , & Drasgow , F. (2004). Examining the effects of differential item functioning and differential test functioning on selection decisions: When are statistically significant effects practically important? Journal of Applied Psychology , 89 (3), 497-508.
62. Steele , C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist , 52, 613-629.
63. Steele , C. M. , & Aronson , J. (1995). Stereotype threat and the intellectual performance of African Americans. Journal of Personality and Social Psychology , 77, 1213-1227.
64. Wang , N. , & Lane , S. (1996). Detection of gender-related differential item functioning in a mathematics performance assessment. Applied Measurement in Education , 9 (2), 175-144.
65. Whitney , D. J. , & Schmitt , N. (1997). Relationship between culture and responses to biodata employment items. Journal of Applied Psychology , 82, 113-129.
66. Wild , C. L. , Durso , R. , & Rubin , D. B. (1982). Effect of increased test-taking time on test scores by ethnic group, years out of school, and sex. Journal of Educational Measurement , 19, 19-28.