Sex-specific differential prediction of college admission tests: A meta-analysis

Journal of Educational Psychology

Volumn 105, Issue 2, 2013, Pages 478-488

  Fischer, Franziska T ^a   Schult, Johannes ^a   Hell, Benedikt ^a  

^a UNIVERSITY OF KONSTANZ   (Germany)

Author keywords

Differential prediction; Gender; Meta analysis; Sex differences; Test bias

Indexed keywords

EID: 84881109968     PISSN: 00220663     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0031956     Document Type: Article

References (97)

1. Aguinis, H., Beaty, J. C., Boik, R. J., & Pierce, C. A. (2005). Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review. Journal of Applied Psychology, 90, 94-107. doi:10.1037/0021-9010.90.1.94
2. Aguinis, H., Culpepper, S. A., & Pierce, C. A. (2010). Revival of test bias research in preemployment testing. Journal of Applied Psychology, 95, 648-680. doi:10.1037/a0018714
3. Aloe, A. M., & Becker, B. J. (2009). Teacher verbal ability and school outcomes: Where is the evidence. Educational Researcher, 38, 612-624. doi:10.3102/0013189X09353939
4. Aloe, A. M., & Becker, B. J. (2011). Advances in combining regression results in meta-analysis. In M. Williams & W. P. Vogt (Eds.), The SAGE handbook of innovation in social research methods (pp. 331-352). London, England: Sage.
5. Alon, S., & Gelbgiser, D. (2011). The female advantage in college academic achievements and horizontal sex segregation. Social Science Research, 40, 107-119. doi:10.1016/j.ssresearch.2010.06.007
6. American College Testing Program. (1973). Assessing students on the way to college: Vol. 1. Technical report for the ACT assessment program. Iowa City, IA: Author.
7. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
8. Bartlett, C. J., Bobko, P., Mosier, S. B., & Hannan, R. (1978). Testing for fairness with a moderated multiple regression strategy: An alternative to differential analysis. Personnel Psychology, 31, 233-241. doi:10.1111/j.1744-6570.1978.tb00442.x
9. Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley, & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 499-525). San Diego, CA: Academic Press. doi:10.1016/B978-012691360-6/50018-5
10. Becker, B. J., & Wu, M. (2007). The synthesis of regression slopes in meta-analysis. Statistical Science, 22, 414-429. doi:10.1214/07-STS243
11. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1, 97-111. doi:10.1002/jrsm.12
12. Borneman, M. J. (2010). Using meta-analysis to increase power in differential prediction analyses. Industrial and Organizational Psychology, 3, 224-227. doi:10.1111/j.1754-9434.2010.01228.x
13. Bridgeman, B., & Lewis, C. (1994). The relationship of essay and multiplechoice scores with grades in college courses. Journal of Educational Measurement, 31, 37-50. doi:10.1111/j.1745-3984.1994.tb00433.x
14. Bridgeman, B., McCamley-Jenkins, L., & Ervin, N. (2000). Predictions of freshman grade-point average from the revised and recentered SAT I: Reasoning test (Research Report No. 2000-1). New York, NY: College Board.
15. Bridgeman, B., Pollack, J., & Burton, N. (2008). Predicting grades in different types of college courses (Research Report No. 2008-1). New York, NY: College Board.
16. Bridgeman, B., & Wendler, C. (1991). Gender differences in predictors of college mathematics performance and in college mathematics course grades. Journal of Educational Psychology, 83, 275-284. doi:10.1037/0022-0663.83.2.275
17. Burton, N. W., & Wang, M. (2005). Predicting long-term success in graduate school: A collaborative validity study (GRE Board Research Report No. 99-14R). Princeton, NJ: Educational Testing Service.
18. Calkins, D. S., & Whitworth, R. (1974). Differential prediction of freshmen grade point average for sex and two ethnic classifications at a southwestern university. Retrieved from ERIC database. (ED102199)
19. Casserly, P. L. (1982). Older students and the SAT (College Board Report No. 82-2). New York, NY: College Board.
20. Chou, T., & Huberty, C. J. (1990). A freshman admissions prediction equation: An evaluation and recommendation. Retrieved from ERIC database. (ED333081)
21. Clark, M. J., & Grandy, J. (1984). Sex differences in the academic performance of Scholastic Aptitude Test takers (College Board Report No. 84-8). New York, NY: College Board.
22. Cleary, T. A. (1968). Test bias: Prediction of grades of Negro and White students in integrated colleges. Journal of Educational Measurement, 5, 115-124. doi:10.1111/j.1745-3984.1968.tb00613.x
23. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
24. Conger, D., & Long, M. C. (2010). Why are men falling behind? Gender gaps in college performance and persistence. Annals of the American Academy of Political and Social Science, 627, 184-214. doi:10.1177/0002716209348751
25. Cowen, S., & Fiori, S. J. (1991, November). Appropriateness of the SAT in selecting students for admission to California State University, Hayward. Paper presented at the annual meeting of the California Educational Research Association, San Diego, CA. Retrieved from ERIC database. (ED343934)
26. Crawford, P. L., Alferink, D. M., & Spencer, J. L. (1986). Postdictions of college GPAs from ACT composite scores and high school GPAs: Comparisons by race and gender. Retrieved from ERIC database. (ED 326541)
27. Dlugosch, S. (2005). Prognose von Studienerfolg dargestellt am Beispiel des Auswahlverfahrens der Bucerius Law School [Prognosis of successful studies described by means of the selection method of the Bucerius Law School]. Aachen, Germany: Shaker.
28. Educational Testing Service. (2009). ETS guidelines for fairness review of assessments. Princeton, NJ: Author.
29. Elliott, R., & Strenta, A. C. (1988). Effects of improving the reliability of the GPA on prediction generally and on comparative predictions for gender and race particularly. Journal of Educational Measurement, 25, 333-347. doi:10.1111/j.1745-3984.1988.tb00312.x
30. Ellis, L., Karadi, K., Hershberger, S., Field, E., Wersinger, S., Pellis, S., Hetsroni, A. (2008). Sex differences: Summarizing more than a century of scientific research. New York, NY: Psychology Press.
31. Goldstein, D., Haldane, D., & Mitchell, C. (1990). Sex differences in visual-spatial ability: The role of performance factors. Memory & Cognition, 18, 546-550. doi:10.3758/BF03198487
32. Gulliksen, H. (1950). Theory of mental tests. New York, NY: Wiley. doi:10.1037/13240-000
33. Gulliksen, H., & Wilks, S. S. (1950). Regression tests for several samples. Psychometrika, 15, 91-114. doi:10.1007/BF02289195
34. Hell, B., Trapmann, S., & Schuler, H. (2007). Eine Metaanalyse der Validität von fachspezifischen Studierfähigkeitstests im deutschsprachigen Raum [A meta-analytic investigation of subject-specific admission tests in German-speaking countries]. Empirische Pädagogik, 21, 251-270.
35. Hewitt, B. N., & Goldman, R. D. (1975). Occam's razor slices through the myth that college women overachieve. Journal of Educational Psychology, 67, 325-330. doi:10.1037/h0077010
36. Hogrebe, M. C., Ervin, L., Dwinell, P. L., & Newman, I. (1983). The moderating effects of gender and race in predicting the academic performance of college developmental students. Educational and Psychological Measurement, 43, 523-530. doi:10.1177/001316448304300221
37. Holden, C. (1989). Court ruling rekindles controversy over SATs. Science, 243, 885-887. doi:10.1126/science.2919279
38. House, J. D. (1998, May). Gender differences in prediction of graduate course performance from admissions test scores: An empirical example of statistical methods for investigating prediction bias. Paper presented at the annual forum of the Association for Institutional Research, Minneapolis, MN. Retrieved from http://eric.ed.gov/
39. House, J. D., & Keeley, E. J. (1993, November). Differential prediction of graduate student achievement from Miller Analogies Test scores. Paper presented at the annual meeting of the Illinois Association for Institutional Research, Oakbrook Terrace, IL. Retrieved from ERIC database. (ED364605)
40. Huff, K. L., Koenig, J. A., Treptau, M. M., & Sireci, S. G. (1999). Validity of MCAT scores for predicting clerkship performance of medical students grouped by sex and ethnicity. Academic Medicine, 74, S41-S44. doi:10.1097/00001888-199910000-00035
41. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.
42. Jones, R. F., & Vanyur, S. (1985, April). An investigation of genderrelated test bias for the Medical College Admission Test. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL. Retrieved from ERIC database. (ED259024)
43. Kirchner, G. L. (1993). Gender as a moderator variable in predicting success in a Master of Arts in Teaching program. Educational and Psychological Measurement, 53, 155-157. doi:10.1177/00131644 93053001017
44. Kobrin, J. L., Patterson, B. F., Shaw, E. J., Mattern, K. D., & Barbuti, S. M. (2008). Validity of the SAT for predicting first-year college grade point average (Research Report No. 2008-5). New York, NY: College Board.
45. Kuncel, N. R., Credé, M., & Thomas, L. L. (2007). A meta-analysis of the predictive validity of the Graduate Management Admission Test (GMAT) and undergraduate grade point average (UGPA) for graduate student academic performance. Academy of Management Learning & Education, 6, 51-68. doi:10.5465/AMLE.2007.24401702
46. Kuncel, N. R., & Hezlett, S. A. (2007). Standardized tests predict graduate students' success. Science, 315, 1080-1081. doi:10.1126/science.1136618
47. Kuncel, N. R., Wee, S., Serafin, L., & Hezlett, S. A. (2010). The validity of the Graduate Record Examination for master's and doctoral programs: A meta-analytic investigation. Educational and Psychological Measurement, 70, 340-352. doi:10.1177/0013164409344508
48. Kyei-Blankson, L. (2005). Predictive validity, differential validity, and differential prediction of the subtests of the Medical College Admission Test (Doctoral dissertation). Retrieved from http://etd.ohiolink.edu/
49. Lawshe, C. H. (1983). A simplified approach to the evaluation of fairness in employee selection procedures. Personnel Psychology, 36, 601-608. doi:10.1111/j.1744-6570.1983.tb02237.x
50. Leonard, D. K., & Jiang, J. (1999). Gender bias and the college predictions of the SATs: A cry of despair. Research in Higher Education, 40, 375-407. doi:10.1023/A:1018759308259
51. Lindberg, S. M., Hyde, J. S., Petersen, J. L., & Linn, M. C. (2010). New trends in gender and mathematics performance: A meta-analysis. Psychological Bulletin, 136, 1123-1135. doi:10.1037/a0021276
52. Linn, R. L. (1973). Fair test use in selection. Review of Educational Research, 43, 139-161. doi:10.2307/1169933
53. Linn, R. L. (1978). Single-group validity, differential validity, and differential prediction. Journal of Applied Psychology, 63, 507-512. doi: 10.1037/0021-9010.63.4.507
54. Linn, R. L. (1982). Admissions testing on trial. American Psychologist, 37, 279-291. doi:10.1037/0003-066X.37.3.279
55. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
56. Luthy, T. L. (1996). Validity and prediction bias of grade performance from Graduate Record Examination scores for students at Northern Illinois University: Age and gender considerations (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 9716551)
57. Lynn, R., & Irwing, P. (2004). Sex differences on the progressive matrices: A meta-analysis. Intelligence, 32, 481-498. doi:10.1016/j.intell.2004.06.008
58. Lynn, R., & Kanazawa, S. (2011). A longitudinal study of sex differences in intelligence at ages 7, 11 and 16 years. Personality and Individual Differences, 51, 321-324. doi:10.1016/j.paid.2011.02.028
59. Lynn, R., & Mau, W. (2001). Ethnic and sex differences in the predictive validity of the Scholastic Achievement Test for college grades. Psychological Reports, 88, 1099-1104.
60. Mattern, K. D., Patterson, B. F., Shaw, E. J., Kobrin, J. L., & Barbuti, S. M. (2008). Differential validity and prediction of the SAT (Report No. 2008-4). New York, NY: College Board.
61. Meade, A. W., & Fetzer, M. (2009). Test bias, differential prediction, and a revised approach for determining the suitability of a predictor in a selection context. Organizational Research Methods, 12, 738-761. doi: 10.1177/1094428109331487
62. Meade, A. W., & Tonidandel, S. (2010). Not seeing clearly with Cleary: What test bias analyses do and do not tell us. Industrial and Organizational Psychology, 3, 192-205. doi:10.1111/j.1754-9434.2010.01223.x
63. Nauels, H., & Meyer, M. (1997). Untersuchungen zur Vorhersagekraft des TMS: Differentielle Aspekte der Studienerfolgsprognose und Testfairne§ [Investigation of the predictive power of the TMS: Differential aspects of the prediction of college success and test fairness]. In G. Trost (Ed.), Test für Medizinische Studiengänge (TMS): Studien zur Evaluation (21. Arbeitsbericht, pp. 76-134). Bonn, Germany: ITB.
64. Norborg, J. M. (1984). A warning regarding the simplified approach to the evaluation of test fairness in employee selection procedures. Personnel Psychology, 37, 483-486. doi:10.1111/j.1744-6570.1984.tb00524.x
65. Pape, T. E. (1992). Selected predictors of examination for professional practice in psychology scores among graduates of Western Conservative Baptist Seminary's doctoral program in clinical psychology (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 9302769)
66. Patterson, B. F., Mattern, K. D., & Kobrin, J. L. (2009). Validity of the SAT for predicting FYGPA: 2007 SAT validity sample (Statistical Report No. 2009-1). New York, NY: College Board.
67. Patton, T. K. (1998). Differential prediction of college performance between gender. Retrieved from ERIC database. (ED029407)
68. Pennock-Román, M. (1994). College major and gender differences in the prediction of college grades (College Board Report No. 94-2). New York, NY: College Board.
69. Qualls, A. L., & Ansley, T. N. (1995). The predictive relationship of ITBS and ITED to measures of academic success. Educational and Psychological Measurement, 55, 485-498. doi:10.1177/00131644 95055003016
70. Ramist, L., Lewis, C., & McCamley-Jenkins, L. (1994). Student group differences in predicting college grades: Sex, language, and ethnic groups (Report No. 93-1). New York, NY: College Board.
71. Reuben, T. C. (2003). Investigating test fairness of GRE scores for veterinary student selection (Doctoral dissertation). Available from Pro- Quest Dissertations and Theses database. (UMI No. 3083156)
72. Sackett, P. R. (2003). Stereotype threat in applied selection settings: A commentary. Human Performance, 16, 295-309. doi:10.1207/S15327043HUP1603_6
73. Sackett, P. R., Borneman, M. J., & Connelly, B. S. (2008). High stakes testing in higher education and employment: Appraising the evidence for validity and fairness. American Psychologist, 63, 215-227. doi:10.1037/0003-066X.63.4.215
74. Sackett, P. R., Hardison, C. M., & Cullen, M. J. (2005). On interpreting research on stereotype threat and test performance. American Psychologist, 60, 271-272. doi:10.1037/0003-066X.60.3.271
75. Sanber, S. R., & Millman, J. (1987). Gender and race effects on standardized tests predictive validity: A meta-analytical study. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC. Retrieved from ERIC database. (ED286914)
76. Schmidt, F. L., & Hunter, J. E. (1982). Two pitfalls in assessing fairness of selection tests using the regression model. Personnel Psychology, 35, 601-607. doi:10.1111/j.1744-6570.1982.tb02212.x
77. Shadish, W. R., & Haddock, C. K. (2009). Combining estimates of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (2nd ed., pp. 257-277). New York, NY: Russell Sage Foundation.
78. Siegert, K. O. (2007). Predicting success in graduate management doctoral programs (GMAC Research Reports No. RR-07-10). McLean, VA: Graduate Management Admission Council.
79. Sireci, S. G., & Talento-Miller, E. (2006). Evaluating the predictive validity of Graduate Management Admission Test scores. Educational and Psychological Measurement, 66, 305-317. doi:10.1177/0013164405282455
80. Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.
81. Spencer, S. J., Steele, C. M., & Quinn, D. M. (1999). Stereotype threat and women's math performance. Journal of Experimental Social Psychology, 35, 4-28. doi:10.1006/jesp.1998.1373
82. Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629. doi: 10.1037/0003-066X.52.6.613
83. Sterne, J. A. C., & Egger, M. (2005). Regression methods to detect publication and other bias in meta-analysis. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 99-110). Chichester, England: Wiley.
84. Stricker, L. J., Rock, D. A., & Burton, N. W. (1993). Sex differences in predictions of college grades from Scholastic Aptitude Test scores. Journal of Educational Psychology, 85, 710-718. doi:10.1037/0022-0663.85.4.710
85. Swinton, S. S. (1987). The predictive validity of the restructured GRE with particular attention to older students (GRE Board Report No. 83-25P). Princeton, NJ: Educational Testing Service.
86. Talento-Miller, E. (2008). Generalizability of GMAT validity to programs outside the U.S. International Journal of Testing, 8, 127-142. doi: 10.1080/15305050802001193
87. Talento-Miller, E. (2009). Validity study of non-MBA programs (GMAC Research Reports No. RR-09-11). McLean, VA: Graduate Management Admission Council.
88. Thomas, C. L. (1973, February). The overprediction phenomenon among black collegians: Some preliminary considerations. Paper presented at the annual meeting of American Educational Research Association, New Orleans, LA. Retrieved from ERIC database. (ED076679)
89. Thomas, C. L. (1979). Relative effectiveness of high school grades for predicting college grades: Sex and ability level effects. Journal of Negro Education, 48, 6-13. doi:10.2307/2294611
90. Willingham, W. W., Pollack, J. M., & Lewis, C. (2002). Grades and test scores: Accounting for observed differences. Journal of Educational Measurement, 39, 1-37. doi:10.1111/j.1745-3984.2002.tb01133.x
91. Wilson, K. M. (1982). A study of the validity of the restructured GRE aptitude tests for predicting first-year performance in graduate study (GRE Board Research Report No. 78-6R). Princeton, NJ: Educational Testing Service.
92. Wood, J. A. (2008). Methodology for dealing with duplicate study effects in a meta-analysis. Organizational Research Methods, 11, 79-95. doi: 10.1177/1094428106296638
93. Wynne, W. D. (2003). An investigation of ethnic and gender intercept bias in the SAT's prediction of college freshman academic performance (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3116464)
94. Young, J. W. (1994). Differential prediction of college grades by gender and by ethnicity: A replication study. Educational and Psychological Measurement, 54, 1022-1029. doi:10.1177/0013164494054004019
95. Young, J. W., & Kobrin, J. L. (2001). Differential validity, differential prediction, and college admission testing: A comprehensive review and analysis. (Research Report No. 2001-6). New York, NY: College Board.
96. Zeidner, M. (1987). A cross-cultural test of sex bias in the predictive validity of scholastic aptitude examinations: Some Israeli findings. Evaluation and Program Planning, 10, 289-295. doi:10.1016/0149-7189(87)90041-3
97. Zwick, R. (2002). Fair game? The use of standardized admissions tests in higher education. New York, NY: Routledge Falmer.