Is affirmative action responsible for the achievement gap between black and white law students?

Northwestern University Law Review

Volumn 101, Issue 4, 2007, Pages 1759-1808

  Barnes, Katherine Y ^a  

^a UNIVERSITY OF ARIZONA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 37749010914     PISSN: 00293571     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Review

References (203)

1. Grutter v. Bollinger, 539 U.S. 306 (2003).
2. See RACIAL PREFERENCE AND RACIAL JUSTICE: THE NEW AFFIRMATIVE ACTION CONTROVERSY (Russell Nieli ed., 1991) (an anthology of views on affirmative action);
3. see also CAROL COHEN & JAMES P. STERBA, AFFIRMATIVE ACTION AND RACIAL PREFERENCE: A DEBATE (2003) (providing two sides of the moral debate over racial preferences).
4. See, e.g., Celestial S.D. Cassman & Lisa R. Pruitt, A Kinder, Gentler Law School? Race, Ethnicity, Gender, and Legal Education at King Hall, 38 U.C. DAVIS L. REV. 1209 (2005) (reporting the results of a study of law school culture and its impact on students at U.C. Davis Law School);
5. David L. Chambers, Richard O. Lempert & Terry K. Adams, Michigan's Minority Graduates in Practice: The River Runs Through Law School, 25 L. & SOC. INQUIRY 395 (2000) (discussing benefits of diversity to University of Michigan Law School graduates);
6. see also Timothy T. Clydesdale, A Forked River Runs Through Law School: Toward Understanding Age, Gender, Race, and Related Gaps in Law School Performance and Bur Passage, 29 L. & SOC". INQUIRY 711 (2004) (providing a cross-institution comparison); The Educational Diversity Project, http://www.unc.edu/edp (ongoing major survey project focusing on the effects of diversity in legal education) (last visited Apr. 30, 2007).
7. See, e.g.. Chambers et al., supra note 3, at 428-30, 456-58 (describing University of Michigan Law School alumni study, which demonstrated that minority law students who graduated in the 1970s performed significantly more pro bono activities than their white counterparts, and were more likely to work for public institutions).
8. Richard H. Sander, A Systemic Analysis of Affirmative Action in American Law Schools, 57 STAN. I.. REV. 367 (2004).
9. Id. at 426. Black students account for 7% of the new matriculants at law school each year.
10. See American Bar Association, African American J.D. Enrollment, 1971-2005, at 1, http://www.abanet.org/ legaled/statistics/charts/stats%20- %2013.pdf (last visited Apr, 30, 2007) (providing data through academic year 2006-2007);
11. American Bar Association, First Year J.D. and Total J.D. Minority Enrollment for 1971-2005, at 1, http://www.abanet.org/legaled/statistics/charts/ stats%20-%208.pdf (last visited Apr. 30, 2007) (providing data through academic year 2006-2007, and showing that the percentage of black first year law students ranged from 6.5% to 7.7% from 2001 to 2006). Total minority enrollment has been approximately 20% of the total first year population of students since the mid 1990s.
12. See First Year J.D. and Total J.D. Minority Enrollment for 1971-2005, supra, at 1. This datum is accurate overall for law students, and for most law schools, with the major exception of historically black institutions.
13. See, e.g., ABA-LSAC, The Official Guide to ABA-Approved Law Schools, 2008 Edition, Left Page 2 Data (Excel Spreadsheets), http://www.abanet.org/legaled/statistics/charts/OG%20Left%20Page%202% 202008.xls (last visited June 14, 2007) (demonstrating that most ABA-approved law schools enroll between 10% and 30% minority students, with most of those schools at 20% or lower; the few schools with 50% or more minority students are historically black institutions or institutions that have large Hispanic student populations);
14. ABA-LSAC, THE OFFICIAL GUIDE TO ABA-APPROVED LAW SCHOOLS 56-65 (2002) (listing the percentage of minority students enrolled at each ABA-approved law school in 2000). As others have noted, some group of people must be in the bottom tenth of the class.
15. David B. Wilkins, A Systematic Response to Systemic Disadvantage: A Response to Sander, 57 STAN. L. REV. 1915, 1917 (2005). And before blacks were admitted to law school, few worried that the white men at the bottom of the class were "outmatched" by their peers. Still, that half of all black students are in the bottom tenth of the class at least suggests that law schools may not be providing as good an education to their black students, for whatever reason. Even if this is not the case-again, some group of students has to be at the bottom of the class-law schools should think carefully about the stereotypes that this fact perpetuates.
16. Sander, supra note 5, at 473.
17. See infra note 44 for a list of the primary ways in which Sander's analysis is flawed.
18. As a self-proclaimed "math geek," I hear confessions from many people about prior math classes. The mantra "I'm just not good at math" may be, at least in part, a widely held, culturally accepted disengagement with the process of learning mathematics. See Elena Nardi & Susan Steward, Is Mathematics T.I.R.E.D? A Profile of Quiet Disaffection in the Secondary Mathematics Classroom, 29 BRIT. EDUC. RES. J. 345, 362 (2003) (noting that "self-images of mathematical ability [were] overwhelmingly negative" and that this contributed to less willingness to engage with mathematics).
19. Sander, supra note 5, at 449-50.
20. Stereotype threat is a term used to describe the phenomenon of minority groups performing more poorly than expected because of the stereotype that the minority groups will, in fact, perform poorly. See generally Claude M, Steele & Joshua Aronson, Stereotype Threat and the Intellectual Test Perform ance of African Americans, 69 J. PERSONALITY & SOC. PSYCHOL. 797 (1995). As a concrete example, black students taking the SAT hear and internalize the stereotype that blacks do not do well on the SAT. They worry about the prospect, they get angry at the stereotype, and they feel that others expect them to do poorly. All of these factors distract them from the task at hand-taking the SAT-and therefore make optimal performance more difficult.
21. Id. at 799.
22. Signals as small as checking a box to indicate the test taker's race have been shown to trigger stereotype threat. Jessi L. Smith, Understanding the Process of Stereotype Threat: A Review of Mediational Variables and New Performance Goal Directions, 16 F.DUC. PSYCHOL. REV. 177, 181 (2004).
23. Several readers of drafts of this Essay commented that such treatment would simply make them work harder. I have two responses to this suggestion. First, as educators, we should not condone a discriminatory culture because it "makes students work harder" or somehow instills values of perseverance. Second, as an individual student, I certainly learned from this experience and did not allow such treatment to affect my performance in subsequent classes. The underlying point, however, is that the educational environment should be created in such a way as to facilitate learning for all students, without creating barriers to learning based upon arbitrary factors such as gender or race.
24. Wills v. Brown University, 184 F.3d 20, 26 (1st Cir. 1999) (defining a hostile environment claim as "harassment severe enough to compromise the victim's employment or educational opportunities").
25. I recognize that there are many measures of law school achievement; grades, participation in moot courts, law review, trial teams, and other activities are just a few. The only measured achievement gap upon which I focus is grades; this is the only measured achievement within law school. I focus on grades primarily because of the common perception that grades are the primary determinant of job prospects after law school and because it is a very stark statistic that I believe deserves some attention.
26. Some critics acknowledge the troubling aspect of the data but suggest that Sander's remedy of dismantling affirmative action is inappropriate. See Ian Ayres & Richard Brooks, Does Affirmative Action Reduce the Number of Black Lawyers?, 57 STAN. L. REV. 1807, 1808-09 (2005). Indeed, just because the legal academy may be doing a poorer job of educating its black students does not mean that the legal academy should get out of the business of doing so.
27. Many schools have some program like this, at least informally. For example, the Law School Admission Council (LSAC) sponsors conferences for academic support personnel. See LSACNet.org: Events & Dates: Calendar of Events, http://www.lsacnet.org (click on "Event Calendar") (last visited June 14, 2007) (listing events including the LSAC Academic Assistance Training Workshop, held in June 2007).
28. This is a consistent argument against affirmative action. See Faye Crosby, Aarti Iyer & Sirinda Sincharoen, Understanding Affirmative Action, 57 ANN. REV. PSYCHOL. 585, 593 (2006) (discussing psychological experiments that at best provide limited empirical support for this theory).
29. The briefing in Grutter v. Bollinger, 539 U.S. 306 (2003), provides a thorough airing of the advantages of diversity in education.
30. See, e.g., Consolidated Brief of Lt. Gen. Julius W. Becton, Jr. et al., as Amici Curiae in Support of Respondents at 13-18, Grutter, 539 U.S. 306 (No. 01-1015), 2003 WL 1787554 (discussing the need for affirmative action to maintain an integrated officers corps in the military);
31. Brief of General Motors Corp. as Amicus Curiae in Support of Respondents at 12, Grutter, 539 U.S. 306 (No, 01-1015), 2003 WL 399096 (arguing that diverse educational environments provide cross-cultural skills necessary to compete in the global economy);
32. see also Brief of the American Educational Research Ass'n. et al. as Amici Curiae in Support of Respondents, Grutter, 539 U.S. 306 (No. 01-1015), 2003 WL 402134 (compiling research on the importance of educational diversity).
33. See infra note 44 and accompanying text for a further discussion of Sander's mistaken conclusion that discrimination cannot be a factor in law student performance.
34. Sander, supra note 5, at 468-78.
35. See generally Ayers & Brooks, supra note 15 (analyzing the LSAC data by comparing the performances of black and white law students who have similar credentials and matriculate at the same tier of school);
36. David L. Chambers, Timothy T. Clydesdale, William C., Kidder & Richard O. Lempert, The Real Impact of Eliminating Affirmative Action in American Law Schools: An Empirical Critique of Richard Sander's Study, 57 STAN. L. REV. 1855 (2005) (reanalyzing the data, correcting several mistakes that Sander makes in coding the data);
37. Daniel E. Ho, Scholarship Comment, Why Affirmative Aclion Does Not Cause Black Student to Fail the Bar, 114 YALE L.J. 1997 (2005) [hereinafter Ho, Scholarship Comment];
38. Daniel E. Ho, Affirmative Action's Affirmative Actions: A Reply to Sander, 114 YALE L.J, 2011 (2005) [hereinafter Ho, Affirmative Actions];
39. Jesse Rothstein & Albert Yoon, Mismatch in Law School (Northwestern Law & Econ., Research Paper No. 881110, 2006), available at http://www.princeton.edu/~jrothst/rothstein-yoon_2006junel5.pdf (using non-parametric reweighting to analyze the data);
40. Daniel E. Ho, Evaluating Affirmative Action in American Law Schools: Does Attending a Better Law School Cause Black Students to Fail the Bar (Mar. 9, 2005) (unpublished manuscript), available at http://people.iq.harvard. edu/~dho/research/sander.pdf [hereinafter Ho, Social Science Critique] (using propensity score matching to analyze the data).
41. See generally andré douglas pond cummings, "Open Water": Affirmative Action, Mismatch Theory and Swarming Predators: A Response to Richard Sander, 44 BRANDEIS L.J. 795 (2006) (arguing, inter alia, that Sander's article is paternalistic and normatively flawed because it does not take into account many positive aspects of affirmative action);
42. Michele Landis Dauber, The Big Muddy, 57 STAN. L. REV. 1899 (2005) (arguing, inter alia, that Sander's assumption, that absent affirmative action black and white law students would perform similarly, is seriously flawed, and that his poor analysis should not have been published because it did not satisfy peer review and replication standards);
43. Kevin R. Johnson & Angela Onwuanchi-Willig, Cry Me A River: The Limits of "A Systemic Analysis of Affirmative Action in American Law Schools, " 7 AFR.-AM. L. & POL'Y REP. 1 (2005) (noting that other explanations for the performance gap between black and white law students are hostile learning environment and the extensive community service that black law students perform);
44. Beverly I, Moran, The Case for Black Inferiority? What Must be True if Professor Sander is Right: A Response to a Systemic Analysis of Affirmative Action in American Law Schools, 5 CONN. PUB. INT. L.J. 41 (2005) (listing six "truths"-logical leaps in his argument-that Sander makes, but fails to justify);
45. Wilkins, supra note 6 (focusing on the value of having black law students graduate from elite institutions).
46. Sander's critics do not always defend affirmative action explicitly, but in evaluating Sander's methodology and results, it is clear that his critics take as a baseline the use of affirmative action. By doing so, Sander's critics require that any deviation from current affirmative action policy be justified by empirical proof. Even the title of the response of Chambers et al., The Real Impact of Eliminating Affirmative Action, emphasizes that affirmative action is the baseline from which there might be a policy change.
47. Chambers et al., supra note 21.
48. See Ayers & Brooks, supra note 15;
49. Chambers et al., supra note 21;
50. cummings, supra note 22;
51. Dauber, supra note 22;
52. Johnson & Onwuanchi-Willig, supra note 22;
53. Moran, supra note 22;
54. Richard H. Sander, Mismeasuring the Mismatch, 114 YALE L.J. 2005 (2005) [hereinafter Sander, Mismeasuring];
55. Richard H. Sander, A Reply to Critics, 57 STAN. L. REV. 1963 (2005) [hereinafter Sander, Reply];
56. Wilkins, supra note 6;
57. Ho, Scholarship Comment, supra note 21;
58. Ho, Affirmative Actions, supra note 21;
59. Rothstein & Yoon, supra note 21 ;
60. Ho, Social Science Critique, supra note 21.
61. To date, I have been unable to find a published article or working paper in an academic venue that defends Sander's work, other than his own. See Sander, Mismeasuring, supra note 23;
62. Sander, Reply, supra note 22.
63. As I use the phrase, "credentials" or "student credentials" is a broad term encompassing any information known to admissions officers that might impact law school performance. LSAT and UGPA are two objective measures of student credentials, but other variables, ranging from difficulty of undergraduate program to maturity level to family support network, may impact law school performance. Consistent with my broad definition, the phrase "student credentials" encompasses all of these variables. In this discussion, I assume that one can accurately measure student credentials, although the empirical tests I report use only the imperfect measures of LSAT and UGPA. It is, however, theoretically possible to measure student credentials, even when broadly construed to include any attribute that will affect student performance. See infra Part III.B.
64. Ayres & Brooks, supra note 15, at 1808-09;
65. Ho, Scholarship Comment, supra note 21, at 1998, 2000;
66. Rothstein & Yoon, supra note 21, at 3-4. Chambers et al. do not directly argue with the mismatch theory per se, but instead focus on the performance of black law students if affirmative action were to be eliminated.
67. See generally Chambers et al., supra note 21. Ayres and Brooks recognize that the credentials of white and black students overlap, but do not explicitly state that this means that white students could be mismatched.
68. Ayres & Brooks, supra note 15, at 1811-12, fig.l.
69. Donald B. Rubin, Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies, 66 J. EDUC PSYCHOL. 688, 692-93 (1974) (discussing variables that might confound the analysis in two examples, and arguing that causal relationships are difficult to prove absent randomization due to confounding).
70. Ayres and Brooks, supra note 15, compare black and white students within a given tier of law schools.
71. Id. at 1813-17,
72. Rothstein and Yoon, supra note 21, compare within tier across race.
73. Id. at 24-28.
74. In addition, these researchers make other comparisons, some of which are more appropriate. Ho makes several comparisons: he compares those students who matriculate at top tier schools versus those who do not, both for all students and for the subset of only black students. Ho, Social Science Critique, supra note 21, at 9. In addition, Ho provides some detail on the same comparison for white law students.
75. Ho, Scholarship Comment, supra note 21, at 2002. Finally, Ho uses propensity scores to create more complex comparison groups.
76. Ho, Social Science Critique, supra note 21, at 9 & tbl.2, fig.5.
77. Ayres and Brooks, supra note 15, also compare black students across tiers,
78. id. at 1825,
79. and Rothstein and Yoon, supra note 21, compare both within race across tier as well,
80. id. at 24-28. Thus, many researchers compare black students who attend different school types, confounding different school cultures with mismatch, and, more troubling, white students with black students at the same school type. This latter comparison is based upon the demonstrably false assumption that white and black students with similar credentials attend different schools. While perhaps true on average, making this comparison confounds mismatch effects with discrimination effects.
81. Again, this assumes that student credentials are accurately measured; I discuss the consequences of biased measures of student credentials in Part III.
82. Power is a term of art in statistics that measures the ex ante probability of rejecting the null hypothesis of a test when the null hypothesis is in fact false. The power of a statistical test increases as sample size increases because with more data, one can discern the difference between random variation and a true effect more accurately.
83. See Ayres & Brooks, supra note 15, at 1828-38;
84. Ho, Social Science Critique, supra note 21, at 9 & fig.4;
85. Rothstein & Yoon, supra note 21, at 24-25.
86. Ho, Scholarship Comment, supra note 21, at 2002-04 & fig. 1, provides some detail on the crosstier comparison for white law students.
87. Statistical models can contain "main" effects and "interaction" effects. "Main" effects estimate the average performance effect for the group. For example, the "main" effect of school type will estimate the average graduation rate for each school type separately, and the credentials "main" effects will estimate the average performance boost a student receives based upon a boost in their credentials. Without interaction terms, however, the model would assume that the effect of credentials upon performance and the effect of school type on performance is simply additive; the two do not interact at all. For example, students in a highly-ranked school would receive the same the same performance boost from being at that school whether their credentials suggested that they were mismatched or not. See generally JAMES JACCARD & ROBERT TURRISI, INTERACTION EFFECTS IN MULTIPLE REGRESSION (2d ed. 2003) (providing a description of when interaction effects are required and how to implement them in multiple regression models).
88. Sander, supra note 5, at 452-53. This assumption is factually inaccurate.
89. See Ayres & Brooks, supra note 15, at 1812 fig.1 (describing the overlap in credentials between black and white law students matriculating at schools in the same tier).
90. Sander, supra note 5, at 439 tbl.5.6, 444 tbl.6.1 (providing Sander's empirical argument);
91. Ho, Scholarship Comment, supra note 21, at 2000 (criticizing Sander's empirical analysis because of posttreatment bias).
92. Title IX provides a cause of action for hostile learning environment. See, e.g., Davis v. Monroe County Bd. of Educ, 74 F.3d 1186 (11th Cir. 1996), aff'd 526 U.S. 629 (1999) (finding that Title IX provides a cause of action for hostile learning environment in the context of sex discrimination);
93. Bonnell v. Lorenzo, 241 F.3d 800 (6th Cir, 2001) (recognizing a school's interest in curtailing a professor's speech in order to prevent hostile learning environment, also in the context of sex discrimination).
94. Direct discrimination in outcomes could include class opportunities, job opportunities, research fellowships, and non-anonymous final grades.
95. In stereotype threat, an individual student incorporates the stereotype that he or she will perform poorly in a specific situation (for example, a final exam), and because of that stereotype, actually performs worse than should be expected given the knowledge the student has obtained, See generally Steele & Aronson, supra note 11 (describing the stereotype threat theory).
96. Because stereotype threat may be counteracted by simple statements suggesting that the test administered is unbiased, in some situations stereotype threat may appear to be a school-specific phenomenon, as some schools counteract the threat while others do not. Smith, supra note 11, at 181-82 (noting that the stereotype threat effect was nullified when a black professor introduced a test by stating that the test was "the first step in an attempt to develop a culturally unbiased test").
97. See supra note 33 for further discussion of main and interaction effects.
98. The Educational Diversity Project, supra note 3, hopes to provide just such data on student experiences in law school; analysis of the data is still ongoing.
99. Recall that student credentials are attributes that affect student performance about which admissions offices have information. See supra note 25. Because applications for admission cannot include every possible attribute that might affect student performance, there will always be unmeasured attributes that affect student performance, that is, unmeasured credentials. Many of these will be "soft" variables: ability to work with others well and to persevere, familial support, familiarity with the law and lawyers, etc.
100. This does not rule out the possibility that race is an important predictor because of discriminatory barriers as well as unmeasured student credentials.
101. See Chambers et al., supra note 21, for a complete description of the poor modeling choices that Sander makes. In brief, Chambers et al. focus on five criticisms. First, Sander measures eliteness (or "tier") as a linear variable, implicitly assuming that the difference between the top and second tiers is equal to the difference between the second and third tiers,
102. Id. at 1872-73. Second, in one of the data sets Sander uses, Sander coded all individuals who declined to provide racial data as white. This is inappropriate, and other methods for dealing with missing values provide substantially different results.
103. Id. at 1878-79. Third, Sander uses 2001 data to predict trends for 2004;
104. Chambers et al. argue that 2001 and the adjacent years were anomalous, and note that the pool of potential law students has changed significantly enough in the past three years to call this assumption into question. Id. at 1860-61 & tbl.l.
105. Fourth, Sander misuses test statistics, id. at 1869-71, and as a result, makes overbroad assertions about his results. Finally, Chambers et al. look beyond Sander's analysis to argue that the pool of law school applicants is not a fixed entity that would remain unchanged absent affirmative action: Black students, when assessing their options, may be less likely to attend law school if it is a less attractive option to them.
106. Id. at 1862-68.
107. Id. at 1878-79.
108. Luckily, a third important variable, race, is accurately measured in the data and therefore does not present a problem, As Chambers ct al. point out, Sander also uses a different set that does have measurement problems with respect to the race variable. Id. at 1878-79. As Chambers et al. demonstrate, how one controls for this data problem affects the results of the estimation.
109. Id.
110. See Sander, supra note 5, at 415 ("[School tiers] correspond roughly to tiers of law school prestige.").
111. Ayres & Brooks, supra note 15, at 1812, 1817;
112. Chambers et al, supra note 21, at 1884;
113. Rothstein & Yoon, supra note 21, at tbl. 1 ;
114. Ho, Social Science Critique, supra note 21, at 10 n.6.
115. LINDA F. WIGHTMAN, USER'S GUIDE: LSAC NATIONAL LONGITUDINAL DATA FILE 15 (1999), available at http://bpsdata.lsac.org/ [hereinafter WIGHTMAN, USER'S GUIDE] (file may be downloaded upon submitting personal information) ("There is no explicit or implicit rank ordering associated with the assigned cluster numbers.").
116. In Part III, infra, I detail the additional data necessary to provide more accurate results.
117. See LINDA F. WIGHTMAN, LSAC NATIONAL LONGITUDINAL BAR PASSAGE STUDY, LSAC RESEARCH REPORT SERIES 8-9 (1998) [hereinafter WIGHTMAN, LSAC STUDY] (describing the school clusters by various demographic characteristics, such as percentage of institutions in the cluster that are public and percentage of minority students in the cluster).
118. I have also performed the analysis with the original school type structure. The results from Part II are substantially similar; the results from Part I differ in two ways: first, the racial barriers tests demonstrate that each of the six school types have different average school cultures that affect minority students differently; second, the mismatch tests show no discernible pattern (either mismatch or reverse mismatch), likely because of the overlap in school ranks across school types. Tables for this alternative analysis are available from the author upon request.
119. To put LSAT and UGPA on the same scale, Sander first normalizes both variables, so the full function is: Credentials = 0.6 times; 1000 ×(LSAT - 10)/38+0.4×1000 × UGPA/4 Sander, supra note 5, at 393. My formula differs from that in Sander's article because Sander uses normalized values for LSAT and UGPA; the formula above does the normalization as well,
120. Ayres & Brooks, supra note 15, at 1817 (using the same academic index that Sander creates);
121. Chambers et al., supra note 21, at 1884 (using the same academic index that Sander creates);
122. Rothstein & Yoon, supra note 21, at 3 (using the same academic index that Sander creates);
123. Ho, Social Science Critique, supra note 21, at 5-6 (matching on LSAT and UGPA separately, or controlling for LSAT and UGPA via propensity scores).
124. In practice, I do not estimate student credentials separately, but include this polynomial of LSAT and UGPA into each logistic regression of the achievement measure of interest.
125. The β coefficients are estimated via logistic regression. The function allows significantly more flexibility than a linear or quadratic function. Specifically, it allows for very low credentials-those most at risk of being mismatched-to affect performance differently than all other credentials. This is the purpose of choosing at least a cubic polynomial: I investigated higher-ordered polynomials of degree 4 or 5, but found little additional benefit from these more complicated models. I also investigated how different my results would be if I used the academic index that Sander created; the results are substantially different. Using the less flexible form significantly underestimates the black student graduation rate and bar passage rate, as compared to the more flexible form.
126. This is approximately $51,260 in 2005 dollars. See Federal Reserve Bank of Minneapolis, Consumer Price Index Calculator, http://minneapolisfed.org/Research/data/us/calc (last visited Apr. 16, 2007).
127. The cutoff is low enough that it includes some federal clerkships, depending on location, I have also investigated other cutoffs, including $50,000 in 1995 dollars ($64,075 in 2005 dollars). Id. The results do not change substantially, although fewer students meet this criterion in general.
128. See Wilkins, supra note 6, at 1934-36 (describing the myriad ways that elite institutions train students to become a part of the elite law firm).
129. Specifically, the model is a logistic regression of graduation status on student credentials, type of school, race, race * school type interactions, and student credentials * school type interactions.
130. Tables IA through 3B all report results from logistic regressions of a performance measure on student credentials, type of school, race, race * school type interactions, and student credentials * school type interactions. The "A" tables report results relevant to the mismatch theory, and the "B" tables report results relevant to the race-based barriers theory. Because of the large number of interaction terms for each table. I report the significance level of a joint test whether or not the interactions are statistically significant. For the "A" tables, this is the mismatch theory test; for the "B" tables, this is the race-based barriers test. In addition, rather than report each parameter for the 27 (school type * credentials) interactions and the 12 (school type * race) interactions, I provide the change in estimated successful performance relative to a "baseline" group of students who are described in each panel. For example, the first three rows of Table IA report the change in graduation rate for students at Historically Black, Lowrange, and Top 30 Schools, respectively, as compared to a baseline of students who matriculated at Midrange Schools, whose credentials are in the fifth percentile of the entire data set. These "baseline" students have an 83.3% chance of graduating from law school.
131. Again, because of the serious limitations in the data described above, I do not want to overclaim. The results suggest that the mismatch and discrimination theories are likely in evidence, but they cannot demonstrate conclusively that either theory affects law students' graduation rates.
132. This number provides the change in probability between the given characteristic and the control, holding credentials at the specified value and race at its modal value, white.
133. For these interaction terms, I provide only an overall likelihood ratio test to test the null hypothesis that all interactions are jointly equal to zero.
134. A p-value measures how likely it is that the observed difference between the model estimates and a "null hypothesis" can be explained by random variation in the data. Traditionally, values less than 0.05 (on a scale of 0 to 1) are considered statistically significant; values between 0.10 and 0.05 are considered marginally significant. For Table 1A, the null hypothesis is that mismatch theory does not affect student graduation rates.
135. This number provides the change in probability between the given characteristic and the control, holding all other factors at their median or modal value (LSAT = 37; UGPA = 3.3; race = white).
136. Specifically, the difference is 28 percentage points for those in the fifth percentile of student credentials, and 20 percentage points for those in the tenth percentile. In a logistic regression, the change in probabilities is not additive, because of the logistic functional form. Thus, to compare Historically Black Schools to schools other than Mid-range Schools (the baseline), one has to return to the underlying coefficients of the model.
137. This number provides the change in probability between the given characteristic and the control, holding credentials at the specified value and race at its modal value, white.
138. This number provides the change in probability between the given characteristic and the control, holding all other factors at their median or modal value (LSAT = 37; UGPA = 3.3; race = white).
139. This number provides the change in probability between the given characteristic and the control, holding credentials at the specified value and race at its modal value, white.
140. In addition to a small sample size, the data on salaries has a low response rate. In all, approximately 3251 of 27,458 students responded, for a response rate of 11.8%. This means two things: first, there is significantly less power to determine differences (particularly for those groups that are a relatively small percentage of the entire data set), and second, that the results are further suspect because of the possibility that the data are missing non-randomly (that is, that a student's characteristics-like race or school type-would affect their probability of answering the salary question in the first place).
141. See Wilkins, supra note 6, at 1936.
142. GITA Z. WILDER, LAW SCHOOL DEBT AMONG NEW LAWYERS: AN AJD MONOGRAPH 3 (2007), available at http://www.nalp.org/assets/ 645_ajddebtmonograph2007final.pdf (demonstrating that Hispanic students have higher median and mean debt levels, and black law students have higher mean debt levels). Black and Hispanic law students are also significantly more like to graduate with debt than white law students.
143. Id. at 9.
144. This number provides the change in probability between the given
145. Unlike Tables IA through 3B, Tables 4 and 5 report the actual graduation or bar passage rate, rather than the change in the rate relative to the baseline.
146. This number provides the absolute probability of graduation for black students, holding credentials at the specified value.
147. This number provides the absolute probability of passing the bar for black students, holding credentials at the specified value.
148. This number provides the absolute probability of obtaining a well-paying job for black students, holding credentials at the specified value.
149. Historically Black Schools are, as others have recognized, a special case. See Sander, supra note 5, at 416;
150. Ayres & Brooks, supra note 15, at 1825-26. Because of this, it is important not to overemphasize the results of the model with respect to these schools. However, this result may be in part due to the greater diversity in these schools, differentially leading to higher white student performance. This is mere speculation; research into different law school cultures and how they affect student performance is outside the scope of this Essay.
151. In their critique of Sander's analysis, Chambers et al. make the point that these assumptions are unrealistic. Chambers et al., supra note 21, at 1888. Any bias, however, will tend to underestimate the negative effect on black students of getting rid of affirmative action; it is highly unlikely that more black students will apply and matriculate at law schools absent affirmative action.
152. Sander does not break out his estimate in the grid model in the text of his article; the model I use is based upon Table 6.2 in his analysis. Sander, supra note 5, at 446 tbl.6.2.
153. I specifically use the term "school tier" here because this is the terminology that Sander uses, see id. at 415, and to differentiate the six "school tiers" that Sander uses in his analysis from the four "school types" that I use in my own analysis.
154. See supra notes 47-49 and accompanying text for a discussion of the school type versus school tier methodological question. In my reanalysis of Sander's model, I use Sander's assumptions with one caveat: I treat the six school tiers as six ordinal variables, rather than one interval scale variable. This flaw in Sander's original analysis,
155. see, e.g., id. at 439 tbl.5.6,
156. is one of the important criticisms of Sander's argument in the critique by Chambers et al., supra note 21, at 1872-73.
157. Underspecified regression models are models that do not control for all possible causes of performance.
158. See JAMES T. MCCLAVE ET AL., STATISTICS 503 (7th ed. 1997).
159. The primary difference between Sander's analysis and Ayres and Brooks's analysis is that Ayres and Brooks do not assume that blacks will suddenly have the same bar passage rates as white students absent affirmative action. Compare Sander, supra note 5, at 448 (performing calculations that assume that absent affirmative action, black students will have the same bar passage rates as white students with the same credentials),
160. with Ayers & Brooks, supra note 15, at 1815 (performing calculations assuming that absent affirmative action, black students will have the same bar passage rate as black students currently enrolled in the median school tier for white students with the same credentials).
161. Ayres & Brooks, supra note 15, at 1814, 1816.
162. Chambers et al., supra note 21, at 1891.
163. Sander, supra note 5, at 473. 89 In order to test any theory and determine whether the result is due to chance, one must have both an estimated value (a "statistic") that provides an estimate of how the theory would alter current numbers (for example, there would be a 7.9% increase in the number of new black lawyers each year) and a measure of the inherent variability of the statistic.
164. See Sander, supra note 5, at 471-72
165. (relying on Linda Wightman, The Threat to Diversity in Legal Education: An Empirical Analysis of the Consequences of Abandoning Race as a Factor in Law School Admission Decisions, 72 N.Y.U. L. REV. 1, 9 (1997)).
166. Wightman, supra note 90, at 9.
167. Sander, supra note 5, at 473.
168. Wightman herself notes the caveats to her model, which assumes that black students will not change application or matriculation behavior. Wightman, supra note 90, at 22-23.
169. In essence, affirmative action becomes the "treatment" that minority students receive; in the thought experiment that one would optimally perform, one would assign individuals randomly to different school types. This would be akin to a particular medical intervention in a clinical trial, See Rubin, supra note 27, at 689 (discussing general "treatments" in the context of both randomized experiments and observational studies).
170. I do not require a student to pass the bar in order to obtain a well-paying job.
171. The grid is created based upon LSAT and UGPA percentiles. Thus, the cut points for LSAT are the twentieth, fortieth, sixtieth, and eightieth percentiles, and the cut points for UGPA are the thirty-third and sixty-seventh percentiles.
172. Chambers et al. point out that the 14.1% figure, based on 2001 data, is a very low percentage compared with other years. Between 1991 and 2004, the percentage of black applicants who would not be accepted at any law school absent affirmative action ranged from 9.1% to 52.5%. Chambers et al., supra note 21, at 1861 tbl.1. In addition, the assumption that the bottom 14.1% of applicants would not be accepted is not completely realistic. Based in part on the randomness of the application process, and in part on unmeasured credentials, some individuals with higher LSAT and UGPA scores will not be admitted, while some within the bottom 14.1% will be admitted to some law schools. Both of these assumptions, like the other assumptions I make regarding the model, would tend to underestimate the negative effects of getting rid of affirmative action, and therefore provide a conservative estimate of what the change would entail.
173. This represents about half of the 14.1% of underrepresented students that Sander assumes would not matriculate at any law school absent affirmative action, Sander, supra note 5, at 473 tbl.8.2.
174. In addition, I must re-normalize the probabilities of matriculation to make sure that the new probabilities add to 100% (not more). To do so, I assume that students prefer to go to higher ranked schools, using school type as the ranking system.
175. I assume no change in the non-underrepresented group of applicants, which means that more students would matriculate to higher ranked schools overall than under current policy. This difference, however, is small, as the number of underrepresented matriculants is small overall.
176. The bootstrap is an extremely useful statistical tool that allows one to calculate standard errors (as well as other quantities of interest) non-parametrically, that is, without making restrictive parametric assumptions, such as the assumption that the data follow a normal distribution. It is particularly useful in complicated models where small deviations from initial parametric assumptions can become magnified into large errors in measurement. See BRADLEY EFRON & ROBERT J. TIBSHIRANI, AN INTRODUCTION TO THE BOOTSTRAP (1994) for details about the bootstrap procedure for determining standard errors.
177. Note that the small drop of sixteen lawyers from affirmative action to affirmative action plus (about a 2% decrease) is not statistically significant; there is no evidence that this drop is anything but random variation.
178. xv is the correlation between X and Y. See DENNIS WACKERLY ET AL., MATHEMATICAL STATISTICS WITH APPLICATIONS 228 thm.5.12 (5th ed. 1996).
179. This range represents a 95% confidence interval for the percentage loss of new black lawyers absent affirmative action. This means that if one were to rerun the entire experiment with new data one hundred times, on average ninety-five of those times the true value of the percentage decline in new black lawyers would be within the confidence interval estimated. See MCCLAVE ET AL., supra note 84, at 255.
180. The absolute number of lawyers in Sander's model is larger because my model predicts bar passage for a subset of the LSAC data-respondents whose race is known-while Sander's model predicts bar passage for the larger group of all respondents. Because neither model predicts the total number of lawyers outside the LSAC sample, what is most relevant is the percentage change in numbers, rather than the absolute numbers.
181. In 1991, Asian law students represented 4.6% of the LSAC random sample of entering law students, and approximately 2.9% of the general population. See WIGHTMAN, USER'S GUIDE, supra note 49, at 6 tbl.2 (listing the percentage of Asian law students who were sent a survey);
182. 1990 U.S. Census Data, Table P006, available at http://factfinder.census.gov/servlet/DTTable?_bm=y&-geo_id= 01000US&-ds_name=DEC_1990_STF1_&-_lang=en&-_caller= geoselect&-state=dt&-format=&-mt_ namc-=DEC_1990_STFl_P006 (listing 7,273,662 Asian Americans and 248,709,873 total Americans). Thus, Asian law students were not particularly underrepresented in the data, but were not a large group of the students surveyed by LSAC.
183. In order to compare my results with those of Sander more directly, 1 simulate Sander's grid model as well. Due to the random variation in the simulation, I estimate a slightly larger increase in the number of new black lawyers absent affirmative action-9.1%, specifically. By simulating the model several times, I am also able to estimate a 95% confidence interval for this number: 9.1% ±3.8%. See supra Table 7.
184. LSAC, BAR PASSAGE STUDY DATASET, http://bpsdata.lsac.org/ (last visited June 14, 2007) [hereinafter LSAC DATA].
185. Because of the low response rate on the salary question in the data, the error in the estimates, and consequently the confidence interval, is quite large. It is still clear, however, that the model predicts a statistically significant drop in the number of black law graduates who obtain well-paying jobs, which is also substantively important.
186. See LSAC DATA, supra note 108 (showing that the first-year law school GPA for black law students is, on average, one standard deviation lower than the first-year law school GPA for white students).
187. Ho, Scholarship Comment, supra note 21, at 2000-02. Post-treatment bias is the bias that results from including in a regression variables that occur after the treatment began (post-treatment variables) and therefore may have been affected by the treatment. In Sander's case, the main culprit is law school grades, which are determined after the treatment of matriculating at a specific school type.
188. The clusters were created based upon a statistical procedure called cluster analysis, which groups items based on their similar characteristics. The researcher determines which characteristics are pertinent to the grouping. See WIGHTMAN, LSAC STUDY, supra note 51, at 8-9 & n.20, for a more complete description of the clusters.
189. See WIGHTMAN, USER'S GUIDE, supra note 49, at 15.
190. Both the Mid-range Public and Mid-range Private school clusters contain 50 schools.
191. Recall that I cannot test which race-based barriers are present in a given school, but would be able to test the cumulative effect of race-based barriers at each school with information about specific school rank.
192. For example, the Census Bureau allows researchers to access detailed data at a few census facilities across the country. Researchers must officially be sworn in as census employees to use the data. U.S. DEPARTMENT OF COMMERCE, RESEARCH OPPORTUNITIES AT THE CENSUS BUREAU, DIR/01-RFP, at 4 (July 2001), available at http://www.census.gov/prod/2001pubs/dir-01rfp.pdf Care must still be taken, however, to protect confidentiality in the final product. In one extreme example, the United States Department of Justice issued a report with over 450 tables of information about less than 1000 observations. This level of detail allowed an entrepreneurial researcher to reverse-engineer the tables in order to reconstitute the private data.
193. See David J. Algranati, Exploring Racial and Geographical Effects in the Decision to Seek the Federal Death Penalty, 1995-2000 (Dec. 2002) (unpublished doctoral thesis, H. John Heinz III Sch. of Pub. Policy & Mgmt, Carnegie Mellon University).
194. One can also infer something about a student's range of choices for law school based upon which law school the student attended, but this is a weaker inference, as there are many reasons unrelated to credentials why a student might choose to attend a given school.
195. Ho uses propensity scores in his analysis of the data. Ho, Social Science Critique, supra note 21, at 8-9. To my knowledge, no one has used instrumental variables, Both are statistical techniques that attempt to allow causal inferences from observational data.
196. See Rubin, supra note 27, at 700 (discussing matching);
197. WILLIAM H. GREENE, ECONOMETRIC ANALYSIS 397-98 (5th ed. 2002) (discussing instrumental variables).
198. This ranking may be as simple as three groups: no, maybe, and yes.
199. Details would have to be worked out, but two main things are clear: students would not actually be admitted to the law schools, and the significant expense of looking through many more files would need to be compensated in some way.
200. See generally Andrew D. Martin & Kevin M. Quinn, Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953-1999, 10 POL. ANALYSIS 134 (2002) (providing a statistical method to rank individuals based on yes or no answers to a series of related questions).
201. This would also require admissions committees to compare 1991 applicants to current applicants. One would have to assume that the relative ranking of students would remain reasonably constant across this time period.
202. Like the school identification information, the specific state or states in which each student passed the bar was in the original, non-public data set. Again, if this data is available in any form, it reduces the need to perform an entirely new study.
203. The Educational Diversity Project, supra note 3.