Is affirmative action responsible for the achievement gap between black and white law students? A correction, a lesson, and an update

Northwestern University Law Review

Volumn 105, Issue 2, 2011, Pages 791-812

  Barnes, Katherine Y ^a  

^a UNIVERSITY OF ARIZONA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 81355141868     PISSN: 00293571     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (71)

1. Katherine Y. Barnes, Is Affirmative Action Responsible for the Achievement Gap Between Black and White Law Students?, 101 Nw. U. L. REV. 1759 (2007).
2. Richard H. Sander, A Systemic Analysis of Affirmative Action in American Law Schools, 57 STAN. L. REV. 367 (2004). Many of the replies to Professor Sander questioned his methods and results.
3. See, e.g., Ian Ayres & Richard Brooks, Response, Does Affirmative Action Reduce the Number of Black Lawyers?, 57 STAN. L. REV. 1807 (2005)
4. David L. Chambers et al., The Real Impact of Eliminating Affirmative Action in American Law Schools: An Empirical Critique of Richard Sander's Study, 57 STAN. L. REV. 1855 (2005)
5. Michele Landis Dauber, The Big Muddy, 57 STAN. L. REV. 1899 (2005)
6. Daniel E. Ho, Reply, Affirmative Action's Affirmative Actions: A Reply to Sander, 114 YALE L.J. 2011 (2005)
7. Daniel E. Ho, Scholarship Comment, Why Affirmative Action Does Not Cause Black Students to Fail the Bar, 114 YALE L.J. 1997 (2005) (reviewing Sander, supra note 2)
8. Jesse Rothstein & Albert Yoon, Mismatch in Law School (2009), http://gsppi.berkeley.edu/faculty/jrothstein/workingpapers/rothstein_yoo n_may2009.pdf. Others questioned the conclusions that should be drawn from his results.
9. See, e.g., David B. Wilkins, A Systematic Response to Systemic Disadvantage: A Response to Sander, 57 STAN. L. REV. 1915 (2005). Professor Sander replied to several critics directly.
10. E.g., Richard Sander, Reply, A Reply to Critics, 57 STAN. L. REV. 1963 (2005)
11. Richard Sander, Response, Mismeasuring the Mismatch: A Response to Ho, 114 YALE L.J. 2005 (2005). There are also several other articles that are not empirical in nature.
12. See, e.g., Kevin R. Johnson & Angela Onwuachi-Willig, Cry Me a River: The Limits of "A Systemic Analysis of Affirmative Action in American Law Schools," 7 AFR.-AM. L. & POL'Y REP. 1, 4 (2005) (arguing that a more appropriate response to the achievement gap between black and white students would be "an open and honest dialogue about the problem of minority underrepresentation in law schools").
13. The response to this Revision describes the mismatch hypothesis as meaning that students with low credentials will learn less in classrooms aimed at the middle student.
14. See E. Douglass Williams et al., Revisiting Law School Mismatch: A Comment on Barnes (2007, 2011), 105 NW. U. L. REV. 813, 813 (2011). This is imprecise: for mismatch to result in worse outcomes, low credential students must learn less than they would have at another school, not less than the middle-range students at their current institution.
15. Barnes, supra note 1, at 1770.
16. Optimally, double coding requires two different people independently coding the same general algorithm using two different statistical packages and then making sure that the output matches. This is a time-consuming and expensive safeguard, but it can detect most errors before they are published. This tests whether the logic of the algorithm is implemented correctly and helps find errors in the logic itself.
17. Transparency is the best way to curtail errors. For example, Professor Sander makes all of his programs and data available on his website, as I also have done for this Revision.
18. See KATHIE BARNES DATA SETS, http://www.law.arizona.edu/faculty/barnesDataSets.cfm (last visited June 26, 2011). Although I commend this practice because it facilitates replication, law reviews should not leave the decision whether to make this information available to the individual researcher. Moreover, in the interest of transparency, all modeling decisions should be explicit. Unfortunately, in the context of a law review article written for a nontechnical audience, including every specific decision is unrealistic, but technical appendices, published on the law review's website, can help offset this concern.
19. See Data Sets for Northwestern University Law Review 105:2, NW. U. L. REV. (Oct. 1, 2011), http://www.law.northwestern.edu/lawreview/issues/105.2.data.html (calculations by Katherine Barnes)
20. see also KATHIE BARNES DATA SETS, supra note 7 (providing same data).
21. Barnes, supra note 1, at 1767-68.
22. Ibid. at 1763-65. Although I use the term "race-based barriers," I cannot make the claim that these barriers are caused by race; instead, my model measures the association of race, school types, and their interaction with student outcomes.
23. Ibid. at 1796-1800
24. see Sander, supra note 2, at 368.
25. Barnes, supra note 1, at 1801-06.
26. Ibid. at 1801.
27. The four school types are historically black schools, low-range schools, mid-range schools, and top 30 schools.
28. Ibid.
29. All report results are from logistic regression models, which predict the probability of a positive outcome (graduation, bar passage, or obtaining a well-paying job) given a flexible function form for student credentials (allowing up to cubic powers of LSAT, UGPA, and their interactions-nine variables), race (white, black, Asian, other), school type, school type × race interactions (nine variables), and school type × credentials interactions (twenty-seven variables).
30. Barnes, supra note 1, at 1769.
31. Ibid. at 1807.
32. See Sander, supra note 2, at 393 (providing an equation for the academic index, which is a weighted average of LSAT and UGPA scores). Professor Williams and his coauthors state that I used separate values for the fifth percentile of LSAT and fifth percentile of UGPA.
33. Williams et al., supra note 4, at 818. This is incorrect. As my original essay stated, for the fifth percentile, I compared against students whose "credentials are in the fifth percentile of the entire data set."
34. Barnes, supra note 1, at 1776 n.61. Indeed, this value is far below the fifth percentile of overall student credentials because most students who have the LSAT scores in the fifth percentile also have better UGPAs and vice versa. Indeed, less than three-quarters of one percent of students have credentials at or below both fifth percentile levels. Professor Williams and his coauthors report significance levels for tests comparing against top 30 schools, which demonstrate that top 30 schools often have statistically significant difference outcomes from other school types.
35. Williams et al., supra note 4, at 827-28 tbl.3. However, this formulation of testing obscures the fact that the remaining school types HBS, low-range schools, and lid-range schools-generally do not have significantly different outcomes.
36. This number provides the change in probability between the given characteristic and the control, holding credentials at the specified value and race at its modal value, white.
37. This number provides the change in probability between the given characteristic and the control, holding all other factors at their median or modal value (LSAT = 37; UGPA = 3.3; race = white).
38. The bar passage results published in this Revision contain a subtle error in the way that bar passage was defined. As Professors Williams, Sander, Luppino, and Bolus point out in their response, the bar passage results drop those individuals who graduated from law school but chose not to take a bar exam; these individuals should have been coded as not passing a bar exam.
39. See Williams et al., supra note 4, at 817-18. Correcting this coding error, the bar passage results differ slightly, but the conclusions from the results remain the same. The professors alerted me to this error in a draft of their response provided in January 2011, and I responded by providing new results after fixing my coding error. Unfortunately, the Northwestern University Law Review editors made the decision not to allow me to report the corrected results in this publication. My understanding is that their decision is based upon the timing of the editing process, which was delayed at many points in the publication process, several of which were my fault, and a prior understanding that the professors could rely on the model and results from my earlier draft in writing their response. Although I believe that their decision to publish incorrect results does not reflect best practices, I have chosen to allow publication of my Revision with incorrect results to engage Professor Williams and his coauthors on the mismatch debate itself. I urge readers to disregard the bar passage results published here and instead rely upon the correct results, published on SSRN
40. see Katherine Y. Barnes, Is Affirmative Action Responsible for the Achievement Gap between Black and White Law Students? A Correction, a Lesson, and an Update (Aug. 12, 2011), http://ssrn.com/abstract=1908530, available on my website
41. see KATHIE BARNES DATA SETS, supra note 7, and available at the Northwestern University Law Review's website
42. see Data Sets for Northwestern University Law Review 105:2, supra note 8. This affects only the bar passage results, which are contained in Tables 2A-B, 5, and 7.
43. This number provides the change in probability between the given characteristic and the control, holding credentials at the specified value and race at its modal value, white.
44. This number provides the change in probability between the given characteristic and the control, holding all other factors at their median or modal value (LSAT = 37; UGPA = 3.3; race = white). Bar passage is defined in the same way as in Table 2A. See supra note 23.
45. Professor Williams and his coauthors note that the results are based upon the definition of a"well-paying job" as a job that pays more than $50,000 per year rather than the $40,000 per year stated in the 2007 essay. The results are essentially unchanged across the two cutoffs. Results with the $40,000 cutoff are available from the Northwestern University Law Review
46. see Data Sets for Northwestern University Law Review 105:2, supra note 8, and my website
47. see KATHIE BARNES DATA SETS, supra note 7.
48. One comparison, between black students and white students who have credentials at the fiftypercent level and have matriculated to Top 30 schools, is statistically significant at the five-percent level. The fact that black students at high-ranked schools are more likely to report obtaining a wellpaying job does not support the mismatch hypothesis.
49. Professor Williams and his coauthors note that, in my 2007 essay, I reported an unweighted population percentage for the percentage of students who obtained a well-paying job.
50. See Williams et al., supra note 4, at 821 (citing Barnes, supra note 1, at 1775). This was an error: using the weighted average is correct in this situation. This does not, however, affect the results of my logistic regression model, for which using unweighted values is the most appropriate method.
51. See Charles F. Manski & Daniel McFadden, Alternative Estimators and Sample Designs for Discrete Choice Analysis, in STRUCTURAL ANALYSIS OF DISCRETE DATA WITH ECONOMETRIC APPLICATIONS 2 (1981).
52. This number provides the change in probability between the given characteristic and the control, holding credentials at the specified value and race at its modal value, white.
53. This number provides the change in probability between the given characteristic and the control, holding all other factors at their median or modal value (LSAT = 37; UGPA = 3.3; race = white).
54. This number provides the probability of graduation for black students matriculating to given school types holding credentials at the specified value.
55. This number provides the probability of bar passage for black students matriculating to given school types holding credentials at the specified value.
56. This number provides the probability of obtaining a well-paying job for black students matriculating to given school types holding credentials at the specified value.
57. The four policies are status quo affirmative action; no affirmative action, in which minority applicants are admitted to institutions based on the probability that a white student with the same credentials would be admitted, assuming that the bottom 14% of minority students would not matriculate to a law school; affirmative action "light," which provides only half the boost in admission rates that minorities currently receive and assumes that 7% of minority students who would otherwise have matriculated to a law school would be denied admission; and affirmative action "plus," which provides twice the boost that minority applicants received. Professor Williams and his coauthors point out in their response that I assume that 14% of underrepresented minority students, rather than 14% of black students, would not matriculate to law schools absent affirmative action. The results remain essentially the same using only black students. Results are available from the Northwestern University Law Review
58. see Data Sets for Northwestern University Law Review 105:2, supra note 8, and from my website
59. see KATHIE BARNES DATA SETS, supra note 7.
60. "s.e." stands for "standard error," which is a measure of how much variability there is in the given statistic across repetitions of the same simulation.
61. Williams et al., supra note 4, at 822.
62. Ibid. at 814.
63. Ibid. at 817-18.
64. See supra note 7. I again urge readers to rely upon the corrected results provided at SSRN
65. Barnes, supra note 23, and at my website
66. KATHIE BARNES DATA SETS, supra note 7.
67. Williams et al., supra note 4, at 818.
68. Ibid. at 821.
69. Sander, supra note 2.
70. Barnes, supra note 1, at 1801-06.
71. Ibid. at 1806-08.