Does unconscious racial bias affect trial judges?

Notre Dame Law Review

Volumn 84, Issue 3, 2009, Pages 1195-1246

  Rachlinski, Jeffrey J ^a   Johnson, Sheri Lynn ^a   Wistrich, Andrew J ^b   Guthrie, Chris ^c  

^a MIT Sloan School of Management   (United States)

^b Central District of California ^*  (United States)

^c UNIVERSITY OF CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 67649556391     PISSN: 07453515     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (249)

1. Ian Ayres and Joel Waldfogel, A Market Test for Race Discrimination in Bail Setting, 46 STAN. L. REV. 987, 992 (1994). To calculate this disparity, Ayres and Waldfogel controlled for eleven other variables, but they conceded that they might still be missing one or more omitted variables that might explain the differential. Id. By comparing differences in both bond rates and bail rates, however, they were able to provide even more compelling evidence that the bail rate differences they observed were racebased. Id. at 993.
2. David B. Mustard, Racial, Ethnic, and Gender Disparities in Sentencing: Evidence from the U. S. Federal Courts, 44 J. L. and ECON. 285, 300 (2001).
3. R. Richard Banks et al., Discrimination and Implicit Bias in a Racially Unequal Society, 94 CAL. L. REV. 1169, 1175 (2006).
4. See Christine Jolls and Cass R. Sunstein, The Law of Implicit Bias, 94 CAL. L. REV. 969, 969-70 (2006) (providing examples of both explicit and implicit bias).
5. See PAUL M. SNIDERMAN and THOMAS PIAZZA, BLACK PRIDE AND BLACK PREJUDICE 6-8 (2002).
6. Anthony G. Greenwald and Linda Hamilton Krieger, Implicit Bias: Scientific Foundations, 94 CAL. L. REV. 945, 951, 961 (2006) ("[E]vidence that implicit attitudes produce discriminatory behavior is already substantial and will continue to accumulate." (footnote omitted));
7. Kirstin A. Lane et al., Implicit Social Cognition and Law, 3 ANN. REV. L. and SOC. SCI. 427, 433 (2007) (calling implicit social cognitions "robust" and "pervasive" ).
8. See Jerry Kang and Mahzarin R. Banaji, Fair Measures: A Behavioral Realist Revision of "Affirmative Action, " 94 CAL. L. REV. 1063, 1065 (2006) (arguing that implicit bias shows that affirmative action programs are necessary to address "discrimination in the here and now" (emphasis omitted)).
9. Jerry Kang, Trojan Horses of Race, 118 HARV. L. REV. 1489, 1512 (2005).
10. In addition to the Implicit Association Test, which we discuss in detail, researchers have used subliminal priming techniques, see, e.g., Sandra Graham and Brian S. Lowery, Priming Unconscious Racial Stereotypes About Adolescent Offenders, 28 L. and HUM. BEHAV. 483, 487-88 (2004);
11. reaction-time studies, see, e.g., Greenwald and Krieger, supra note 6, at 950-53 (labeling studies of implicit bias as studies of biases in reaction times);
12. and novel brain-imaging techniques, see, e.g., Elizabeth A. Phelps et al., Performance on Indirect Measures of Race Evaluation Predicts Amygdala Activation, 12 J. COGNITIVE NEUROSCI. 729, 729-30 (2000).
13. Alexander R. Green et al., Implicit Bias Among Physicians and Its Prediction of Thrombolysis Decisions for Black and White Patients, 22 J. GEN. INTERNAL MED. 1231, 1231-32 (2007).
14. See Greenwald and Krieger, supra note 6, at 952.
15. See, e.g., Michael Orey, White Men Can't Help It, Bus. WK., May 15, 2006, at 54 (discussing the role of expert witness testimony on "unconscious bias theory" in gender and race employment discrimination cases);
16. Diane Cole, Don't Race to Judgment, U. S. NEWS AND WORLD REP., Dec. 26, 2005/Jan. 2, 2006, at 90.
17. See Project Implicit, General Information, http://www. projectimplicit.net/generalinfo.php (last visited Mar. 9, 2009) ("Visitors have completed more than 4.5 million demonstration tests since 1998, currently averaging over 15, 000 tests completed each week.").
18. Greenwald and Krieger, supra note 6, at 952-53 (describing the basic IAT technique).
19. See Online Psychology Laboratory, Implicit Association Test (Race), http://opl.apa.org/Experiments/About/AboutIATRace.aspx (last visited Mar. 9, 2009).
20. See id.
21. See Brian A. Nosek et al., Harvesting Implicit Group Attitudes and Beliefs from a Demonstration Web Site, 6 GROUP DYNAMICS 101, 105 (2002) (reporting data indicating that white adults taking the IAT strongly favored the white/good versus the black/bad pairing on the IAT).
22. Id. at 104.
23. Id. at 105.
24. Id.
25. Id. Throughout, we adopt the convention that a "strong" bias means a tendency to favor one pairing over another on the IAT by over three-quarters of a standard deviation, a "small" bias means an effect of less than one-quarter of a standard deviation, and a "moderate" effect means an effect that is in between one-quarter and three-quarters of a standard deviation.
26. Id.
27. Id.
28. Id. at 110.
29. See Hal R. Arkes and Philip E. Tetlock, Attributions of Implicit Prejudice, or "Would Jesse Jackson 'Fail' the Implicit Association Test?, "15 PSYCHOL. INQUIRY 257, 257-58 (2004) (arguing that the IAT does not measure bias or prejudice) ;
30. Mahzarin R. Banaji et al., No Place for Nostalgia in Science: A Response to Arkes and Tetlock, 15 PSYCHOL. INQUIRY 279, 279 (2004) (responding to the arguments of arkes and tedock).
31. SEE J. RIDLEY STROOP, STUDIES OF INTERFERENCE IN SERIAL VERBAL REACTIONS, 18 J. EXPERIMENTAL PSYCHOL. 643, 659-60 (1935) (presenting evidence that words colored differently from their semantic meaning are difficult to read).
32. See Project Implicit, supra note 13.
33. Id.
34. See Anthony G, Greenwald et al., Understanding and Using the Implicit Association Test: I. An Improved Scoring Algorithm, 85 J. PERSONALITY and SOC. PSYCHOL. 197, 209-11 (2003) (discussing mechanisms for reducing order effects);
35. see also Anthony G. Greenwald 8c Brian A. Nosek, Health of the Implicit Association Test at Age 3, 48 ZEITSCHRIFT FÜR EXPERIMENTELLE PSYCHOLOGIE 85, 87 (2001) ("Subject handedness was found to have essentially zero relation to magnitude of the race IAT effect.").
36. See, e.g., Samuel R. Bagenstos, Implicit Bias, "Science, " and Antidiscrimination Law, 1 HARV. L. and POL'Y REV. 477, 477 (2007);
37. Greenwald et al., supra note 29, at 199-200.
38. Anthony G. Greenwald et al., Understanding and Using the Implicit Association Test: III. Meta-Analysis of Predictive Validity, J. PERSONALITY and SOC. PSYCHOL, (forthcoming 2009).
39. Note that some of the papers Greenwald and his co-authors include in their analysis report multiple studies using independent samples of subjects. Id. (manuscript at 10, 21).
40. Id. (manuscript at 21).
41. To be precise, the square of the correlation coefficient of 0.24 is 0.0576, which we round up to 6%.
42. See NATL CTRFOR STATE COURTS, EXAMINING THE WORKOF STATE COURTS, 2006, at 45-46 (Robert C. LaFountain et al. eds., 2006), http://www.ncsconline.org/D- Research/csp/2006-files/EWSC-2007WholeDocument.pdf (providing data for criminal cases entering state courts in 2005).
43. ADMIN. OFF. OF THE U. S. COURTS, FEDERAL JUDICIAL CASELOAD STATISTICS: MARCH 31, 2007, at 58 tbl. D (2007), http://www.uscourts.gov/ caseload2007/tables/D00CMar07.pdf (observing U. S. district courts to have 71, 652 and 69, 697 cases pending in the twelve-month periods ending March 31, 2006 and 2007, respectively).
44. Kang and Banaji, supra note 7, at 1073.
45. See Jennifer Eberhardt et al., Looking Deathworthy: Perceived Stereotypicality of Black Defendants Predicts Capital-Sentencing Outcomes, 17 PSYCHOL. SCI. 383, 384 (2006) ("Defendants whose appearance was perceived as more stereotypically Black were more likely to receive a death sentence than defendants whose appearance was perceived as less stereotypically Black.").
46. See Jack Glaser and Eric D. Knowles, Implicit Motivation to Control Prejudice, 44 J. EXPERIMENTAL SOC. PSYCHOL. 164, 164-65, 170-71 (2008).
47. See Bridget C. Dunton and Russell H. Fazio, An Individual Difference Measure of Motivation to Control Prejudiced Reactions, 23 PERSONALITY and SOC. PSYCHOL. BULL. 316, 324-26 (1997) ;
48. E. Ashby Plant and Patricia G. Devine, Internal and External Motivation to Respond Without Prejudice, 75 J. PERSONALITY and SOC. PSYCHOL. 811, 824-28 (1998).
49. See John A. Bargh, The Cognitive Monster: The Case Against the Controllability of Automatic Stereotype Effects, in DUAL- PROCESS THEORIES IN SOCIAL PSYCHOLOGY 361, 375-78 (Shelly Chaiken and Yaacov Trope eds., 1999) ;
50. Patricia G. Devine et al., The Regulation of Explicit and Implicit Race Bias: The Role of Motivations to Respond Without Prejudice, 82 J. PERSONALITY and SOC. PSYCHOL. 835, 845-47 (2002) ;
51. John F. Dovidio et al., On the Nature of Prejudice: Automatic and Controlled Processes, 33 J. EXPERIMENTAL SOC. PSYCHOL. 510, 535-36 (1997) ;
52. Russell H. Fazio et al., Variability in Automatic Activation as an Unobtrusive Measure of Racial Attitudes: A Bona Fide Pipeline?, 69 J. PERSONALITY and SOC. PSYCHOL. 1013, 1025-26 (1995).
53. Green et al., supra note 10.
54. Id. at 1232-33.
55. Id. at 1235. The researchers also found that white doctors who express white preferences on the IAT were more likely to diagnose black patients than white patients as having coronary artery disease, based upon the same symptoms. Id. at 1234-35. Indeed, the doctors offered the appropriate treatment-thrombolysis-to an equal number of black patients as white patients! Id. As the authors rightly point out, this does not mean there was no disparity; among patients who were diagnosed as suffering from coronary artery disease, black patients were less likely to be offered the appropriate treatment. Id. It is at least curious, however, that doctors with implicit white preferences would be more likely to diagnose coronary artery disease for black patients than white patients, but less likely to treat it. The diagnosis disparity runs in the opposite direction of the treatment-for- diagnosis disparity, and ultimately, the two effects actually cancel each other out. Id. at 1236-37. Of course, if doctors behaved the same way in the real world, black and white patients who presented the same symptoms would be treated in the same way. Thus, though the IAT predicted discriminatory acts, implicit bias does not seem to result in discrimination overall. Id. at 1234-37. This aspect of the study has been the source of some debate. See John Tierney, In Bias Test, Shades of Gray, N. Y. TIMES, NOV. 18, 2008, at Dl. One other recent study also shows no correlation between measures of implicit bias and medical decisions among physicians. See Janice A. Sabin et al., Physician Implicit Attitudes and Stereotypes About Race and Quality of Medical Care, 46 MED. CARE 678, 682 (2008) ("We did not find a relationship between difference in treatment recommendations by patient race and implicit measures.").
56. Green et al., supra note 10, at 1235.
57. Id. at 1237.
58. Glaser and Knowles, supra note 39, at 167-71.
59. Joshua Correll et al., The Police Officer's Dilemma: Using Ethnicity to Disambiguate Potentially Threatening Individuals, 83 J. PERSONALITY and Soc. PSYCHOL. 1314, 1315-17 (2002).
60. Id. at 1315-16.
61. Id. at 1320.
62. Id. at 1320-21; Glaser and Knowles, supra note 39, at 168-69.
63. Glaser and Knowles, supra note 39, at 169-70.
64. Id. at 171.
65. Robert W. Livingston, When Motivation Isn't Enough: Evidence of Unintentional Deliberative Discrimination Under Conditions of Response Ambiguity 9-10 (2002) (unpublished manuscript, on file with the Notre Dame Law Review).
66. See Arnd Florack et al., Der Einfluss Wahrgenommener Bedrohung auf die Nutzung Automatischer Assoziationen bei der Personenbeurteilung [The Impact of Perceived Threat on the Use of Automatic Associations in Person Judgments], 32 ZEITSCHRIFT FÜR SOZIALPSYCHOLOGIE 249 (2001).
67. Id. at 255 tbl.l.
68. We recognize that we have emphasized disparities concerning black Americans, rather than other races. We have done so for three reasons. First, even though Latinos, Native Americans, and Asian Americans are also targets of racism, both explicit and implicit, in the United States some of the most striking disparities involve black Americans in the legal system. Second, the research on the IAT has emphasized biases concerning black Americans as well. Third, our sample of judges includes a large group of black American judges, but few Latinos, few Asian Americans, and no Native Americans. We thus cannot draw any conclusions about the reactions of judges of these ethnicities. We therefore focus our attention here on biases involving black Americans.
69. See Chris Guthrie et al., Blinking on the Bench: How Judges Decide Cases, 93 CORNELL L. REV. 1, 13 (2007) [hereinafter Guthrie et al., How Judges Decide]; Chris Guthrie et al., Inside the Judicial Mind, 86 CORNELL L. REV. 777, 814-15 (2001) [hereinafter Guthrie et al., Judicial Mind]; Jeffrey J. Rachlinski et al., Inside the Bankruptcy Judge's Mind, 86 B. U. L. REV. 1227, 1256-59 (2006) ;
70. Andrew J. Wistrich et al., Can Judges Ignore Inadmissible Information? The Difficulty of Deliberately Disregarding, 153 U. PA. L. REV. 1251, 1323-24 (2005).
71. At two of the conferences, we collected data from judges attending a plenary session. At the third, we collected data from judges attending an optional session.
72. Their concerns might be justified. Some of our previous work has been reported in the New York Times and the American Bar Association Journal, among other places.
73. See, e.g., Patricia Cohen, Judicial Reasoning Is All Too Human, N. Y. TIMES, June 30, 2001, at B7;
74. Debra Cassens Weiss, Judges Hunk Story Problem Test, Showing Intuitive Decision-Making, A. B. A. J., Feb. 19, 2008, https://abajournal.com/news/ judges-flunk-story-problem-test-showing-intuitive-decision-making/. The latter report leads with the unfortunate headline "Judges Flunk Story Problem Test, " which casts the judges in a more negative light than the data warrant. Interest in the present Article is sufficiently high that, despite our own efforts to limit its use before it was finalized, it was cited by Judge Jack Weinstein in a published opinion, United States v. Taveras, 424 F. Supp. 2d 446, 462 (E. D. N. Y. 2006), and discussed at length in a recent volume of the Annual Review of Law and Social Science, Lane et al., supra note 6, at 441-45.
75. Eighty judges attended the session at which we collected data, but we excluded ten from our study. We excluded one judge at his or her request. We excluded nine other judges because they failed to provide us with demographic information. We believe that these failures were largely accidental. To complete the demographic page, the judges had to return to the written materials after completing the final IAT, and these nine judges failed to do so. We did not realize that this process would cause problems at our presentation in the eastern jurisdiction, and hence we did not obtain this data. In the subsequent presentations, we made sure that the judges completed the last page as we collected the surveys.
76. Forty-eight judges attended the session at which we collected the data, but we excluded three from our study. One judge neglected to provide demographic information, and we lost the data for two other judges due to a computer malfunction.
77. Over ninety percent of the judges in the eastern jurisdiction attended this conference (although, as noted, we did not obtain data from all of them). Attendance was lower among the western judges; the sample includes roughly half of the judges in their jurisdiction. These judges' willingness to participate in our study was thus unlikely to have been affected by their interest (or lack thereof) in the content of the material. In fact, the judges were not aware of the subject matter of the talk before the session began. This was not our first presentation to the eastern judges. Three years earlier, we had presented a completely different set of materials to the same educational conference. Some of the results from that earlier session have been published, also without identifying the jurisdiction. Wistrich et al., supra note 58, at 1279-81. Many of the judges were therefore familiar with our methods, although the present study differs from our earlier work. Our prior work dealt largely with judicial reliance on heuristics in making judgments, whereas this research is entirely devoted to the influence of race and gender on judgment. This was our first presentation to the western judges. The regional judges differed from the eastern and western judges in that they opted not only to attend the judicial education conference at which we spoke but also to attend our optional session.
78. We include these questions below in Appendix A.
79. The computer tasks were all conducted on laptop computers rented for the purpose of running the experiment. They were all relatively contemporary machines of similar makes. At the eastern and western sessions, all were Hewlett-Packard NX9010; at the regional conference, they were IBM ThinkPads. All had fifteen-inch screens. The software to run the tasks was designed with a program called Inquisit 2.0, created specifically for measuring implicit associations by a company called Millisecond Software. See Inquisit, http://www.millisecond.com (last visited Mar. 7, 2009).
80. The instructions on the survey were as follows:
81. Many of the points to be discussed at this session are best experienced directly. We therefore ask that before the session starts, you participate in a series of exercises on the laptop computer and evaluate a series of hypothetical cases in the pages that follow. (Participation in all aspects of this exercise is voluntary, of course.) Please do not discuss these materials while you are participating. We shall collect these surveys before the discussion and present the results during the session.
82. The first part of the exercise consists of a computer task. Please do not begin the task or turn this page until asked to do so.
83. The instructions on the computer screen were:
84. JURISDICTION: Judicial Education Conference, DATE
85. We shall begin by making announcements as to the nature of this exercise.
86. Please DO NOT BEGIN until after the announcements.
87. After the announcements, please press the space bar to begin.
88. Judge Wistrich conducted the introduction at the eastern and western conferences; Professor Rachlinski did it at the regional conference.
89. We also conducted an IAT related to gender after the race IAT, but do not report those results here.
90. We also included a scenario in which we manipulated the gender of a target legal actor as the third scenario. We do not report these results here.
91. The order of the materials was thus as follows: the priming task; the written scenario of the shoplifter; the written scenario of the armed robber; the gender scenario (not reported here);
92. the battery case; the race IAT; the gender IAT (not reported here);
93. and the demographics page.
94. We analyzed the three groups of judges separately, but there were no significant differences between the judges, except as noted below, so we have kept them together throughout the analysis. Similarly, we found no differences between the judges on the basis of the gender, political affiliation, or experience. Because previous research on the IAT suggests that Latinos score somewhat closer to black Americans on the IAT we used, we combined the few Latino judges with the blackjudges for these analyses. Nosek et al., supra note 17, at 110 tbl.2. Similarly, we combined the Asian American judges with the white judges.
95. The exact instructions at the outset of the IAT were as follows:
96. The remaining computer tasks involve making CATEGORY JUDGMENTS. Once the tasks begin, a word or words describing the CATEGORIES will appear in the upper left and upper right corners of the computer screen.
97. A TARGET word or picture will also be displayed in the center of the screen, which you must assign to one of the two categories
98. Please respond AS RAPIDLY AS POSSIBLE, but don't respond so fast that you make many errors. (Occasional errors are okay.)
99. An "X" will appear when you make an error. Whenever the "X" appears, correct the mistake by pressing the other key.
100. For a more detailed account of our IAT procedure, see Appendix B.
101. See, e.g., Nosek et al., supra note 17, at 104-05 (reporting average differences in response latencies among large samples of subjects obtained through the Internet).
102. See Greenwald et al., supra note 29, at 209-10 (describing standardized measures). The full account of our scoring methods is included as Appendix C.
103. The specific statistical result was: t (82) = 4.94, p <.0001. Throughout this Article, we reserve the use of the words "significant" and "significantly" for statistical significance.
104. The specific statistical result was: t (42) = 0.18, p =.86. In conducting this test, we took the effect size among the Internet sample of 0.16 standard deviations to be the "population" effect size among black participants on the Internet, and tested whether our observed difference, with our observed standard deviation, would be likely to be reliably higher or lower than the effect in the Internet data. The priming condition did not appear to affect the judges' IAT scores. Also, the judges themselves varied somewhat in their IAT scores. White judges in the eastern jurisdiction expressed an average standardized preference of 0.33, compared to 0.48 and 0.55 in the western jurisdiction and the regional conferences, respectively. These differences were marginally significant. Because the black judges in our study were concentrated largely in the eastern jurisdiction, similar tests for variations among these judges would not be reliable.
105. The specific statistical result was: t (84) = 2.26, p =.026. We compared our results to those of the Internet sample reported in Nosek et al., supra note 17, at 105. In making this comparison, we took the effect size among the Internet sample of 0.83 standard deviations to be the "population" effect size among white participants on the Internet, and tested whether our observed difference, with our observed standard deviation, would likely be reliably higher or lower than the effect in the Internet data.
106. We selected data collection and scoring procedures so as to minimize the effects of order of presentation. Greenwald and his fellow authors reported that the effect of order of presentation is less than one percent, using the methods we followed. See Greenwald et al., supra note 29, at 210 tbl.2.
107. See id. at 200 ("IAT effects will be artificially larger for any subjects who respond slowly.").
108. Throughout this Article we follow the convention of using the terms "black" and "white" to denote race, as the terms more closely reflect the faces in the IAT, the instructions in the IAT (which refer to black and white), and might more closely reflect how the black judges would describe themselves (although there would be variation on this). When referring to the criminal defendants, however, we use African American and Caucasian, following the references mentioned in the hypothetical cases.
109. Graham and Lowery, supra note 9, at 487-88.
110. At the beginning of the task, three asterisks appeared in the center of the screen. A sixteen-character letter string then appeared in one of the four quadrants of the screen. The judges were instructed to press a specific key on the left-hand side of the computer (the "E" key, which was marked with a red dot) when the letter string appeared in one of the quadrants on the left and to press a specific key on the righthand side of the computer (the "I" key, which was also marked with a red dot) when a word appeared in one of the two quadrants on the right. Reminders as to which key to press also remained on the computer screen throughout the first task (that is, "press the 'E' key for left" and "press the T key for right"). When the judges identified the quadrant correctly, the word "correct" would appear in the center in letters. When the judges made an error, the word "error" would appear instead. In either case, the three asterisks would then replace the words "correct" or "error" and the task would repeat. The exact instructions the judges saw are below.
111. *) will appear in the center. Focus your attention on these. A string of letters will then appear in the upper-right, lower-right, upper-left, or lower-left portion of the computer screen.
112. If the string appears on the left-hand side (either up or down), press the "E" key.
113. If the string appears on the right-hand side (either up or down), press the
114. "I" key.
115. If you correctly identify the position, the screen will flash the word "correct"; if you identify the wrong position, the screen will flash the word "error." The task will then repeat a number of times. Other words may appear with the letter string. Ignore these and try to identify the position of the letters as quickly as possible.
116. When you are ready, press the space bar to begin the task.
117. Each trial thus proceeded as follows: the three asterisks would appear in the center of the screen; 1200 milliseconds later (1.2 seconds) one of the prime words (selected at random) would appear in one of the four quadrants (at random as determined by the computer);
118. milliseconds after that, the letter-string would appear over the prime; this would remain until the judge pressed either the "E" or "I" key; then either the "correct" or "error" in the center (depending upon the judge's response) and would remain for roughly one second; then the three asterisks would replace the word "correct" or "error"; and the process would repeat. Due to an error in the computer programming, the judges in the eastern conference were only exposed to the subliminal prime for sixty-four milliseconds, rather than 153 milliseconds.
119. Graham and Lowery reported that none of the officers in their study was able to identify the nature of the words being shown to them. Graham and Lowery, supra note 9, at 491. We did not ask our judges their assessment of what the words were.
120. The words came direcdy from the Graham and Lowery study: graffiti, Harlem, homeboy, jerricurl, minority, mulatto, negro, rap, segregation, basketball, black, Cosby, gospel, hood, Jamaica, roots, afro, Oprah, Islam, Haiti, pimp, dreadlocks, plantation, slum, Tyson, welfare, athlete, ghetto, calypso, reggae, rhythm, soul. Id. at 489 n. 5.
121. These words also came directly from Graham and Lowery: baby, enjoyment, heaven, kindness, summer, sunset, truth, playful, accident, coffin, devil, funeral, horror, mosquito, stress, toothache, warmth, trust, sunrise, rainbow, pleasure, paradise, laughter, birthday, virus, paralysis, loneliness, jealousy, hell, execution, death, agony. Graham and Lowery used neutral words that matched the words associated with black Americans for positive or negative associations. Id.
122. Our study differed from that of Graham and Lowery in several ways, any of which might have affected the results. First, Graham and Lowery used eighty trials, rather than the sixty we used. Id. at 489-90. Second, because we ran a large group of judges at the same time, we did not use audible beeps to indicate correct responses. Id. Third, our hypothetical defendants differed. We did not have access to the original materials Graham and Lowery used, and so wrote our own. See fact pattern infra Appendix A. Fourth, we asked fewer questions concerning the hypothetical defendants. Although we do not see how any of these differences would necessarily affect the results, priming tasks can be sensitive to details.
123. The following appeared on the screen:
124. Thank you for completing the first computer task.
125. Now please turn to the written materials.
126. Please leave this computer on with the screen up.
127. After you have completed four pages of written materials, please press the
128. space bar to continue with the final computer tasks.
129. In case a judge accidentally or mistakenly hit the space bar, we added another intervening page before the second computer task, which appeared once the space bar was pressed. It read as follows:
130. If you have completed the four case summaries, please press the space bar to begin the final computer task.
131. The location of the crime would reveal the jurisdiction and hence we delete it. The location was an upscale shopping district.
132. The exact materials for this scenario and all others are included infra Appendix A.
133. The options were as follows:
134. Dismiss it with an oral warning
135. Adjourn the case in contemplation of dismissal (assuming William gets in no further trouble)
136. Put William on probation for six months or less
137. Put William on probation for more than six months
138. Commit William to a juvenile detention facility for six months or less
139. Commit William to a juvenile detention facility for more than six months
140. Transfer William to adult court.
141. The results were as follows: Question 1, x = 0.51, p =.61; Question 2, z = 0.73, p =.46; Question 3, z =1.09, p =.28.
142. To accomplish this analysis, we conducted an ordered logit regression of the judges' disposition against the priming condition, the judges' IAT scores, and an interaction of the two. The interaction term reflects the effect of the IAT score on how the prime affected the judge. This term was marginally significant in the model, z = 1.84, p=.07.
143. For the first recidivism question, z = 1.41, p =.16. On the second recidivism question, z = 1.49, p =.14. On these questions, the black judges and the white judges seemed to respond in similar ways. We ran the full model (predictors of prime, race of judge, IAT, and all interactions between these variables) on all three variables as well. Adding the race-of-judge terms and interactions did not produce any significant effects.
144. The use of an armed robbery breaks somewhat with Graham and Lowery, who had used two simple property crimes. See Graham and Lowery, supra note 9, at 490.
145. The results were as follows: Question 1, z = 0.17, p =.87; Question 2, z = 0.09, p =.93; and Question 3, z = 1.62, p =.11.
146. Our findings were: z = 1.85, p =.06.
147. For the first recidivism question, z = 0.62, p =.53; on the second recidivism question, z = 0.54, p =.59. As above, on these questions, the black judges and the white judges seemed to respond in similar ways. We ran the full model (predictors of prime, race of judge, IAT, and all interactions between these variables) on all three variables as well. Adding the raceofjudge terms and interactions did not produce any significant effects.
148. See Graham and Lowery, supra note 9, at 493-94, 496.
149. Id. Only police officers predicted that the defendant was more likely to recidivate; parole officers did not show any differences on this question. Id.
150. Samuel R. Sommers and Phoebe C. Ellsworth, White Juror Bias: An Investigation of Prejudice Against Black Defendants in the American Courtroom, 7 PSYCHOL. PUB. POL'Y and L. 201, 216-17 (2001). We thank the authors for graciously sending us the materials and giving us permission to use them.
151. We used the same question to elicit verdicts and confidence ratings as the one Sommers and Ellsworth used: "Based on the available evidence, if this were a bench trial, would you convict the defendant?" Below this were the words 'Yes" and "No. " Finally, we asked the judges, "How confident are you that your judgment is correct?" Below this question, the materials presented a nine-point scale, with "1" labeled "Not at all Confident" and "9" labeled "Very Confident." Id. at 217; see also infra Appendix A (providing the materials used in our study).
152. This difference was not statistically significant. Fishers exact test, p =.62.
153. The difference between our results and those obtained by Sommers and Ellsworth is significant: χ2 (l) = 6.74, p <.01 (using the expected conviction rates of seventy percent for Caucasian defendants and ninety percent for African American defendants, as reported by Sommers and Ellsworth, Sommers and Ellsworth, supra note 102, at 217).
154. The analysis consisted of a logistic regression of the verdict against the race of the defendant, the race of the judge, and the interaction of these two parameters. The interaction was significant, z = 2.12, p =.03, which was the result of the differential treatment of the two defendants by the black judges. The race of the defendant was also significant, z = 2.81, p =.005, indicating that overall, the judges were less likely to convict the African American defendant than the Caucasian defendant.
155. We combined the nine-point confidence measure with the binary outcome to create an eighteen-point scale. In our coding, a "1" corresponded to ajudge who was very confident that the defendant should be acquitted, whereas an "18" corresponded to a judge who was very confident that the defendant should be convicted. The average confidence that the judges expressed in the defendant's guilt were as follows: white judges judging Caucasian defendants-13.64; white judges judging African American defendants-12.2; black judges judging Caucasian defendants-16.08; black judges judging African American defendants-9.89. Statistical analysis of these results (by ANOVA) produced results consistent with the analysis of the verdicts alone. That is, the judges were significantly more convinced of the Caucasian defendant's guilt than of the African American's guilt (F (1, 129) = 15.04, p <.001). This disparity was much more pronounced among black judges (F (1, 129) = 5.84, p <.025).
156. To accomplish this analysis, we conducted a logistic regression of the judges' verdict against the priming condition, the judges' IAT scores, and an interaction of the two. The interaction term reflects the effect of the IAT score on how the race of the defendant affected the judges' verdict. This term was not significant in the model, z = 1.04, p =.30.
157. We also replicated this analysis with the eighteen-point confidence ratings. See infra note 112. Specifically, we regressed the judges' confidence in the defendant's guilt against the defendant's race, the judges' IAT score, and the interaction between the race and IAT score. As with the verdict itself, this analysis showed that the race of the defendant was significant, t-ratio = 3.49, p <.001, but the interaction between race of defendant and IAT score was not, t-ratio = 1.51, p =.13.
158. In this analysis, the race of the defendant and the interaction between race of judge and race of the defendant were significant, just as they were in the simpler models. (Race of defendant, z = 1.99, p =.05; interaction between race of the judge and race of the defendant, z = 2.35, p =.02. The interaction of the defendant's race and IAT score was not significant, z = 1.00, p =.23.)
159. The result was as follows: z = 2.18, p =.03.
160. Regressing the eighteen-point confidence rating against the race of the judge, the race of the defendant, the judges' IAT scores, and all interactions between these variables revealed significant effects for race of the defendant, t-ratio = 2.95, p =.005; a significant interaction of race of the defendant with race of the judge, t-ratio = 2.68, p =.01; and the three-way interaction of race of judge, race of defendant, and IAT score, t-ratio = 2.68, p =.02. The interaction of race of defendant and IAT scores was still not significant in this model, t-ratio = 1.27, p =.20.
161. The results are as follows: z = 1.15, p =.25.
162. The results are as follows: z = 1.87, p =.06. Given the high conviction rate of the black judges for the Caucasian defendant, this trend actually meant that they were more likely to convict the African American defendants to the extent that they exhibited greater white preferences on the IAT.
163. The white judges displayed a greater propensity to convict the Caucasian defendant relative to the African American defendant as the IAT score increased, but the trend did not approach significance, t-ratio = 1.00, p =.40. The black judges showed the opposite trend, which was significant: t-ratio = 2.25, p =.03.
164. Siri Carpenter, Buried Prejudice: The Bigot in Your Brain, SCI. AM. MIND, May 2008, at 32, 32.
165. See Gordon B. Moskowitz and Amanda R. Salomon, Preconsciously Controlling Stereotyping: Implicitly Activated Egalitarian Goals Prevent the Activation of Stereotypes, 18 SOC. COGNITION 151, 155 (2000).
166. See Theodore Eisenberg and Sheri Lynn Johnson, Implicit Racial Attitudes of Death Penalty Lawyers, 53 DEPAUL L. REV. 1539, 1540 (2004) ("One would hope that those who represent capital defendants (or at least African-American capital defendants) would themselves be free of racialized thinking....").
167. Id. at 1546-48.
168. See Sommers and Ellsworth, supra note 102, at 217.
169. See MODEL CODE OF JUDICIAL CONDUCT, at Canon 2 (2008) ("A judge shall perform the duties of judicial office impartially, competently, and diligently.").
170. See, e.g., AM. BAR ASS'N, BLACK LETTER GUIDELINES FOR THE EVALUATION OF JUDICIAL PERFORMANCE, at Guideline 5-2.3 (2005), available at http://www.abanet.org/jd/lawyersconf/pdf/jpec-final.pdf (prescribing "[a]bsence of favor or disfavor toward anyone, including but not limited to favor or disfavor based upon race, sex, religion, national origin, disability, age, sexual orientation, or socioeconomic status").
171. See Glaser and Knowles, supra note 39, at 171.
172. During our presentation, one of us asked for a show of hands to indicate how many thought we were studying race. While not the most ideal way to make this inquiry, and while we did not keep a precise count, most of the judges raised their hands.
173. See, e.g., Kathryn Abrams, Black Judges and Ascriptive Group Identification, in NORMS AND THE LAW 208, 215 (John N. Drobak ed., 2006) ("The most noteworthy feature of these studies is that they find no consistent, and only a few salient, differences in decisionmaking that correlate with the race of the judge.").
174. See Carpenter, supra note 116, at 37-38.
175. These data were collected by us at a conference of New York City administrative law judges in the summer of 2008. As one of the questions, we asked the following:
176. Relative to the other judges attending this conference, how would you rate yourself on the following:
177. Avoiding racial bias in making decisions
178. - In the highest quartile (meaning that you are more skilled at this than 75% of the judges attending this conference)
179. - In the second highest quartile (meaning that you are more skilled at this than 50% of the judges in this room, but less skilled than 25% of the judges attending this conference)
180. - In the second lowest quartile (meaning that you are more skilled at this than 25% of the judges in this room, but less skilled than 50% of the judges attending this conference)
181. - In the lowest quartile (meaning that you are less skilled at this than 75% of the judges attending this conference).
182. Jolls and Sunstein, supra note 4, at 988-90; Kang and Banaji, supra note 7, at 1105-08.
183. See, e.g., Kang and Banaji, supra note 7, at 1112 ("In Grutter v. Bollinger, the Court emphasized that student diversity was valuable because it could help 'break down racial stereotypes.'" (quoting Grutterv. Bollinger, 539 U. S. 306, 330 (2003)));
184. see also Kang, supra note 8, at 1579-83 (arguing that public broadcasting should be regulated so as to promote positive images of minorities).
185. BUREAU OF JUSTICE STATISTICS, U. S. DEP'T OF JUSTICE, FELONY DEFENDANTS IN LARGE URBAN COUNTIES, 2004, at 1 (2004), available at http://www.ojp. usdoj.gov/bjs/pub/pdf/fdluc04.pdf (stating that an estimated forty percent of defendants were black).
186. See Eisenberg and Johnson, supra note 118, at 1553-56.
187. Others have made tentative suggestions that the IAT be used as a screening device for certain professions. See, e.g., IAN AYRES, PERVASIVE PREJUDICE? 424 (2001) ("Implicit attitude testing might also itself be used as a criterion for hiring both governmental and nongovernmental actors.").
188. Green et al., supra note 10, at 1237 ("These findings support the IAT's value as an educational tool.").
189. See id. (recommending "securely and privately administered IATs to increase physicians' awareness of unconscious bias").
190. See Carpenter, supra note 116, at 32.
191. Timothy D. Wilson et al., Mental Contamination and the Debiasing Problem, in HEURISTICS AND BIASES 185, 190 (Thomas Gilovich et al. eds., 2002).
192. See Guthrie et al., Judicial Mind, supra note 58, at 814-15.
193. See Green et al, supra note 10, at 1237.
194. Wilson et al., supra note 136, at 185.
195. Id. at 187.
196. See id. at 191 ("Three kinds of errors have been found: insufficient correction (debiasing in the direction of accuracy that does not go far enough), unnecessary correction (debiasing when there was no bias to start with), and overcorrection (too much debiasing, such that judgments end up biased in the opposite direction).").
197. See id. (suggesting that people's "corrected judgments might be worse than their uncorrected ones");
198. see also Antony Page, Batson's Blind-Spot: Unconscious Stereotyping and the Peremptory Challenge, 85 B. U. L. REV. 155, 239-40 (2005) ("One major problem for any correction strategy is determining the magnitude of the correction required. Unfortunately, people are not very good at this determination. Some research suggests that among those who are very motivated to avoid discrimination, overcorrection is a common problem.... A second problem is that a correction strategy appears to require significant cognitive resources...." (citations omitted));
199. id. at 241-42 ("'[T]o consciously and willfully regulate one's own... evaluations [and] decisions... requires considerable effort and is relatively slow. Moreover, it appears to require a limited resource that is quickly used up, so conscious self-regulatory acts can only occur sparingly and for a short time.'" (omissions in original) (quoting John A. Bargh and Tanya L. Chartrand, The Unbearable Automaticity of Being, 54 AM. PSYCHOL. 462, 476 (1999))).
200. See Christopher A. Parsons et al., Strike Three: Umpires' Demand far Discrimination 24-25 (Nat'l Bureau of Econ. Research, Working Paper Series, Paper No. 13665, 2007), available at http://ssrn. com/abstractsl077091; Joseph Price and Justin Wolfers, Racial Discrimination Among NBA Referees 30 (Nat'l Bureau of Econ. Research, Working Paper Series, Paper No. 13206, 2007), available at http://ssrn. com/abstract=997562.
201. Accountability improves performance in other contexts, so it likely would do so forjudges as well. See Jennifer S. Lerner and Philip E. Tetlock, Accounting for the Effects of Accountability, 125 PSYCHOL. BULL. 255, 270-71 (1999).
202. See Guthrie et al., How Judges Decide, supra note 58, at 32.
203. See, e.g., Jean E. Dubofsky, Judicial Performance Review: A Balance Between Judicial Independence and Public Accountability, 34 FORDHAM URB. LJ. 315, 320-22 (2007) (explaining that the judicial performance review system in Colorado focuses only on a judge's performance in a particular case).
204. See Michel E. Solimine, Congress, Ex Parte Young, and the Fate of the Threejudge District Court, 70 U. PITT. L. REV. 101, 128-134 (2008).
205. Jennifer L. Peresie, Note, Female Judges Matter: Gender and Collegial Decisionmaking in the Federal Appellate Courts, 114 YALE L. J. 1759, 1778 (2005).
206. Note, Judicial Limitation of Three-Judge Court Jurisdiction, 85 YALE L. J. 564, 564 (1976).
207. Arthur D. Hellman, Legal Problems of Dividing a State Between Federal Judicial Circuits, 122 U. PA. L. REV. 1188, 1225 (1974).
208. See Peresie, supra note 148, at 1778.
209. The faces were taken from the Project Implicit website. See Brian A. Nosek et al., Project Implicit, Stimulus Materials (2006), http://www.projectimplicit.net/stimuli.php. They include only the center of the face, with ears, hair, and anything below the chin cropped out. None of the faces has facial hair, eyeglasses, or distinguishing features. Id. (providing faces that can be downloaded under the "race faces" stimulus set).
210. In this respect we varied from the procedures recommended by Greenwald and his colleagues, see Greenwald et al., supra note 29, at 198, by reducing the practice rounds from the twenty they suggested to sixteen. We did this in the interest of saving time. We did retain the forty trials in the critical rounds. We had more time available in the western jurisdiction, and increased the length of rounds three and six to twenty trials.
211. The exact instructions were as follows:
212. In the first round, the two CATEGORIES that you are to distinguish are:
213. BLACK vs. WHITE faces.
214. Press the "E" key if the TARGET is a WHITE face.
215. Press the "I" key if the TARGET is a BLACK face.
216. Remember that an "X" will appear when you make an error. Whenever the "X" appears, correct the mistake by pressing the other key.
217. Please respond AS RAPIDLY AS POSSIBLE, but don't respond so fast that you make many errors. (Occasional errors are okay.)
218. Press the space bar when you are ready to begin.
219. Greenwald et al., supra note 29, at 212-15.
220. Nosek et al., supra note 17, at 103-04.
221. Greenwald et al., supra note 29, at 212-15.
222. Id. at 201-02.
223. Id. at 203.
224. Id. at 214 tbl.4.
225. Nosek et al., supra note 17, at 103-04.
226. Project Implicit, Background Information (2002), https://implicit. harvard. edu/implicit/demo/background/index.jsp (last visited on Mar. 9, 2009).
227. See Nosek et al., supra note 17, at 104.
228. Id.
229. Id.
230. Id.
231. Id.
232. Id.
233. Id.
234. Id.
235. None of the judges provided latencies that were less than 300 ms in either of the two critical rounds measuring the race IAT; two of the judges provided responses that were faster than 300 ms in the gender IAT (one round each). Many more of the judges produced latencies that exceeded 3000 ms. On the race IAT, fifty-eight judges (or 50.4%) produced at least one latency greater than 3000 ms in the stereotypecongruent round (round four). Specifically, in the stereotype-congruent round: thirty-three judges produced one long latency; twenty produced two; three produced three; and two produced four. In the stereotype-incongruent round on the race IAT (round seven), sixty-eight judges (or 59.1%) produced at least one latency greater than 3000 ms. Specifically, in the stereotype-incongruent round: thirty-three judges produced one long latency; twelve produced two; ten produced three; four produced four; two produced five; four produced six; and three produced seven. On the gender IAT, fifty-seven judges (or 49.6%) produced at least one latency greater than 3000 ms in the stereotype-congruent round (round seven). Specifically, in the stereotypecongruent round: thirty-six judges produced one long latency; seven produced two; nine produced three; three produced four; one produced five; and one produced eight. In the stereotype-incongruent round on the gender IAT (round four), fifty-six judges (or 48.7%) produced at least one latency greater than 3000 ms. Specifically, in the stereotype-incongruent round: twenty-seven judges produced one long latency; fifteen produced two; six produced three; three produced four; two produced five; one produced six; and one produced seven. Note that because some of these long latencies fell into the first two rounds, they are not included in the analysis.
236. One of the judges violated both criteria. We calculated both means after excluding the first two rounds.
237. Four judges violated both criteria.
238. Nosek et al., supra note 17, at 104.
239. Id.
240. Id.
241. Greenwald et al., supra note 29, at 210 tbl.2, report the effect of order with a correlation coefficient, rather than a mean or percent difference. They report that the correlation varies with the IAT, noting that the gender IAT that we used here produces a higher correlation between order and IAT score than do other IATs. They report correlations as high as 0.29 (depending upon the scoring method), which would mean that order can account for up to ten percent of the IAT score. Id. By contrast, the race IAT that we used produces small correlations with order, ranging from 0.002 to 0.054; thus, order accounts for, at most, one-quarter of one percent of the IAT score. The order effects seem to vary with context, and hence we cannot be certain of the extent of the influence of order on our materials.
242. Had we randomized the order, each judge's IAT score would have varied with the order to some extent. This would have introduced some variation to the IAT score that would inherently reduce the correlation we observed across all judges. Our measure of the IAT score across all judges would have been more reliable had we randomized, but the IAT score for the individual judges would have been less consistent, thereby interfering with the correlation.
243. Greenwald et al., supra note 29, at 199-200.
244. Id.
245. In the eastern and western samples we reduced the number of trials in the practice rounds (rounds 1, 2, 3, 5, and 6) from twenty to sixteen, so as to save time.
246. Greenwald et al., supra note 29, at 213.
247. Id. at 214-15.
248. In the race IAT, twenty-nine out of the thirty-three instances in which judges produced latency scores of greater than 10, 000 ms on a trial (or 87.9%) occurred during the practice rounds. In the gender IAT, the two instances in which judges exhibited trials that exceeded 10, 000 ms occurred in the target round.
249. Note that these correlations used all judges, with no exclusions for speed, did not bound the data between 300 and 3000 ms, and did not exclude the first two rounds, as we did for calculating the mean differences.