The problem of infra-marginality in outcome tests for discrimination

Annals of Applied Statistics

Volumn 11, Issue 3, 2017, Pages 1193-1216

  Simoiu, Camelia ^a   Corbett Davies, Sam ^a   Goel, Sharad ^a  

^a Stanford University   (United States)

Author keywords

Benchmark test; Infra marginality; Outcome test; Policing; Tests for discrimination; Traffic stops

Indexed keywords

EID: 85029036158     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/17-AOAS1058     Document Type: Article

References (36)

1. ALPERT, G. P., SMITH, M. R. and DUNHAM, R. G. (2004). Toward a better benchmark: Assessing the utility of not-at-fault traffic crash data in racial profiling research. Justice Res. Policy 6 43–69.
2. ANTONOVICS, K. and KNIGHT, B. G. (2009). A new look at racial profiling: Evidence from the Boston police department. Rev. Econ. Stat. 91 163–177.
3. ANWAR, S. and FANG, H. (2006). An alternative test of racial prejudice in motor vehicle searches: Theory and evidence. Am. Econ. Rev. 96 127–151.
4. ARROW, K. (1973). The theory of discrimination. In Discrimination in Labor Markets Princeton Univ. Press, Princeton.
5. AYRES, I. (2002). Outcome tests of racial disparities in police practices. Justice Res. Policy 4 131– 142.
6. BECKER, G. S. (1957). The Economics of Discrimination. Univ. Chicago Press, Chicago, IL.
7. BECKER, G. S. (1993). Nobel lecture: The economic way of looking at behavior. J. Polit. Econ. 101 385–409.
8. CARPENTER, B., GELMAN, A., HOFFMAN, M., LEE, D., GOODRICH, B., BETANCOURT, M., BRUBAKER, M. A., GUO, J., LI, P. and STAN, A. R. (2016). A probabilistic programming language. J. Stat. Softw.
9. CARR, J. H. and MEGBOLUGBE, I. F. (1993). The Federal Reserve Bank of Boston study on mortgage lending revisited. J. Hous. Res. 4 277–313.
10. CORBETT-DAVIES, S., PIERSON, E., FELLER, A., GOEL, S. and HUQ, A. (2017). Algorithmic decision making and the cost of fairness. Preprint. Available at 1701.08230.
11. DUANE, S., KENNEDY, A. D., PENDLETON, B. J. and ROWETH, D. (1987). Hybrid Monte Carlo. Phys. Lett. B 195 216–222.
12. ENGEL, R. S. and CALNON, J. M. (2004). Comparing benchmark methodologies for police-citizen contacts: Traffic stop data collection for the Pennsylvania State Police. Police Q. 7 97–125.
13. ENGEL, R. S. and TILLYER, R. (2008). Searching for equilibrium: The tenuous nature of the outcome test. Justice Q. 25 54–71.
14. EPP, C. R., MAYNARD-MOODY, S. and HAIDER-MARKEL, D. P. (2014). Pulled over: How Police Stops Define Race and Citizenship. Univ. Chicago Press, Chicago, IL.
15. GALSTER, G. C. (1993). The facts of lending discrimination cannot be argued away by examining default rates. Hous. Policy Debate 4 141–146.
16. GELMAN, A., FAGAN, J. and KISS, A. (2007). An analysis of the New York City Police Depart-ment’s “stop-and-frisk” policy in the context of claims of racial bias. J. Amer. Statist. Assoc. 102 813–823. MR2411646
17. GELMAN, A., MENG, X.-L. and STERN, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sinica 6 733–807. MR1422404
18. GELMAN, A. and RUBIN, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 457–472.
19. GELMAN, A., CARLIN, J. B., STERN, H. S. and RUBIN, D. B. (2004). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC, Boca Raton, FL. MR2027492
20. GOEL, S., RAO, J. M. and SHROFF, R. (2016). Precinct or prejudice? Understanding racial disparities in New York City’s stop-and-frisk policy. Ann. Appl. Stat. 10 365–394. MR3480500
21. GOEL, S., PERELMAN, M., SHROFF, R. and SKLANSKY, D. (2017). Combatting police discrimination in the age of big data. New Crim. Law Rev. 20 181–232.
22. GROGGER, J. and RIDGEWAY, G. (2006). Testing for racial profiling in traffic stops from behind a veil of darkness. J. Amer. Statist. Assoc. 101 878–887. MR2324089
23. HETEY, R., MONIN, B., MAITREYI, A. and EBERHARDT, J. (2016). Data for change: A statistical analysis of police stops, searches, handcuffings, and arrests in oakland, Calif., 2013-2014. Technical report, Stanford University, SPARQ: Social Psychological Answers to Real-World Questions.
24. HOFFMAN, M. D. and GELMAN, A. (2014). The no-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res. 15 1593–1623. MR3214779
25. JORDAN, M. I. (2004). Graphical models. Statist. Sci. 19 140–155. MR2082153
26. KNOWLES, J., PERSICO, N. and TODD, P. (2001). Racial bias in motor vehicle searches: Theory and evidence. J. Polit. Econ. 109 203–229.
27. LANGE, J. E., BLACKMAN, K. O. and JOHNSON, M. B. (2001). Speed violation survey of the New Jersey turnpike: Final report, Public Services Research Institute.
28. MACLIN, T. (2008). Good and bad news about consent searches in the Supreme Court. McGeorge Law Rev. 39 27.
29. MCCONNELL, E. H. and SCHEIDEGGER, A. R. (2001). Race and speeding citations: Comparing speeding citations issued by air traffic officers with those issued by ground traffic officers. In Annual Meeting of the Academy of Criminal Justice Sciences, Washington, DC.
30. METROPOLIS, N., ROSENBLUTH, A. W., ROSENBLUTH, M. N., TELLER, A. H. and TELLER, E. (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21 1087–1092.
31. NEAL, R. M. (1994). An improved acceptance procedure for the hybrid Monte Carlo algorithm. J. Comput. Phys. 111 194–203. MR1271540
32. PHELPS, E. S. (1972). The statistical theory of racism and sexism. Am. Econ. Rev. 62 659–661.
33. PIERSON, E., CORBETT-DAVIES, S. and GOEL, S. (2017). Fast threshold tests for detecting discrimination. Preprint. Available at 1702.08536.
34. RIDGEWAY, G. (2006). Assessing the effect of race bias in post-traffic stop outcomes using propensity scores. J. Quant. Criminol. 22 1–29.
35. RIDGEWAY, G. and MACDONALD, J. M. (2009). Doubly robust internal benchmarking and false discovery rates for detecting racial bias in police stops. J. Amer. Statist. Assoc. 104 661–668. MR2751446
36. WALKER, S. (2003). Internal benchmarking for traffic stop data: An early intervention system approach. Technical report, Police Executive Research Forum.