The limits of abstract evaluation metrics: The case of hate speech detection

WebSci 2017 - Proceedings of the 2017 ACM Web Science Conference

Volumn , Issue , 2017, Pages 405-406

  Olteanu, Alexandra ^a   Talamadupula, Kartik ^a   Varshney, Kush R ^a  

^a IBM RESEARCH   (United States)

Author keywords

Evaluation metrics; Hate speech; Human centered metrics

Indexed keywords

COSTS; ERRORS;

EVALUATION METRICS; HUMAN-CENTERED METRICS; PERCEIVED COSTS; PROBLEM DOMAIN; SPEECH DETECTION; USER CHARACTERISTICS; USER CONTEXT; USER PERCEPTIONS;

SPEECH RECOGNITION;

EID: 85026782700     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/3091478.3098871     Document Type: Conference Paper

References (7)

1. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proc. of ICWSM.
2. Moritz Hardt. 2014. How big data is unfair: Understanding sources of unfairness in data driven decision making. Medium. Available at: https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de (2014).
3. Pat Langley. 2011. The changing science of machine learning. Machine Learning 82, 3 (2011).
4. Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kiciman. 2016. Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries. SSRN Preprint (2016).
5. Kush R Varshney. 2015. Data science of the people, for the people, by the people: A viewpoint on an emerging dichotomy. In Bloomberg Data for Good Exch. Conf.
6. Kiri Wagstaff. 2012. Machine learning that matters. In Proc. of ICML.
7. Ryen White. 2013. Beliefs and biases in web search. In Proc. of SIGIR.