Critical Reflections on Evaluation Practices in Coreference Resolution

Proceedings of the 2013 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2013 - Student Research Workshop

Volumn , Issue , 2013, Pages 1-7

  Holen, Gordana Ilić ^a  

^a UNIVERSITY OF OSLO   (Norway)

Author keywords

[No Author keywords available]

Indexed keywords

'CURRENT; COREFERENCE RESOLUTION; CRITICAL REFLECTIONS; HUMAN JUDGMENTS; HUMAN PERCEPTION; INFORMATION VALUE; ITS EVALUATION; QUANTITATIVE EVALUATION; RESOLUTION SYSTEMS;

EID: 84906924468     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563 – 566, Granada, Spain.
2. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classication. Procesamiento del Lenguaje Natural, 42:87–96.
3. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 Shared Task. In Proceedings of the 15th Conference on Computational Natural Language Learning: Shared Task, pages 28–34, Portland, Oregon.
4. Xiaoqiang Luo, Abe Ittycheriah Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 136–143, Barcelona, Spain.
5. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 25–32, Vancouver, Canada.
6. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. CoNLL-2011 Shared Task: Modeling unrestricted coreference in OntoNotes. In Proceedings of the 15th Conference on Computational Natural Language Learning: Shared Task, pages 1–27, Portland, Oregon.
7. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Joint Conference on EMNLP and CoNLL: Shared Task, pages 1–40, Jeju Island, Korea.
8. W. M. Rand. 1971. Objective criteria for evaluation of clustering methods. Journal of American Statistical Association, 66(336):846–850.
9. Marta Recasens and Eduard Hovy. 2011. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, 17(04):485–510.
10. Marta Recasens, Lluis Màrquez, Emili Sapena, M. Antònia Marti, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. Semeval-2010 task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 1–8, Uppsala, Sweden.
11. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the Sixth Message understanding Conference (MUC-6), pages 45–52, San Francisco, CA.
12. Xiaofeng Yang, Guodong Zhou, Jian Su, and Chew Lim Tan. 2003. Coreference resolution using competition learning approach. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 176–183, Sapporo, Japan.