BLANC: Implementing the Rand index for coreference evaluation

Natural Language Engineering

Volumn 17, Issue 4, 2011, Pages 485-510

  Recasens, M ^a   Hovy, E ^b  

^a UNIVERSITY OF BARCELONA   (Spain)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COREFERENCE; COREFERENCE RESOLUTION; FINE GRANULARITY; RAND INDEX; SCORING MEASURES; WEAK POINTS;

ARTIFICIAL INTELLIGENCE;

SOFTWARE ENGINEERING;

EID: 79956355712     PISSN: 13513249     EISSN: 14698110     Source Type: Journal    
DOI: 10.1017/S135132491000029X     Document Type: Article

References (32)

1. Amigó, E., Gonzalo, J., Artiles, J., and Verdejo, F. 2009. A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information Retrieval 12(4): 461-486.
2. Bagga, A., and Baldwin, B. 1998. Algorithms for scoring coreference chains. In Proceedings of the LREC 1998 Workshop on Linguistic Coreference, Granada, Spain, pp. 563-566.
3. Bengtson, E., and Roth, D. 2008. Understanding the value of features for coreference resolution. In Proceedings of EMNLP, Honolulu, HI, pp. 294-303.
4. Cai, J., and Strube, M. 2010. Evaluation metrics for end-to-end coreference resolution systems. In Proceedings of SIGDIAL, University of Tokyo, Japan, pp. 28-36.
5. Choi, Y., and Cardie, C. 2007. Structured local training and biased potential functions for conditional random fields with application to coreference resolution. In Proceedings of HLT-NAACL, Rochester, NY, pp. 65-72.
6. Culotta, A., Wick, M., Hall, R., and McCallum, A. 2007. First-order probabilistic models for coreference resolution. In Proceedings of HLT-NAACL, Rochester, NY, pp. 81-88.
7. Daumé III, H., and Marcu, D. 2005. A large-scale exploration of effective global features for a joint entity detection and tracking model. In Proceedings of HLT-EMNLP, Vancouver, Canada, pp. 97-104.
8. Denis, P., and Baldridge, J. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del Lenguaje Natural 42: 87-96.
9. Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., and Weischedel, R. 2004. The automatic content extraction (ACE) program -tasks, data and evaluation. In Proceedings of LREC, Lisbon, Portugal, pp. 837-840.
10. Finkel, J. R., and Manning, C. D. 2008. Enforcing transitivity in coreference resolution. In Proceedings of ACL-HLT, Columbus, OH, pp. 45-48.
11. Haghighi, A., and Klein, D. 2007. Unsupervised coreference resolution in a nonparametric Bayesian model. In Proceedings of ACL, Prague, Czech Republic, pp. 848-855.
12. Haghighi, A., and Klein, D. 2009. Simple coreference resolution with rich syntactic and semantic features. In Proceedings of EMNLP, Suntec, Singapore, pp. 1152-1161.
13. Hirschman, L., and Chinchor, N. 1997. MUC-7 coreference task definition -version 3.0. In Proceedings of MUC-7. Washington, DC.
14. Hubert, L., and Arabie, P. 1985. Comparing partitions. Journal of Classification 2(1): 193-218.
15. Luo, X. 2005. On coreference resolution performance metrics. In Proceedings of HLT-EMNLP, Vancouver, Canada, pp. 25-32.
16. Luo, X., Ittycheriah, A., Jing, H., Kambhatla, N., and Roukos, S. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In Proceedings of ACL, Barcelona, Spain, pp. 21-26.
17. Luo, X., and Zitouni, I. 2005. Multi-lingual coreference resolution with syntactic features. In Proceedings of HLT-EMNLP, Vancouver, Canada, pp. 660-667.
18. Ng, V. 2009. Graph-cut-based anaphoricity determination for coreference resolution. In Proceedings of NAACL-HLT, Boulder, CO, pp. 575-583.
19. Ng, V., and Cardie, C. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of ACL, Philadelphia, PA, pp. 104-111.
20. Poon, H., and Domingos, P. 2008. Joint unsupervised coreference resolution with Markov logic. In Proceedings of EMNLP, Honolulu, HI, pp. 650-659.
21. Popescu-Belis, A. 2000. Évaluation numérique de la résolution de la référence: Critiques et propositions. T.A.L.: Traitement automatique de la langue 40(2): 117-146.
22. Popescu-Belis, A., Rigouste, L., Salmon-Alt, S., and Romary, L. 2004. Online evaluation of coreference resolution. In Proceedings of LREC, Lisbon, Portugal, pp. 1507-1510.
23. Rahman, A., and Ng, V. 2009. Supervised models for coreference resolution. In Proceedings of EMNLP, Suntec, Singapore, pp. 968-977.
24. Rand, W. M. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66(336): 846-850.
25. Recasens, M., and Hovy, E. 2010. Coreference resolution across corpora: Languages, coding schemes, and preprocessing information. In Proceedings of ACL, Uppsala, Sweden, pp. 1423-1432.
26. Recasens, M., Màrquez, L., Sapena, E., Martí, M. A., Taulé, M., Hoste, V., Poesio, M., and Versley, Y. 2010. SemEval-2010 task 1: Coreference resolution in multiple languages. In Proceedings of the Fifth International Workshop on Semantic Evaluation (SemEval 2010), Uppsala, Sweden, pp. 1-8.
27. Recasens, M., and Martí, M. A. 2010. AnCora-CO: Coreferentially annotated corpora for Spanish and Catalan. Language Resources and Evaluation 44(4): 315-345.
28. Soon, W. M., Ng, H. T., and Lim, D. C. Y. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics 27(4): 521-544. (Pubitemid 33597451)
29. Stoyanov, V., Gilbert, N., Cardie, C., and Riloff, E. 2009. Conundrums in noun phrase coreference resolution: Making sense of the state-of-the-art. In Proceedings of ACLIJCNLP, Suntec, Singapore, pp. 656-664.
30. Vilain, M., Burger, J., Aberdeen, J., Connolly, D., and Hirschman, L. 1995. A model-theoretic coreference scoring scheme. In Proceedings of MUC-6, San Francisco, CA, pp. 45-52.
31. Wick, M., and McCallum, A. 2009. Advances in learning and inference for partition-wise models of coreference resolution. Technical Report UM-CS-2009-028, Department of Computer Science, University of Massachusetts.
32. Yang, X., Su, J., Lang, J., Tan, C. L., Liu, T., and Li, S. 2008. An entity-mention model for coreference resolution with inductive logic programming. In Proceedings of ACL-HLT, Columbus, OH, pp. 843-851.