Combining evaluation metrics via the unanimous improvement ratio and its application to clustering tasks

Journal of Artificial Intelligence Research

Volumn 42, Issue , 2011, Pages 689-718

  Amigó, Enrique ^a   Gonzalo, Julio ^a   Artiles, Javier ^b   Verdejo, Felisa ^a  

^a UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA   (Spain)

^b CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EMPIRICAL RESULTS; EVALUATION METRICS; F-MEASURE; IMPROVEMENT RATIO; PRECISION AND RECALL; QUALITY CRITERIA; RELATIVE WEIGHTS; SYSTEM RANKINGS; TEXT CLUSTERING; THEORETICAL FOUNDATIONS; WEIGHTING SCHEME;

ARTIFICIAL INTELLIGENCE; EQUIPMENT TESTING;

QUALITY CONTROL;

EID: 84856439880     PISSN: None     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.3401     Document Type: Article

References (19)

1. Amiǵo, E., Gonzalo, J., Artiles, J., & Verdejo, F. (2008). A comparison ofextrinsic clustering evaluation metrics based on formal constraints. Information Retrieval, 12(4), 461-486.
2. Artiles, J., Gonzalo, J., & Sekine, S. (2009). We PS-2 Evaluation Campaign: Overview of the Web People Search Clustering Task. In Proceedings Of The 2nd Web People Search Evaluation Workshop (We PS 2009).
3. Artiles, J., Gonzalo, J., & Sekine, S. (2007). The Sem Eval-2007 We PS evaluation: Es-tablishing a Benchmark for the Web People Search Task. In Proceedings of the 4th International Workshop on Semantic Evaluations, Sem Eval '07, pp. 64-69 Strouds-burg, PA, USA. Association for Computational Linguistics.
4. Bagga, A., & Baldwin, B. (1998). Entity-Based Cross-Document Coreferencing Using the Vector Space Model. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (COLING-ACL'98), pp. 79-85.
5. Carreras, X., & M̀arquez, L. (2004). Introduction to the Co NLL-2004 Shared Task: Semantic Role Labeling. In Ng, H.T.,& Riloff, E. (Eds.), HLT-NAACL 2004 Workshop: Eighth Conference on Computational Natural Language Learning (Co NLL-2004), pp. 89-97 Boston, Massachusetts, USA. Association for Computational Linguistics.
6. Cormack, G.V.,& Lynam, T. R. (2005). TREC 2005 Spam Track Overview. In Proceedings of the fourteenth Text RE trieval Conference (TREC-2005).
7. Debreu, G. (1959). Topological methods in cardinal utility theory. Mathematical Methods in the Social Sciences, Stanford University Press, 1(76), 16-26.
8. Ghosh, J. (2003). Scalable clustering methods for data mining. In Ye, N. (Ed.), Handbook of Data Mining. Lawrence Erlbaum.
9. Halkidi, M., Batistakis, Y., & Vazirgiannis, M. (2001). On Clustering Validation Techniques. Journal of Intelligent Information Systems, 17(2-3), 107-145. (Pubitemid 34050790)
10. Luce, R., & Tukey, J. (1964). Simultaneous conjoint measurement: a new scale type of fundamental measurement. Journal of Mathematical Psychology, 1(1).
11. Mann, G. S. (2006). Multi-Document Statistical Fact Extraction and Fusion. Ph.D. thesis, Johns Hopkins University.
12. Meila, M. (2003). Comparing clusterings. In Proceedings of COLT 03.
13. Narens, L., & Luce, R. D. (1986). Measurement: The theory of numerical assignments. Psy-chological Bulletin, 99.
14. Steinbach, M., Karypis, G., & Kumar, V. (2000). A comparison of document clustering techniques. In KDD Workshop on Text Mining,2000.
15. Su, K.-Y., Su, J., Wiebe, J., & Li, H. (Eds.). (2009). Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Association for Computational Linguis-tics, Suntec, Singapore.
16. Tao Li, C. Z., & Zhu, S. (2006). Empirical Studies on Multilabel Classification. In Proceed-ings of the 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2006).
17. van Rijsbergen, C. J. (1974). Foundation ofevaluation. Journal of Documentation, 30(4), 365-373.
18. Weng, C. G., & Poon, J. (2008). A New Evaluation Measure for Imbalanced Datasets. In Roddick, J. F., Li, J., Christen, P., & Kennedy, P. J. (Eds.), Seventh Australasian Data Mining Conference (Aus DM 2008), Vol. 87 of CRPIT, pp. 27-32 Glenelg, South Australia. ACS.
19. Zhao, Y., & Karypis, G. (2001). Criterion functions for document clustering: Experiments and analysis. Technical Report TR 01-40, Department of Computer Science, Univer-sity of Minnesota, Minneapolis, MN.