Performance measures for multilabel evaluation: A case study in the area of image classification

MIR 2010 - Proceedings of the 2010 ACM SIGMM International Conference on Multimedia Information Retrieval

Volumn , Issue , 2010, Pages 35-44

  Nowak, Stefanie ^a   Lukashevich, Hanna ^a   Dunker, Peter ^b   Rüger, Stefan ^c  

^a FRAUNHOFER IDMT   (Germany)

^b Gracenote Inc   (United States)

^c OPEN UNIVERSITY   (United Kingdom)

Author keywords

Benchmark; Evaluation; Image annotation

Indexed keywords

BENCHMARK EVALUATION; CONCEPT-BASED; CONFIDENCE VALUES; EVALUATION MEASURES; F-MEASURE; IMAGE ANNOTATION; IMAGECLEF; MULTI-LABEL; MULTIMEDIA DOCUMENTS; ONTOLOGY-BASED; PERFORMANCE MEASURE; RANDOM NUMBERS; RESEARCH GROUPS; SEMANTIC INDEXING; STANDARD EVALUATIONS; STRUCTURE INFORMATION;

IMAGE ANALYSIS; INFORMATION RETRIEVAL; RESEARCH; SEMANTIC WEB;

ONTOLOGY;

EID: 77952369139     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1743384.1743398     Document Type: Conference Paper

References (31)

1. A. Bernstein, E. Kaufmann, C. Bürki, and M. Klein. How similar is it? Towards personalized similarity measures in ontologies. In 7th Intern. Conference Wirtschaftsinformatik, Germany. Springer, 2005.
2. A. Binder and M. Kawanabe. Fraunhofer FIRST's Submission to ImageCLEF2009 Photo Annotation Task: Non-sparse Multiple Kernel Learning. CLEF working notes, 2009.
3. H. Blockeel, M. Bruynooghe, S. Dzeroski, J. Ramon, and J. Struyf. Hierarchical multi-classi cation. In SIGKDD Workshop on Multi-Relational Data Mining, pages 21-35, 2002.
4. N. Cesa-Bianchi, C. Gentile, and L. Zaniboni. Incremental algorithms for hierarchical classi cation. Journal of Machine Learning Research, 7:31-54, 2006.
5. B. Daroczy, I. Petras, A. Benczur, Z. Fekete, D. Nemeskey, D. Siklosi, and Z. Weiner. SZTAKI ImageCLEF 2009. CLEF working notes, 2009.
6. M. Douze, M. Guillaumin, T. Mensink, C. Schmid, and J. Verbeek. INRIA-LEARs participation to ImageCLEF 2009. CLEF working notes, 2009.
7. H. Escalante, J. Gonzalez, C. Hernandez, A. Lopez, M. Montex, E. Morales, E. Ruiz, L. Sucar, and L. Villasenor. TIA-INAOE's Participation at ImageCLEF 2009. CLEF working notes, 2009.
8. A. Fakeri-Tabrizi, S. Tollari, L. Denoyer, and P. Gallinari. UPMC/LIP6 at ImageCLEF annotation 2009: Large Scale Visual Concept Detection and Annotation. CLEF working notes, 2009.
9. J. Fan, Y. Gao, H. Luo, and R. Jain. Mining multilevel image semantics via hierarchical classi cation. IEEE Trans. on Multimedia, 10(2):167, 2008.
10. M. Ferecatu and H. Sahbi. TELECOM ParisTech at ImageClef 2009: Large Scale Visual Concept Detection and Annotation Task. CLEF working notes, 2009.
11. A. Freitas and A. de Carvalho. A tutorial on hierarchical classi cation with applications in bioinformatics. Intelligent Information Technologies: Concepts, Methodologies, Tools and Applications, 2007.
12. H. Glotin, A. Fakeri-Tabrizi, P. Mulhem, M. Ferecatu, Z. Zhao, S. Tollari, G. Quenot, H. Sahbi, E. Dumont, and P. Gallinari. Comparison of Various AVEIR Visual Concept Detectors with an Index of Carefulness. CLEF working notes, 2009.
13. J. Hare and P. Lewis. IAM ImageCLEF Photo Annotation 2009: Naive application of a linearalgebraic semantic space. CLEF working notes, 2009.
14. M. J. Huiskes and M. S. Lew. The MIR Flickr Retrieval Evaluation. In MIR '08: Proceedings of the 2008 ACM Intern. Conf. on Multimedia Information Retrieval, New York, NY, USA, 2008. ACM.
15. A. Iftene, L. Vamanu, and C. Croitoru. UAIC at ImageCLEF 2009 Photo Annotation Task. CLEF working notes, 2009.
16. Y. Liu and E. Shriberg. Comparing evaluation metrics for sentence boundary detection. In Intern. Conf. on Acoustics, Speech and Signal Processing, 2007.
17. A. Llorente, S. Little, and S. Rüger. MMIS at ImageCLEF 2009: Non-parametric Density Estimation Algorithms. CLEF working notes, 2009.
18. P. Lord, R. Stevens, A. Brass, and C. Goble. Investigating semantic similarity measures across the Gene Ontology: The relationship between sequence and annotation. volume 19. Oxford Univ Press, 2003.
19. D. Lowe. Object recognition from local scale-invariant features. In Intern. Conf. on Computer Vision, volume 2, pages 1150-1157. Corfu, Greece, 1999.
20. C. Manning, P. Raghavan, and H. Schutze. An Introduction to Information Retrieval [Draft]. Cambridge, UK: Cambridge University Press, April 2009. http://www.informationretrieval.org/.
21. P. Mulhem, J.-P. Chevallet, G. Quenon, and R. Al Batal. MRIM-LIG at ImageCLEF 2009: Photo Retrieval and Photo Annotation tasks. CLEF working notes, 2009.
22. J. Ngiam and H. Goh. I2R ImageCLEF Photo Annotation 2009 Working Notes. CLEF working notes, 2009.
23. S. Nowak and P. Dunker. A Consumer Photo Tagging Ontology: Concepts and Annotations. In THESEUS/ImageCLEF Pre-Workshop, 2009.
24. S. Nowak and P. Dunker. Overview of the CLEF 2009 Large-Scale Visual Concept Detection and Annotation Task. CLEF working notes, 2009.
25. S. Nowak and H. Lukashevich. Multilabel Classi cation Evaluation using Ontology Information. In Proc. of IRMLeS Workshop, ESWC, 2009.
26. P. Resnik. Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of arti cial intelligence research, 1999.
27. S. Sarin and W. Kameyama. Joint Contribution of Global and Local Features for Image Annotation. CLEF working notes, 2009.
28. X. Shen, M. Boutell, J. Luo, and C. Brown. Multi-label machine learning and its application to semantic scene classi cation. In Intern. Symp. on Electronic Imaging, San Jose, CA, 2004.
29. G. Tsoumakas and I. Vlahavas. Random k-labelsets: An ensemble method for multilabel classi cation. Lecture Notes in Computer Science, 4701:406, 2007.
30. K. van de Sande, T. Gevers, and A. Smeulders. The University of Amsterdam's Concept Detection System at ImageCLEF 2009. CLEF working notes, 2009.
31. Z.-Q. Zhao, H. Glotin, and E. Dumont. LSIS Scale Photo Annotations: Discriminant Features SVM versus Visual Dictionary based on Image Frequency. CLEF working notes, 2009.