Semantic hierarchies for image annotation: A survey

Pattern Recognition

Volumn 45, Issue 1, 2012, Pages 333-345

  Tousch, Anne Marie ^a,b   Herbin, Stéphane ^a   Audibert, Jean Yves ^b,c  

^a ONERA   (France)

^b UNIVERSITÉ PARIS EST   (France)

^c ECOLE NORMALE SUPÉRIEURE   (France)

Author keywords

Image annotation; Image retrieval; Semantic description; Structured vocabulary

Indexed keywords

IMAGE ANALYSIS; IMAGE RETRIEVAL; SEMANTICS; SURVEYS;

AUTOMATIC ANNOTATION; SEMANTIC DESCRIPTIONS; SEMANTIC HIERARCHIES; STRUCTURED VOCABULARY; USER NEED;

IMAGE ANNOTATION;

EID: 80052769306     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.05.017     Document Type: Article

References (104)

1. N. Ahuja, and S. Todorovic Learning the taxonomy and models of categories present in arbitrary images ICCV07 2007 1 8
2. S. Aksoy, K. Koperski, C. Tusk, G. Marchisio, J. Tilton, Learning Bayesian classifiers for scene classification with a visual grammar, in: GeoRS, vol. 43, no. 3, 2005, pp. 581589. (Pubitemid 40320279)
3. Y. Amit, M. Fink, N. Srebro, and S. Ullman Uncovering shared structures in multiclass classification ICML07 2007 17 24
4. O. Arandjelovic, and A. Zisserman Automatic face recognition for film character retrieval in feature-length films CVPR05 2005 860 867 (Pubitemid 43897283)
5. L.H. Armitage, and P.G. Enser Analysis of user need in image archives Journal of Information Science 24 4 1997 287 299 (Pubitemid 127600717)
6. Y.A. Aslandogan, C. Thier, C.T. Yu, J. Zou, N. Rishe, Using semantic contents and wordnet in image retrieval, in: SIGIR Forum, vol. 31, no. SI, 1997, pp. 286295. (Pubitemid 127720329)
7. A. Bar-Hillel, and D. Weinshall Subordinate class recognition using relational object models NIPS06 2006 73 80
8. K. Barnard, P. Duygulu, and D. Forsyth Clustering art CVPR01 2001 pp. II 434II 439
9. K. Barnard, Q. Fan, R. Swaminathan, A. Hoogs, R. Collins, P. Rondot, and J. Kaufhold Evaluation of localized semantics: data methodology, and experiments International Journal of Computer Vision 77 13 2008 199 217 (Pubitemid 351294742)
10. K. Barnard, and D. Forsyth Learning the semantics of words and pictures ICCV01 vol. 2 2001 IEEE Vancouver 408 415 (Pubitemid 32795085)
11. E. Bart, I. Porteous, P. Perona, and M. Welling Unsupervised learning of visual taxonomies CVPR08 2008 1 8
12. I. Biederman Recognition-by-components: a theory of human image understanding Psychological Review 94 2 1987 115 147
13. A. Binder, M. Kawanabe, and U. Brefeld Efficient classification of images with taxonomies ACCV09 2009
14. A. Bosch, X. Muñoz, and R. Martí Which is the best way to organize/classify images by content? IVC 25 6 2007 778 791 (Pubitemid 46541657)
15. A. Bosch, A. Zisserman, and X. Munoz Image classification using random forests and ferns ICCV07 2007 1 8
16. M.R. Boutell, J. Luo, X. Shen, and C.M. Brown Learning multi-label scene classification Pattern Recognition 37 9 2004 1757 1771 (Pubitemid 38804465)
17. G. Carneiro, A.B. Chan, P.J. Moreno, and N. Vasconcelos Supervised learning of semantic classes for image annotation and retrieval IEEE Transactions on Pattern Analysis and Machine Intelligence 29 3 2007 394 410 (Pubitemid 46336416)
18. K. Crammer, and Y. Singer On the algorithmic implementation of multiclass kernel-based vector machines Journal of Machine Learning Research 2 2002 265 292
19. R. Datta, W. Ge, J. Li, and J.Z. Wang Toward bridging the annotation-retrieval gap in image search by a generative modeling approach MULTIMEDIA06 2006 977 986 (Pubitemid 47113664)
20. R. Datta, W. Ge, J. Li, and J.Z. Wang Toward bridging the annotation-retrieval gap in image search International Transactions on Multimedia 14 3 2007 24 35 (Pubitemid 47322512)
21. R. Datta, D. Joshi, J. Li, and J.Z. Wang Image retrieval: ideas, influences, and trends of the new age ACM Computing Surveys 40 2 2008 1 60
22. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei Imagenet: a large-scale hierarchical image database CVPR09 2009 248 255
23. P. Duygulu, K. Barnard, J.F.G.D. Freitas, and D.A. Forsyth Object recognition as machine translation: learning a lexicon for a fixed image vocabulary ECCV02 2002 97 112
24. J.P. Eakins Towards intelligent image retrieval Pattern Recognition 35 1 2002 3 14 (Pubitemid 33098713)
25. J.P. Eakins, P. Briggs, and B. Burford Image retrieval interfaces: a user perspective CIVR04 2004 628 637
26. P. Enser Visual image retrieval: seeking the alliance of concept-based and content-based paradigms Journal of Information Science 26 4 2000 199 210
27. P.G.B. Enser, and C.J. Sandom Towards a comprehensive survey of the semantic gap in visual image retrieval CIVR03 2003 291 299
28. B. Epshtein, and S. Ullman Feature hierarchies for object classification ICCV05 2005 220 227 (Pubitemid 44054962)
29. B. Epshtein, and S. Ullman Semantic hierarchies for recognizing objects and parts CVPR07 2007 1 8
30. J. Fan, Y. Gao, and H. Luo Hierarchical classification for automatic image annotation SIGIR07 2007 111 118 (Pubitemid 350164951)
31. J. Fan, Y. Gao, and H. Luo Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation IEEE Transactions on Image Processing 17 3 2008 407 426 (Pubitemid 351395085)
32. J. Fan, Y. Gao, H. Luo, and R. Jain Mining multilevel image semantics via hierarchical classification IEEE Transactions on Multimedia 10 2 2008 167 187
33. X. Fan, and D. Geman Hierarchical object indexing and sequential learning ICPR04 2004 65 68
34. L. Fei-Fei, R. Fergus, and P. Perona A Bayesian approach to unsupervised one-shot learning of object categories ICCV03 2003 1134
35. R. Fergus, P. Perona, A. Zisserman, Object class recognition by unsupervised scale-invariant learning, in: CVPR02, vol. 264, 2003.
36. S. Fidler, and A. Leonardis Towards scalable representations of object categories: learning a hierarchy of parts CVPR07 2007 1 8
37. C.O. Frost, B. Taylor, A. Noakes, S. Markel, D. Torres, and K.M. Drabenstott Browse and search patterns in a digital image database Information Retrieval 1 4 2000 287 313
38. Y. Gao, and J. Fan Incorporating concept ontology to enable probabilistic concept reasoning for multi-level image annotation MIR06 2006 79 88 (Pubitemid 47168204)
39. L.M. Garshol Metadata? Thesauri? Taxonomies? Topic Maps! Making Sense of it all Journal of Information Science 30 4 2004 378 391
40. C. Gerlach Category-specificity in visual object recognition Cognition 111 3 2009 281 301
41. A. Gilchrist Thesauri, taxonomies and ontologies - an etymological note Journal of Documentation 59 1 2003 7 18 (Pubitemid 36224429)
42. G. Griffin, A. Holub, P. Perona, Caltech-256 Object Category Dataset, Technical Report no. 7694, California Institute of Technology, 2007.
43. G. Griffin, and P. Perona Learning and using taxonomies for fast visual categorization CVPR08 2008 1 8
44. A. Gupta, and L.S. Davis Beyond nouns: exploiting prepositions and comparative adjectives for learning visual classifiers ECCV08 2008 16 29
45. J.S. Hare, P.H. Lewis, P.G.B. Enser, C.J. Sandom, Mind the gap: another look at the problem of the semantic gap in image retrieval, in: Multimedia Content Analysis, Management and Retrieval, SPIE, vol. 6073, 2006, pp. 607309.1607309.12.
46. L. Hollink, G. Schreiber, B. Wielemaker, and B. Wielinga Semantic annotation of image collections KCAP03 2003
47. L. Hollink, G. Schreiber, B. Wielinga, and M. Worring Classification of user image descriptions International Journal of Human Computer Studies 61 5 2004 601 626
48. ImageNet Large Scale Visual Recognition Challenge 2010, url 〈 http://www.image-net.org/challenges/LSVRC/2010/index 〉.
49. A. Jaimes Human factors in automatic image retrieval system design and evaluation SPIE vol. 6061 2006 pp. 606103.1606103.9
50. A. Jaimes, and S.-F. Chang Conceptual framework for indexing visual information at multiple levels SPIE vol. 3964 2000 2 15 (Pubitemid 30535439)
51. J. Jeon, V. Lavrenko, and R. Manmatha Automatic image annotation and retrieval using cross-media relevance models SIGIR03 2003 119 126
52. Y. Jin, L. Khan, L. Wang, and M. Awad Image annotations by combining multiple evidence wordnet MULTIMEDIA05 2005 706 715
53. P. Jolicoeur, M.A. Gluck, and S.M. Kosslyn Pictures and names: making the connection Cognitive Psychology 16 2 1984 243 275
54. C. Jörgensen Attributes of images in describing tasks Information Processing & Management 34 23 1998 161 174 (Pubitemid 128392788)
55. A. Kushal, C. Schmid, and J. Ponce Flexible object models for category-level 3D object recognition CVPR07 2007 1 8
56. T. Lam, and R. Singh Semantically relevant image retrieval by combining image and linguistic analysis ISVC06 2006 770 779 (Pubitemid 44891517)
57. S. Lazebnik, C. Schmid, and J. Ponce Beyond bags of features: spatial pyramid matching for recognizing natural scene categories CVPR06 2006 2169 2178 (Pubitemid 44931582)
58. B. Leibe, A. Ettlin, and B. Schiele Learning semantic object parts for object categorization Image and Vision Computing 26 1 2008 15 26 (Pubitemid 350051536)
59. K. Levi, M. Fink, and Y. Weiss Learning from a small number of training examples by exploiting object categories LCV04 2004 96
60. J. Li, and J. Wang Real-time computerized annotation of pictures IEEE Transactions on Pattern Analysis and Machine Intelligence 30 6 2008 985 1002 (Pubitemid 351650103)
61. L.-J. Li, R. Socher, and L. Fei-Fei Towards total scene understanding: classification annotation and segmentation in an automatic framework CVPR09 2009 2036 2043
62. S. Liu, L.-T. Chia, S. Chan, On the move to meaningful Internet systems 2004: CoopIS, DOA, and ODBASE, in: Lecture Notes in Computer Sciences, vol. 3291, 2004, pp. 10501061.
63. Y. Liu, D. Zhang, G. Lu, and W.-Y. Ma A survey of content-based image retrieval with high-level semantics Pattern Recognition 40 1 2007 262 282 (Pubitemid 44486285)
64. N.E. Maillot, and M. Thonnat Ontology based complex object recognition Image and Vision Computing 26 1 2008 102 113 (Pubitemid 350051532)
65. A. Makadia, V. Pavlovic, and S. Kumar A new baseline for image annotation ECCV 08 2008 316 329
66. M. Markkula, and E. Sormunen End-user searching challenges indexing practices in the digital newspaper photo archive Information Retrieval 1 4 2000 259 285
67. M. MarszaŁek, and C. Schmid Semantic hierarchies for visual object recognition CVPR07 October 2007 1 7
68. M. MarszaŁek, and C. Schmid Constructing category hierarchies for visual recognition ECCV08 2008 479 491
69. A. Mohan, C. Papageorgiou, and T. Poggio Example-based object detection in images by components IEEE Transactions on Pattern Analysis and Machine Intelligence 23 4 2001 349 361 (Pubitemid 32416300)
70. A. Mojsilovic, J. Gomes, and B. Rogowitz Semantic-friendly indexing and querying of images based on the extraction of the objective semantic cues International Journal of Computer Vision 56 12 2004 79 107
71. J. Mundy Object recognition in the geometric era: a retrospective CLOR06 2006 3 28
72. G.L. Murphy, and E.E. Smith Basic-level superiority in picture categorization Journal of Verbal Learning and Verbal Behavior 21 1 1982 1 20
73. G.T. Papadopoulos, V. Mezaris, S. Dasiopoulou, and I. Kompatsiaris Semantic image analysis using a learning approach and spatial context SAMT06 2006
74. D. Parikh, and T. Chen Hierarchical semantics of objects (hsos) ICCV07 2007 1 8
75. F. Perronnin Universal and adapted vocabularies for generic visual categorization IEEE Transactions on Pattern Analysis and Machine Intelligence 30 7 2008 1243 1256
76. A. Pinz Object categorization Foundations and Trends in Computer Graphics and Vision 1 4 2005 255 353 (Pubitemid 46100518)
77. J. Ponce, M. Hebert, C. Schmid, A. Zisserman, Toward category-level object recognition, in: Lecture Notes in Computer Sciences, vol. 4170, 2006.
78. A. Popescu, C. Millet, and P.-A. Moëllic Ontology driven content based image retrieval CIVR07 2007 387 394 (Pubitemid 350229650)
79. M. Rege, M. Dong, and F. Fotouhi Building a user-centered semantic hierarchy in image databases Multimedia Systems 12 45 2007 325 338
80. M. Richardson, and P. Domingos Markov logic networks Machine Learning 62 12 2006 107 136 (Pubitemid 43202307)
81. E. Rosch, C.B. Mervis, W.D. Gray, D.M. Johnson, and P. Boyes-Braem Basic objects in natural categories Cognitive Psychology 8 3 1976 382 439
82. D.A. Rosenthal, and R. Bajcsy Visual and conceptual hierarchy: a paradigm for studies of automated generation of recognition strategies IEEE Transactions on Pattern Analysis and Machine Intelligence 6 3 1984 319 325 (Pubitemid 14563807)
83. Y. Rui, T. Huang, and S. Chang Image retrieval: current, techniques promising directions, and open issues Journal of Visual Communication and Image Representation 10 1 1999 39 62
84. S. Santini, and R. Jain Beyond query by example WMSP98 1998 3 8
85. S. Shatford Layne Some issues in the indexing of images Journal of the American Society for Information Science 45 8 1994 583 588
86. J. Sivic, B. Russell, A. Zisserman, W. Freeman, and A. Efros Unsupervised discovery of visual object class hierarchies CVPR08 2008 1 8
87. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain Content-based image retrieval at the end of the early years IEEE Transactions on Pattern Analysis and Machine Intelligence 22 12 2000 1349 1380 (Pubitemid 32088465)
88. V.-W. Soo, C.-Y. Lee, C.-C. Li, S.L. Chen, and C.-C. Chen Automated semantic annotation and retrieval based on sharable ontology and case-based learning techniques JCDL03 2003 61 72
89. M. Srikanth, J. Varner, M. Bowden, and D. Moldovan Exploiting ontologies for automatic image annotation SIGIR05 2005 552 558
90. E.B. Sudderth, A. Torralba, W.T. Freeman, and A.S. Willsky Learning hierarchical models of scenes, objects, and parts ICCV05 vol. II 2005 1331 1338 (Pubitemid 44042334)
91. A. Thomas, V. Ferrari, B. Leibe, T. Tuytelaars, and L.V. Gool Shape-from-recognition: recognition enables meta-data transfer CVIU 2009
92. S. Todorovic, and N. Ahuja Learning subcategory relevances to category recognition CVPR08 2008
93. A. Torralba, R. Fergus, and W. Freeman 80 million tiny images: a large dataset for non-parametric object and scene recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 30 11 2008 1958 1970
94. A. Torralba, K.P. Murphy, and W.T. Freeman Sharing visual features for multiclass and multiview object detection IEEE Transactions on Pattern Analysis and Machine Intelligence 29 5 2007 854 869 (Pubitemid 46534781)
95. A.-M. Tousch, S. Herbin, and J.-Y. Audibert Semantic lattices for multiple annotation of images MIR08 2008 342 349
96. M.J. Tovée An Introduction to the Visual System 1996 Cambridge University Press
97. N. Vasconcelos Image indexing with mixture hierarchies CVPR01 2001 3 10
98. J. Vogel, and B. Schiele Semantic modeling of natural scenes for content-based image retrieval International Journal of Computer Vision 72 2 2007 133 157 (Pubitemid 46094757)
99. G. Wang, Y. Zhang, and L. Fei-Fei Using dependent regions for object categorization in a generative framework CVPR06 2006 1597 1604 (Pubitemid 44931510)
100. X. Wang, W. Ma, and X. Li Data-driven approach for bridging the cognitive gap in image retrieval ICME04 vol. 3 2004
101. P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, P. Perona, Caltech-UCSD Birds 200, California Institute of Technology, CNS-TR-2010-001, 2010.
102. C. Yang, M. Dong, and F. Fotouhi Learning the semantics in image retrieval - a natural language processing approach CVPRW04 2004 137
103. C. Yang, M. Dong, and F. Fotouhi Semantic feedback for interactive image retrieval MULTIMEDIA05 2005 415 418
104. A. Zweig, and D. Weinshall Exploiting object hierarchy: combining models from different category levels ICCV07 2007 1 8