Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization

Neurocomputing

Volumn 76, Issue 1, 2012, Pages 50-60

  Caicedo, Juan C ^a   BenAbdallah, Jaafar ^b   González, Fabio A ^a   Nasraoui, Olfa ^b  

^a UNIVERSIDAD NACIONAL DE COLOMBIA   (Colombia)

^b University of Louisville   (United States)

Author keywords

Content based image retrieval; Image annotation; Image collection analysis; Latent semantic indexing; Multimodal indexing; Multimodal representation; Non negative matrix factorization

Indexed keywords

CONTENT BASED IMAGE RETRIEVAL; IMAGE ANNOTATION; IMAGE COLLECTIONS; LATENT SEMANTIC INDEXING; MULTI-MODAL; NONNEGATIVE MATRIX FACTORIZATION;

BLIND SOURCE SEPARATION; IMAGE RETRIEVAL; INDEXING (OF INFORMATION); SEMANTICS; SINGULAR VALUE DECOMPOSITION; USER INTERFACES;

FACTORIZATION;

ALGORITHM; ARTICLE; AUTOMATIC IMAGE ANNOTATION; AUTOMATION; CONTROLLED STUDY; IMAGE ANALYSIS; IMAGE INDEXING; IMAGE PROCESSING; IMAGE RETRIEVAL; INFORMATION PROCESSING; MATHEMATICAL PHENOMENA; NONNEGATIVE MATRIX FACTORIZATION; PRIORITY JOURNAL; SINGULAR VALUE DECOMPOSITION;

EID: 80555123004     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.04.037     Document Type: Article

References (44)

1. Smeulders A.W.M., Worring M., Santini S., Gupta A., Jain R. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(12):1349-1380.
2. Atrey P., Hossain M., ElSaddik A., Kankanhalli M. Multimodal fusion for multimedia analysis: a survey. Multimedia Systems 2010, 16(6):345-379.
3. P. Duygulu, K. Barnard, J. de Freitas, D. Forsyth, Object recognition as machine translation: learning a lexicon for a fixed image vocabulary, in: Computer Vision-ECCV 2002, Lecture Notes in Computer Science, 2002, pp. 349-354 (Chapter 7).
4. Huiskes M.J., Lew M.S. The MIRFlickr retrieval evaluation. MIR '08: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval 2008, ACM, New York, NY, USA.
5. F.A. González, J.C. Caicedo, O. Nasraoui, J. Ben-Abdallah, NMF-based multimodal image indexing for querying by visual example, in: Proceedings of the ACM International Conference on Image and Video Retrieval, Xi'an, China, July 05-07, 2010, ACM CIVR '10, 2010, pp. 366-373.
6. J. Ben-Abdallah, J.C. Caicedo, F.A. González, O. Nasraoui, Multimodal image annotation using non-negative matrix factorization, in: Proceedings of the IEEE Web Intelligence Conference, Toronto, Canada, August 31, 2010, IEEE-WIC 10, 2010.
7. Datta R., Joshi D., Li J., Wang J.Z. Image retrieval: ideas, influences, and trends of the new age. ACM Computing Surveys 2008, 40(2):1-60.
8. Ah-Pine J., Bressan M., Clinchant S., Csurka G., Hoppenot Y., Renders J.-M. Crossing textual and visual content in different application scenarios. Multimedia Tools and Applications 2009, 42(1):31-56.
9. Agrawal R., Grosky W., Fotouhi F. Image retrieval using multimodal keywords. International Symposium on Multimedia 2006, 817-822.
10. Song H., Li X., Wang P. Multimodal image retrieval based on annotation keywords and visual content. International Workshop on Computer-Aided Software Engineering 2009, 295-298.
11. Barnard K., Duygulu P., Forsyth D., de Freitas N., Blei D.M., Jordan M.I. Matching words and pictures. Journal of Machine Learning Research 2003, 3:1107-1135.
12. D.M. Blei, M.I. Jordan, Modeling annotated data, in: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '03, ACM, New York, NY, USA, 2003, pp. 127-134. doi:10.1145/860435.860460.
13. Monay F., Gatica-Perez D. Modeling semantic aspects for cross-media image indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 2007, 29(10):1802-1817.
14. Hare J.S., Samangooei S., Lewis P.H., Nixon M.S. Semantic spaces revisited: investigating the performance of auto-annotation and semantic retrieval using semantic spaces. CIVR '08: Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval 2008, 359-368. ACM, New York, NY, USA.
15. Tang J., Lewis P.H. Non-negative Matrix Factorisation for Object Class Discovery and Image Auto-annotation 2008, ACM Press, New York, New York, USA, doi:10.1145/1386352.1386370. URL 〈http://portal.acm.org/citation.cfm?id=1386352.1386370〉.
16. Gupta S.K., Phung D., Adams B., Tran T., Venkatesh S. Nonnegative Shared Subspace Learning and its Application to Social Media Retrieval 2010, ACM Press, New York, New York, USA, doi:10.1145/1835804.1835951. URL 〈http://portal.acm.org/citation.cfm?id=1835804.1835951〉.
17. M. Cooper, J. Foote, Summarizing video using non-negative similarity matrix factorization, in: IEEE Workshop on Multimedia Signal Processing, 2002.
18. W. Liu, N. Zheng, X. Lu, Non-negative matrix factorization for visual coding, in: IEEE International Conference on Acoustics, Speech and Signal Processing, 2003.
19. Liang D., Yang J., Chang Y. Supervised non-negative matrix factorization based latent semantic image indexing. Chinese Optics Letters 2006, 4:272-274.
20. Smeaton A.F., Quigley I. Experiments on using semantic distances between words in image caption retrieval. SIGIR '96: Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 1996, 174-180. ACM, New York, NY, USA.
21. Bosch A., Muñoz X., Martí R. Which is the best way to organize/classify images by content?. Image and Vision Computing 2007, 25(6):778-791.
22. Manning C.D., Raghavan P., Schütze H. Introduction to Information Retrieval 2008, Cambridge University Press.
23. Hofmann T. Probabilistic latent semantic indexing. Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '99 1999, 50-57. ACM, New York, NY, USA, doi:10.1145/312624.312649.
24. C. Liu, H. chih Yang, J. Fan, L.-W. He, Y.-M. Wang, Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce, in: 19th International Conference on World Wide Web (WWW '10), ACM, New York, NY, USA, 2010.
25. Lee D.D., Seung H.S. Algorithms for nonnegative matrix factorization. Advances in Neural Information Processing Systems 2001, 13:556-562.
26. V.Y.F. Tan, C. Fevotte, Automatic relevance determination in nonnegative matrix factorization, in: SPARS'09-Signal Processing with Adaptive Sparse Structured Representations, 2009.
27. Hare J., Lewis P., Enser P., Sandom C. A linear-algebraic technique with an application in semantic image retrieval. Lecture Notes in Computer Science 2006, vol. 4071:31.
28. T. Tsikrika, J. Kludas, Overview of the wikipediamm task at imageclef 2009, in: Cross Language Evaluation Forum (CLEF) Working Notes, 2009.
29. H. M"uller, I. Eggel, S. Bedrick, S. Radhouani, B. Bakke, C. Kahn Jr., W. Hersh, Overview of the clef 2009 medical image retrieval track, in: Cross Language Evaluation Forum (CLEF) Working Notes, 2009.
30. Lee D.D., Seung H.S. Learning the parts of objects by non-negative matrix factorization. Nature 1999, 401(6755):788-791.
31. Makadia A., Pavlovic V., Kumar S. A new baseline for image annotation. ECCV '08: Proceedings of the 10th European Conference on Computer Vision 2008, 316-329. Springer-Verlag, Berlin, Heidelberg.
32. Verbeek J., Guillaumin M., Mensink T., Schmid C. Image annotation with tagprop on the MIRFlickr set. MIR '10: Proceedings of the International Conference on Multimedia Information Retrieval 2010, 537-546. ACM, New York, NY, USA.
33. Jeon J., Lavrenko V., Manmatha R. Automatic image annotation and retrieval using cross-media relevance models. SIGIR '03: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2003, 119-126. ACM, New York, NY, USA.
34. Rasiwasia N., Moreno P.J., Vasconcelos N. Bridging the gap: query by semantic example. IEEE Transactions on Multimedia 2007, 9(5):923-938.
35. Hare J., Lewis P. Automatically annotating the MIRFlickr dataset: experimental protocols. MIR '10: Proceedings of the International Conference on Multimedia Information Retrieval 2010, 547-556. ACM.
36. J.E. Camargo, J.C. Caicedo, A.M. Chavarro, F.A. Gonzalez, A kernel-based strategy for exploratory image collection search, in: Proceedings of the 2010 International Workshop on Content-Based Multimedia Indexing (CBMI), 2010, pp. 1-6. doi:10.1109/CBMI.2010.5529893.
37. Xie L., Chang S.F. Pattern mining in visual concept streams. IEEE International Conference on Multimedia and Expo 2006, 297-300. doi: 10.1109/ICME.2006.262457.
38. J. Han, Data mining for image/video processing: a promising research frontier, in: Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, CIVR '08, ACM, New York, NY, USA, 2008, pp. 1-2. doi:10.1145/1386352.1386353.
39. M. Inoue, On the need for annotation-based image retrieval, in: Workshop on Information Retrieval in Context, 2004.
40. Clough P., Grubinger M., Deselaers T., Hanbury A., M"uller H. Overview of the imageclef 2006 photographic retrieval and object annotation tasks. Evaluation of Multilingual and Multi-modal Information Retrieval 2007, 579-594.
41. Fan X., Xie X., Li Z., Li M., Ma W. Photo-to-search: using multimodal queries to search the web from mobile devices. Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval 2005, 143-150. ACM, Hilton, Singapore.
42. Yeh T., Grauman K., Tollmar K., Darrell T. A picture is worth a thousand keywords: image-based object search on a mobile platform. CHI '05 Extended Abstracts on Human Factors in Computing Systems 2005, 2025-2028. ACM, Portland, OR, USA.
43. Xu W., Liu X., Gong Y. Document clustering based on non-negative matrix factorization. SIGIR '03: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2003, 267-273. ACM, New York, NY, USA.
44. Ding C., Li T., Peng W. On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Computational Statistics & Data Analysis 2008, 52(8):3913-3927.