Im2Text and Text2Im: Associating images and texts for cross-modal retrieval

BMVC 2014 - Proceedings of the British Machine Vision Conference 2014

Volumn , Issue , 2014, Pages

  Verma, Yashaswi ^a   Jawahar, C V ^a  

^a INTERNATIONAL INSTITUTE OF INFORMATION TECHNOLOGY   (India)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; FORECASTING; SEMANTICS; STATISTICS; SUPPORT VECTOR MACHINES;

CANONICAL CORRELATION ANALYSIS; LATENT DIRICHLET ALLOCATION; PREDICTION TASKS; QUERY IMAGES; RETRIEVAL FRAMEWORKS; SEMANTIC ASSOCIATIONS; TEXTUAL DATA; UNIFIED FORMULATIONS;

PROBABILITY DISTRIBUTIONS;

EID: 85088059797     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.5244/c.28.97     Document Type: Conference Paper

References (40)

1. D. Blei, A. Ng, and M. Jordan. Latent dirichlet allocation. JMLR, 12(1):234-278, 2003.
2. R. Datta, D. Joshi, J. Li, and J. Wang. Image retrieval: Ideas, influences and trends of new age. ACM Computing Surveys, 40(2):1-60, 2008.
3. Kun Duan, David J. Crandall, and Dhruv Batra. Multimodal learning in looselyorganized web images. In CVPR, 2014.
4. P. Duygulu, K. Barnard, J. F. G. de Freitas, and D. A. Forsyth. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In ECCV, 2002.
5. Ali Farhadi, Mohsen Hejrati, Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. Every picture tells a story: Generating sentences for images. In ECCV, 2010.
6. S. L. Feng, R. Manmatha, and V. Lavrenko. Multiple Bernoulli relevance models for image and video annotation. In CVPR, 2004.
7. Yunchao Gong, Qifa Ke, Michael Isard, and Svetlana Lazebnik. A multi-view embedding space for modeling internet images, tags, and their semantics. IJCV, 106(2): 210-233, 2013.
8. M. Grubinger. Analysis and Evaluation of Visual Information Systems Performance. PhD thesis, Victoria University, Melbourne, Australia, 2007.
9. Sergio Guadarrama, Niveda Krishnamoorthy, Girish Malkarnenkar, Subhashini Venugopalan, Raymond Mooney, Trevor Darrell, and Kate Saenko. YouTube2Text: Recognizing and describing arbitrary activities using semantic hierarchies and zero-shot recognition. In ICCV, 2013.
10. M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. Tagprop: Discriminative metric learning in nearest neighbour models for image auto-annotation. In ICCV, 2009.
11. Ankush Gupta, Yashaswi Verma, and C. V. Jawahar. Choosing linguistics over vision to describe images. In AAAI, 2012.
12. Micah Hodosh, Peter Young, and Julia Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. JAIR, 47:853-899, 2013.
13. H. Hotelling. Relations between two sets of variates. Biometrika, 28:321-377, 1936.
14. Girish Kulkarni, Visruth Premraj, Sagnik Dhar, Siming Li, Yejin Choi, Alexander C. Berg, and Tamara L. Berg. Baby Talk: Understanding and generating simple image descriptions. In CVPR, 2011.
15. Polina Kuznetsova, Vicente Ordonez, Alexander C. Berg, Tamara L. Berg, and Yejin Choi. Collective generation of natural image descriptions. In ACL, 2012.
16. Siming Li, Girish Kulkarni, Tamara L. Berg, Alexander C. Berg, and Yejin Choi. Composing simple image descriptions using web-scale n-grams. In CoNLL, 2011.
17. C.-Y. Lin and E. Hovy. Automatic evaluation of summaries using n-gram co-occurrence statistics. In NAACLHLT, 2003.
18. David G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60 (2):91-110, 2004.
19. Ameesh Makadia, Vladimir Pavlovic, and Sanjiv Kumar. A new baseline for image annotation. In ECCV, 2008.
20. C. Meadow, B. Boyce, D. Kraft, and C. Barry. Text information retrieval systems. Emerald Group Pub Ltd, 2007.
21. Margaret Mitchell, Jesse Dodge, Amit Goyal, Kota Yamaguchi, Karl Sratos, Xufeng Han, Alysssa Mensch, Alexander C. Berg, Tamara L. Berg, and Hal Daumé III. Midge: Generating image descriptions from computer vision detections. In EACL, 2012.
22. Vicente Ordonez, Girish Kulkarni, and Tamara L. Berg. Im2text: Describing images using 1 million captioned photographs. In NIPS, 2011.
23. K. Papineni, S. Roukos, T. Ward, and W. Zhu. Bleu: A method for automatic evaluation of machine translation. In ACL, 2002.
24. M. Paramita, M. Sanderson, and P. Clough. Diversity in photo retrieval: overview of the imageclefphoto task 2009. CLEF working notes, 2009.
25. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. Collecting image annotation using amazon's mechanical turk. In NAACLHLT Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, 2010.
26. N. Rasiwasia, J. C. Pereira, E. Coviello, G. Doyle, G. R. G. Lanckriet, R. Levy, and N. Vasconcelos. A new approach to cross-modal multimedia retrieval. In ACM MM, 2010.
27. Jose Rodriguez and Florent Perronnin. Label embedding for text recognition. In BMVC, 2013.
28. Marcus Rohrbach, Wei Qiu, and Ivan Titov. Translating video content to natural language descriptions. In ICCV, 2013.
29. M. A. Sadeghi and A. Farhadi. Recognition using visual phrases. In CVPR, 2011.
30. A. F. Smeaton, P. Over, andW. Kraaij. Evaluation campaigns and trecvid. In MIR: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, 2006.
31. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. PAMI, 22(12):1349-1380, 2000.
32. Ioannis Tsochantaridis, Thomas Hofmann, Thorsten Joachims, and Yasemin Altun. Support vector machine learning for interdependent and structured output spaces. In ICML, 2004.
33. Yoshitaka Ushiku, Tatsuya Harada, and Yasuo Kuniyoshi. Understanding images with natural sentences. In ACM MM, 2011.
34. A. Vedaldi. A MATLAB wrapper of SVMstruct. http://www.vlfeat.org/~vedaldi/code/svm-struct-matlab.html, 2011.
35. Yashaswi Verma and C. V. Jawahar. Image annotation using metric learning in semantic neighbourhoods. In ECCV, 2012.
36. Yashaswi Verma and C. V. Jawahar. Exploring SVM for image annotation in presence of confusing labels. In BMVC, 2013.
37. Yashaswi Verma, Ankush Gupta, Prashanth Mannem, and C. V. Jawahar. Generating image descriptions using semantic similarities in the output space. In V&L Net Workshop on Language for Vision, in conjunction with CVPR, 2013.
38. Jason Weston, Samy Bengio, and Nicolas Usunier. WSABIE: Scaling up to large vocabulary image annotation. In IJCAI, 2011.
39. Y. Yang, C. L. Teo, Hal Daumé III, and Y. Aloimonos. Corpus-guided sentence generation of natural images. In EMNLP, 2011.
40. B. Z. Yao, X. Yang, L. Lin, M. W. Lee, and S.-C. Zhu. I2T: Image parsing to text description. In Proceedings of the IEEE, 2008.