Fast zero-shot image tagging

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 5985-5994

  Zhang, Yang ^a   Gong, Boqing ^a   Shah, Mubarak ^a  

^a UNIVERSITY OF CENTRAL FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; PATTERN RECOGNITION; VECTORS;

CONSTANT TIME; DEEP NEURAL NETWORKS; IMAGE TAGGING; LINEAR MAPPING; LINEAR VECTORS; PRINCIPAL DIRECTIONS; TAGGING MODELS; TRAINING SETS;

VECTOR SPACES;

EID: 84986272569     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.644     Document Type: Conference Paper

References (60)

1. Z. Akata, F. Perronnin, Z. Harchaoui, and C. Schmid. Label-Embedding for Attribute-Based Classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 819-826, June 2013.
2. Z. Akata, S. Reed, D. Walter, H. Lee, and B. Schiele. Evaluation of Output Embeddings for Fine-Grained Image Classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2927-2936, 2015.
3. K. Barnard, P. Duygulu, D. Forsyth, N. De Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. The Journal of Machine Learning Research, 3: 1107-1135, 2003.
4. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio. Theano: A CPU and GPU math expression compiler. In Proceedings of the Python for scientific computing conference (SciPy), volume 4, page 3. Austin, TX, 2010.
5. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning, pages 89-96. ACM, 2005.
6. G. Carneiro, A. B. Chan, P. J. Moreno, and N. Vasconcelos. Supervised learning of semantic classes for image annotation and retrieval. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29 (3): 394-410, 2007.
7. M. Chen, A. Zheng, and K. Weinberger. Fast image tagging. In Proceedings of the 30th international conference on Machine Learning, pages 1274-1282, 2013.
8. T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng. NUS-WIDE: A real-world web image database from National University of Singapore. In Proceedings of the ACM international conference on image and video retrieval, page 48. ACM, 2009.
9. C. Cortes and V. Vapnik. Support-vector networks. Machine learning, 20 (3): 273-297, 1995.
10. K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. The Journal of Machine Learning Research, 2: 265-292, 2002.
11. S. C. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. JAsIs, 41 (6): 391-407, 1990.
12. A. Dehghan, H. Idrees, and M. Shah. Improving Semantic Concept Detection through the Dictionary of Visuallydistinct Elements. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2585-2592, 2014.
13. A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1778-1785. IEEE, 2009.
14. S. Feng, R. Manmatha, and V. Lavrenko. Multiple bernoulli relevance models for image and video annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume 2, pages II-1002. IEEE, 2004.
15. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, T. Mikolov, and others. Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems, pages 2121-2129, 2013.
16. J. Fu, Y. Wu, T. Mei, J. Wang, H. Lu, and Y. Rui. Relaxing From Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging. In Proceedings of the IEEE International Conference on Computer Vision, pages 1985-1993, 2015.
17. Y. Fu, Y. Yang, T. Hospedales, T. Xiang, and S. Gong. Transductive Multi-label Zero-shot Learning. pages 7. 1-7. 11. British Machine Vision Association, 2014.
18. Y. Fu, Y. Yang, T. M. Hospedales, T. Xiang, and S. Gong. Transductive Multi-class and Multi-label Zero-shot Learning. ArXiv preprint arXiv: 1503. 07884, 2015.
19. Y. Gong, Y. Jia, T. Leung, A. Toshev, and S. Ioffe. Deep convolutional ranking for multilabel image annotation. ArXiv preprint arXiv: 1312. 4894, 2013.
20. M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In Computer Vision, 2009 IEEE 12th International Conference on, pages 309-316. IEEE, 2009.
21. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. ArXiv preprint arXiv: 1207. 0580, 2012.
22. D. Jayaraman and K. Grauman. Zero-shot recognition with unreliable attributes. In Advances in Neural Information Processing Systems, pages 3464-3472, 2014.
23. J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 119-126. ACM, 2003.
24. T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133-142. ACM, 2002.
25. M. Kalayeh, H. Idrees, and M. Shah. NMF-KNN: Image Annotation Using Weighted Multi-view Non-negative Matrix Factorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 184-191, June 2014.
26. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. ArXiv preprint arXiv: 1412. 2306, 2014.
27. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. ArXiv preprint arXiv: 1411. 2539, 2014.
28. C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 951-958, June 2009.
29. C. H. Lampert, H. Nickisch, and S. Harmeling. Attributebased classification for zero-shot visual object categorization. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 36 (3): 453-465, 2014.
30. V. Lavrenko, R. Manmatha, and J. Jeon. A model for learning the semantics of pictures. In Advances in neural information processing systems, page None, 2003.
31. R. Lebret, P. Pinheiro, and R. Collobert. Phrase-based Image Captioning. In D. Blei and F. Bach, editors, Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 2085-2094. JMLRWorkshop and Conference Proceedings, 2015.
32. S. Lee, W. De Neve, and Y. M. Ro. Visually weighted neighbor voting for image tag relevance learning. Multimedia tools and applications, 72 (2): 1363-1386, 2014.
33. X. Li, C. G. Snoek, and M. Worring. Learning social tag relevance by neighbor voting. Multimedia, IEEE Transactions on, 11 (7): 1310-1322, 2009.
34. X. Li, T. Uricchio, L. Ballan, M. Bertini, C. G. M. Snoek, and A. Del Bimbo. Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval. ArXiv: 1503. 08248 [cs], Mar. 2015. ArXiv: 1503. 08248.
35. A. Makadia, V. Pavlovic, and S. Kumar. Baselines for image annotation. International Journal of Computer Vision, 90 (1): 88-105, 2010.
36. T. Mei, Y. Wang, X.-S. Hua, S. Gong, and S. Li. Coherent image annotation by learning semantic distance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2008.
37. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient Estimation of Word Representations in Vector Space. ArXiv: 1301. 3781 [cs], Jan. 2013. ArXiv: 1301. 3781.
38. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111-3119, 2013.
39. T. Mikolov, W.-t. Yih, and G. Zweig. Linguistic Regularities in Continuous SpaceWord Representations. In HLT-NAACL, pages 746-751, 2013.
40. F. Monay and D. Gatica-Perez. PLSA-based image autoannotation: constraining the latent space. In Proceedings of the 12th annual ACM international conference on Multimedia, pages 348-351. ACM, 2004.
41. E. Moxley, T. Mei, and B. S. Manjunath. Video annotation through search and graph reinforcement mining. Multimedia, IEEE Transactions on, 12 (3): 184-193, 2010.
42. Z. Niu, G. Hua, X. Gao, and Q. Tian. Semi-supervised relational topic model for weakly annotated image recognition in social media. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4233-4240. IEEE, 2014.
43. M. Norouzi, T. Mikolov, S. Bengio, Y. Singer, J. Shlens, A. Frome, G. S. Corrado, and J. Dean. Zero-shot learning by convex combination of semantic embeddings. ArXiv preprint arXiv: 1312. 5650, 2013.
44. M. Palatucci, D. Pomerleau, G. E. Hinton, and T. M. Mitchell. Zero-shot learning with semantic output codes. In Advances in neural information processing systems, pages 1410-1418, 2009.
45. J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), 12, 2014.
46. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. Technical report, DTIC Document, 1985.
47. B. Sigurbjrnsson and R. Van Zwol. Flickr tag recommendation based on collective knowledge. In Proceedings of the 17th international conference on World Wide Web, pages 327-336. ACM, 2008.
48. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv: 1409. 1556, 2014.
49. R. Socher, J. Bauer, C. D. Manning, and A. Y. Ng. Parsing with compositional vector grammars. In In Proceedings of the ACL conference. Citeseer, 2013.
50. R. Socher, M. Ganjoo, C. D. Manning, and A. Ng. Zero-shot learning through cross-modal transfer. In Advances in neural information processing systems, pages 935-943, 2013.
51. A. Tariq and H. Foroosh. Feature-Independent Context Estimation for Automatic Image Annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1958-1965, 2015.
52. S. Tellex, B. Katz, J. Lin, A. Fernandes, and G. Marton. Quantitative evaluation of passage retrieval algorithms for question answering. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 41-47. ACM, 2003.
53. G. Tsoumakas and I. Katakis. Multi-label classification: An overview. Dept. of Informatics, Aristotle University of Thessaloniki, Greece, 2006.
54. L. Van der Maaten and G. Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9 (2579-2605): 85, 2008.
55. J. Weston, S. Bengio, and N. Usunier. Large scale image annotation: learning to rank with joint word-image embeddings. Machine learning, 81 (1): 21-35, 2010.
56. J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In IJCAI, volume 11, pages 2764-2770, 2011.
57. F. Xia, T.-Y. Liu, J. Wang, W. Zhang, and H. Li. Listwise approach to learning to rank: Theory and algorithm. In Proceedings of the 25th international conference on Machine learning, pages 1192-1199. ACM, 2008.
58. O. Yakhnenko and V. Honavar. Annotating images and image objects using a hierarchical dirichlet process model. In Proceedings of the 9th International Workshop on Multimedia Data Mining: held in conjunction with the ACM SIGKDD 2008, pages 1-7. ACM, 2008.
59. X. Zhu, W. Nejdl, and M. Georgescu. An adaptive teleportation random walk model for learning social tag relevance. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, pages 223-232. ACM, 2014.
60. W. Y. Zou, R. Socher, D. M. Cer, and C. D. Manning. Bilingual Word Embeddings for Phrase-Based Machine Translation. In EMNLP, pages 1393-1398, 2013.