Deep visual-semantic hashing for cross-modal retrieval

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn 13-17-August-2016, Issue , 2016, Pages 1445-1454

  Cao, Yue ^a   Long, Mingsheng ^a   Wang, Jianmin ^a   Yang, Qiang ^c   Yuy, Philip S ^b  

^a TSINGHUA UNIVERSITY   (China)

^b UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

^c HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Cross modal retrieval; Deep hashing; Multimodal embedding

Indexed keywords

CODES (SYMBOLS); DATA MINING; DIGITAL STORAGE; HASH FUNCTIONS; IMAGE RETRIEVAL; NEAREST NEIGHBOR SEARCH; NETWORK ARCHITECTURE; NEURAL NETWORKS; RECURRENT NEURAL NETWORKS; SEMANTICS;

CONVOLUTIONAL NEURAL NETWORK; CROSS-MODAL; DEEP HASHING; FEATURE REPRESENTATION; MULTI-MODAL; SIMILARITY PRESERVING; SPATIAL DEPENDENCIES; STORAGE AND RETRIEVALS;

VISUAL LANGUAGES;

EID: 84985036624     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2939672.2939812     Document Type: Conference Paper

References (46)

1. J. Andreas, M. Rohrbach, T. Darrell, and D. Klein. Deep compositional question answering with neural module networks. CVPR, 2016.
2. G. Andrew, R. Arora, J. Bilmes, and K. Livescu. Deep canonical correlation analysis. In ICML, 2013.
3. Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. TPAMI, 35, 2013.
4. M. Bronstein, A. Bronstein, F. Michel, and N. Paragios. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In CVPR. IEEE, 2010.
5. Y. Cao, M. Long, J. Wang, H. Zhu, and Q. Wen. Deep quantization network for Effcient image retrieval. In AAAI, 2016.
6. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015.
7. J. Donahue, Y. Jia, O. Vinyals, J. Homan, N. Zhang, E. Tzeng, and T. Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In ICML, 2014.
8. F. Feng, X. Wang, and R. Li. Cross-modal retrieval with correspondence autoencoder. In MM. ACM, 2014.
9. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, T. Mikolov, et al. Devise: A deep visual-semantic embedding model. In NIPS, pages 2121-2129, 2013.
10. H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu. Are you talking to a machine? dataset and methods for multilingual image question answering. In NIPS, 2015.
11. A. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In ICML, pages 1764-1772, 2014.
12. A. Gretton, K. Borgwardt, M. Rasch, B. Scholkopf, and A. Smola. A kernel two-sample test. JMLR, 13:723-773, Mar. 2012.
13. M. Grubinger, P. Clough, H. Muller, and T. Deselaers. The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In International Workshop OntoImage, pages 13-23, 2006.
14. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.
15. R. Hu, H. Xu, M. Rohrbach, J. Feng, K. Saenko, and T. Darrell. Natural language object retrieval. CVPR, 2016.
16. Y. Hu, Z. Jin, H. Ren, D. Cai, and X. He. Iterative multi-view hashing for cross media indexing. In MM. ACM, 2014.
17. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Cae: Convolutional architecture for fast feature embedding. In MM. ACM, 2014.
18. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, pages 3128-3137, 2015.
19. R. Kiros, R. Salakhutdinov, and R. Zemel. Multimodal neural language models. In T. Jebara and E. P. Xing, editors, ICML, pages 595-603. JMLR Workshop and Conference Proceedings, 2014.
20. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. In NIPS, 2014.
21. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
22. S. Kumar and R. Udupa. Learning hash functions for cross-view similarity search. In IJCAI, 2011.
23. H. Lai, Y. Pan, Y. Liu, and S. Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR. IEEE, 2015.
24. T. Lin, M. Maire, S. J. Belongie, L. D. Bourdev, R. B. Girshick, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: common objects in context. CoRR, abs/1405. 0312, 2014.
25. Z. Lin, G. Ding, M. Hu, and J. Wang. Semantics-preserving hashing for cross-view retrieval. In CVPR, 2015.
26. W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang. Supervised hashing with kernels. In CVPR. IEEE, 2012.
27. X. Liu, J. He, C. Deng, and B. Lang. Collaborative hashing. In CVPR. IEEE, 2014.
28. M. Long, Y. Cao, J. Wang, and M. I. Jordan. Learning transferable features with deep adaptation networks. In ICML, 2015.
29. M. Long, Y. Cao, J. Wang, and P. S. Yu. Composite correlation quantization for Effcient multimodal search. SIGIR, 2016.
30. J. Masci, M. M. Bronstein, A. M. Bronstein, and J. Schmidhuber. Multimodal similarity-preserving hashing. TPAMI, 36, 2014.
31. M. Ou, P. Cui, F. Wang, J. Wang, W. Zhu, and S. Yang. Comparing apples to oranges: A scalable solution with heterogeneous hashing. In KDD, pages 230-238. ACM, 2013.
32. A. W. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. TPAMI, 22, 2000.
33. J. Song, Y. Yang, Y. Yang, Z. Huang, and H. T. Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In SIGMOD. ACM, 2013.
34. N. Srivastava and R. Salakhutdinov. Multimodal learning with deep boltzmann machines. JMLR, 15, 2014.
35. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.
36. J. Wang, H. T. Shen, J. Song, and J. Ji. Hashing for similarity search: A survey. arXiv preprint arXiv:1408. 2927, 2014.
37. W. Wang, B. C. Ooi, X. Yang, D. Zhang, and Y. Zhuang. effective multi-modal retrieval based on stacked auto-encoders. In VLDB. ACM, 2014.
38. Y. Wei, Y. Song, Y. Zhen, B. Liu, and Q. Yang. Scalable heterogeneous translated hashing. In KDD, pages 791-800. ACM, 2014.
39. B. Wu, Q. Yang, W. Zheng, Y. Wang, and J. Wang. Quantized correlation hashing for fast cross-modal search. In IJCAI, 2015.
40. R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan. Supervised hashing for image retrieval via image representation learning. In AAAI. AAAI, 2014.
41. Z. Yu, F. Wu, Y. Yang, Q. Tian, J. Luo, and Y. Zhuang. Discriminative coupled dictionary hashing for fast cross-media retrieval. In SIGIR. ACM, 2014.
42. W. Zaremba and I. Sutskever. Learning to execute. CoRR, abs/1410. 4615, 2014.
43. D. Zhang and W. Li. Large-scale supervised multimodal hashing with semantic correlation maximization. In AAAI, 2014.
44. Y. Zhen and D.-Y. Yeung. Co-regularized hashing for multimodal data. In NIPS, 2012.
45. Y. Zhen and D.-Y. Yeung. A probabilistic model for multimodal hash function learning. In SIGKDD. ACM, 2012.
46. H. Zhu, M. Long, J. Wang, and Y. Cao. Deep hashing network for Effcient similarity retrieval. In AAAI, 2016.