Learning Compact Hash Codes for Multimodal Representations Using Orthogonal Deep Structure

IEEE Transactions on Multimedia

Volumn 17, Issue 9, 2015, Pages 1404-1416

  Wang, Daixin ^a   Cui, Peng ^a   Ou, Mingdong ^a   Zhu, Wenwu ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Deep learning; multimodal hashing; Similarity search

Indexed keywords

CODES (SYMBOLS);

DEEP LEARNING; LEARNING ABILITIES; MULTI-MODAL; MULTIPLE MODALITIES; ORTHOGONAL STRUCTURES; REAL-WORLD DATASETS; SIMILARITY SEARCH; WEIGHTING MATRICES;

HASH FUNCTIONS;

EID: 84939232131     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2015.2455415     Document Type: Article

References (55)

1. X. Li, C. Shen, A. Dick, and A. van den Hengel, "Learning compact binary codes for visual tracking," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2013, pp. 2419-2426.
2. H. Fan, M. Yang, Z. Cao, Y. Jiang, and Q. Yin, "Learning compact face representation: Packing a face into an int32," in Proc. ACM Int. Conf. Multimedia, 2014, pp. 933-936.
3. J. A. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol. 9, no. 3, pp. 293-300, 1999.
4. A. Hyvärinen and E. Oja, "Independent component analysis: Algorithms and applications," Neural Netw., vol. 13, no. 4, pp. 411-430, 2000.
5. S. Wold, K. Esbensen, and P. Geladi, "Principal component analysis," Chemometrics Intell. Lab. Syst., vol. 2, no. 1, pp. 37-52, 1987.
6. N. Srivastava and R. Salakhutdinov, "Learning representations for multimodal data with deep belief nets," in Proc. Int. Conf. Mach. Learning Workshop, 2012.
7. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learning, vol. 2, no. 1, pp. 1-127, 2009.
8. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
9. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. NIPS, 2012, vol. 1, no. 2, pp. 1097-1105.
10. A. Bordes, X. Glorot, J. Weston, and Y. Bengio, "Joint learning of words and meaning representations for open-text semantic parsing," in Proc. Int. Conf. Artificial Intell. Statist., 2012, pp. 127-135.
11. D. Erhan et al., "Why does unsupervised pre-training help deep learning," JMLR, vol. 11, pp. 625-660, 2010.
12. J. Ngiam et al., "Multimodal deep learning," in Proc. ICML, 2011, pp. 689-696.
13. N. Srivastava and R. Salakhutdinov, "Multimodal learning with deep Boltzmann machines," in Proc. NIPS, 2012, pp. 2231-2239.
14. M. Norouzi and D. M. Blei, "Minimal loss hashing for compact binary codes," in Proc. ICML, 2011, pp. 353-360.
15. Y. Kang, S. Kim, and S. Choi, "Deep learning to hash with multiple representations," in Proc. ICDM, 2012, pp. 930-935.
16. J. Ngiam, A. Coates, A. Lahiri, B. Prochnow, Q. V. Le, and A. Y. Ng, "On optimization methods for deep learning," in Proc. ICML, 2011, pp. 265-272.
17. M. Guillaumin, J. Verbeek, and C. Schmid, "Multimodal semi-supervised learning for image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 902-909.
18. M. J. Huiskes, B. Thomee, and M. S. Lew, "New trends and ideas in visual concept detection: The MIR Flickr retrieval evaluation initiative," in Proc. Int. Conf. Multimedia Inf. Retrieval, 2010, pp. 527-536.
19. S. Roller and S. S. Im Walde, "A multimodal LDA model integrating textual, cognitive and visual modalities," in Proc. Conf. Empirical Methods Natural Language Process., 2013, pp. 1146-1157.
20. M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, "Locality-sensitive hashing scheme based on p-stable distributions," in Proc. SOCG, 2004, pp. 253-262.
21. Q. Lv, W. Josephson, Z. Wang, M. Charikar, and K. Li, "Multi-probe LSH: Efficient indexing for high-dimensional similarity search," in Proc. VLDB, 2007, pp. 950-961.
22. A. Joly and O. Buisson, "A posteriori multi-probe locality sensitive hashing," in Proc. ACM Multimedia, 2008, pp. 209-218.
23. Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing," in Proc. NIPS, 2008, vol. 9, no. 1, p. 6.
24. R. Salakhutdinov and G. Hinton, "Semantic hashing," IJAR, vol. 50, no. 7, pp. 969-978, 2009.
25. A. Torralba, R. Fergus, and Y. Weiss, "Small codes and large image databases for recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2008, pp. 1-8.
26. R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, "Supervised hashing for image retrieval via image representation learning," in Proc. AAAI, 2014, pp. 2156-2162.
27. W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang, "Supervised hashing with kernels," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2074-2081.
28. M. Norouzi, D. J. Fleet, and R. Salakhutdinov, "Hamming distance metric learning," in Proc. NIPS, 2012, pp. 1070-1078.
29. J. Wang, S. Kumar, and S.-F. Chang, "Semi-supervised hashing for scalable image retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 3424-3431.
30. M. M. Bronstein, A. M. Bronstein, F. Michel, and N. Paragios, "Data fusion through cross-modality metric learning using similarity-sensitive hashing," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 3594-3601.
31. D. Zhang, F. Wang, and L. Si, "Composite hashing with multiple information sources," in Proc. SIGIR, 2011, pp. 225-234.
32. Y. Zhen and D.-Y. Yeung, "A probabilistic model for multimodal hash function learning," in Proc. SIGKDD, 2012, pp. 940-948.
33. D. Zhai et al., "Parametric local multimodal hashing for cross-view similarity search," in Proc. IJCAI, 2013, pp. 2754-2760.
34. J. Song, Y. Yang, Y. Yang, Z. Huang, and H. T. Shen, "Inter-media hashing for large-scale retrieval from heterogeneous data sources," in Proc. SIGMOD, 2013, pp. 785-796.
35. F. Wu et al., "Sparse multi-modal hashing," IEEE Trans. Multimedia, vol. 16, no. 2, pp. 427-439, Feb. 2014.
36. D. Zhang and W.-J. Li, "Large-scale supervised multimodal hashing with semantic correlation maximization," in Proc. AAAI, 2014, pp. 2177-2183.
37. S. Kumar and R. Udupa, "Learning hash functions for cross-view similarity search," in Proc. IJCAI, 2011, vol. 22, no. 1, pp. 1360-1366.
38. M. Rastegari, J. Choi, S. Fakhraei, D. Hal, and L. Davis, "Predictable dual-view hashing," in Proc. ICML, 2013, pp. 1328-1336.
39. M. Ou et al., "Comparing apples to oranges: A scalable solution with heterogeneous hashing," in Proc. SIGKDD, 2013, pp. 230-238.
40. W. Wang, B. C. Ooi, X. Yang, D. Zhang, and Y. Zhuang, "Effective multi-modal retrieval based on stacked auto-encoders," in Proc. VLDB Endowment, 2014, vol. 7, no. 8, pp. 649-660.
41. F. Feng, X. Wang, and R. Li, "Cross-modal retrieval with correspondence autoencoder," in Proc. ACM Multimedia, 2014, pp. 7-16.
42. G. Hinton, "A practical guide to training restricted Boltzmann machines," Momentum, vol. 9, no. 1, pp. 926-945, 2010.
43. M. Welling, M. Rosen-Zvi, and G. E. Hinton, "Exponential family harmoniums with an application to information retrieval," in Proc. NIPS, 2004, pp. 1481-1488.
44. G. E. Hinton and R. Salakhutdinov, "Replicated softmax: An undirected topic model," in Proc. NIPS, 2009, pp. 1607-1614.
45. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, no. 8, pp. 1771-1800, 2002.
46. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," CoRR, vol. abs/1207.0580, 2012 [Online]. Available: http://arxiv.org/abs/1207.0580
47. Z. Wu, Y.-G. Jiang, J. Wang, J. Pu, and X. Xue, "Exploring inter-feature and inter-class relationships with deep neural networks for video classification," in Proc. ACM Int. Conf. Multimedia, 2014, pp. 167-176.
48. N. Rasiwasia et al., "A new approach to cross-modal multimedia retrieval," in Proc. ACM Multimedia, 2010, pp. 251-260.
49. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent dirichlet allocation," JMLR, vol. 3, pp. 993-1022, 2003.
50. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth, "Every picture tells a story: Generating sentences from images," in Computer Vision-ECCV 2010. New York, NY, USA: Springer, 2010, pp. 15-29.
51. T.-S. Chua et al., "Nus-wide: A real-world web image database from National University of Singapore," in Proc. CIVR, 2009, pp. 48-56.
52. D. G. Lowe, "Object recognition from local scale-invariant features," in Proc. 7th IEEE Int. Conf. Comput. Vis., Sep. 1999, vol. 2, pp. 1150-1157.
53. W. Liu, J. Wang, S. Kumar, and S.-F. Chang, "Hashing with graphs," in Proc. 28th Int. Conf. Mach. Learning, 2011, pp. 1-8.
54. Y. Gong and S. Lazebnik, "Iterative quantization: A procrustean approach to learning binary codes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 817-824.
55. W. Wang, B. C. Ooi, X. Yang, D. Zhang, and Y. Zhuang, "Effective multi-modal retrieval based on stacked auto-encoders," in Proc. PVLDB, 2014, pp. 649-660.