Hierarchical Recurrent Neural Hashing for Image Retrieval with Hierarchical Convolutional Features

IEEE Transactions on Image Processing

Volumn 27, Issue 1, 2018, Pages 106-120

  Lu, Xiaoqiang ^a   Chen, Yaxiong ^a   Li, Xuelong ^a  

^a XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

Author keywords

hierarchical convolutional features; hierarchical RNN; Image retrieval; supervised hashing

Indexed keywords

BINARY CODES; BINS; CODES (SYMBOLS); COMPUTATIONAL EFFICIENCY; CONVOLUTION; CONVOLUTIONAL CODES; DEEP LEARNING; FEATURE EXTRACTION; HASH FUNCTIONS; RECURRENT NEURAL NETWORKS; SEMANTICS;

HIERARCHICAL CONVOLUTIONAL FEATURES; HIERARCHICAL RNN; IMAGE REPRESENTATIONS; LEARNING ARCHITECTURES; QUANTIZATION ERRORS; SEMANTIC INFORMATION; SUPERVISED HASHING; TRAINING DATA;

IMAGE RETRIEVAL;

ARTICLE; DEEP LEARNING; EMBEDDING; FEATURE EXTRACTION; IMAGE RETRIEVAL; LOSS OF FUNCTION MUTATION; RECURRENT NEURAL NETWORK; SEMANTICS; VELOCITY;

EID: 85030698243     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2017.2755766     Document Type: Article

References (72)

1. X. Lu, X. Zheng, and X. Li, "Latent semantic minimal hashing for image retrieval, " IEEE Trans. Image Process., vol. 26, no. 1, pp. 355-368, Jan. 2017.
2. A. Gionis, P. Indyk, and R. Motwani, "Similarity search in high dimensions via hashing, " in Proc. VLDB, 1999, pp. 518-529.
3. Y. Gong and S. Lazebnik, "Iterative quantization: A procrustean approach to learning binary codes, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 817-824.
4. X. Nie, Y. Chai, J. Liu, J. Sun, and Y. Yin, "Spherical torus-based video hashing for near-duplicate video detection, " Sci. China Inf. Sci., vol. 59, p. 059101, May 2016.
5. P. Jain, B. Kulis, and K. Grauman, "Fast image search for learned metrics, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
6. M. Raginsky and S. Lazebnik, "Locality-sensitive binary codes from shift-invariant kernels, " in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 1509-1517.
7. Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing, " in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 1753-1760.
8. F. Zhao, Y. Huang, L. Wang, and T. Tan, "Deep semantic ranking based hashing for multi-label image retrieval, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1556-1564.
9. R. Zhang, L. Lin, R. Zhang, W. Zuo, and L. Zhang, "Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification, " IEEE Trans. Image Process., vol. 24, no. 12, pp. 4766-4779, Dec. 2015.
10. G. Shakhnarovich, "Learning task-specific similarity, " Ph. D. dissertation, Dept. Elect. Eng. Comput. Sci., Massachusetts Inst. Technol., Cambridge, MA, USA, 2005.
11. M. Norouzi, D. J. Fleet, and R. R. Salakhutdinov, "Hamming distance metric learning, " in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1061-1069.
12. J. Wang, W. Liu, A. X. Sun, and Y.-G. Jiang, "Learning hash codes with listwise supervision, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 3032-3039.
13. B. Kulis and T. Darrell, "Learning to hash with binary reconstructive embeddings, " in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 1042-1050.
14. W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang, "Supervised hashing with kernels, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 2074-2081.
15. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope, " Int. J. Comput. Vis., vol. 42, no. 3, pp. 145-175, 2001.
16. D. G. Lowe, "Object recognition from local scale-invariant features, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Sep. 1999, p. 1150.
17. R. Salakhutdinov and G. Hinton, "Semantic hashing, " Int. J. Approx. Reasoning, vol. 50, no. 7, pp. 969-978, Jul. 2009.
18. A. Torralba, R. Fergus, and Y. Weiss, "Small codes and large image databases for recognition, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
19. A. Babenko and V. Lempitsky, "Aggregating local deep features for image retrieval, " in Proc. ICCV, 2015, pp. 1269-1277.
20. R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, "Supervised hashing for image retrieval via image representation learning, " in Proc. AAAI Conf. Artif. Intell., 2014, pp. 2156-2162.
21. L. Zheng, Y. Zhao, S. Wang, J. Wang, and Q. Tian. (2016). "Good practice in CNN feature transfer. " [Online]. Available: https://arxiv. org/abs/1604. 00133
22. G. Tolias, R. Sicre, and H. Jégou. (2016). "Particular object retrieval with integral max-pooling of CNN activations. " [Online]. Available: https:// arxiv. org/abs/1511. 05879
23. P. Pan, Z. Xu, Y. Yang, F. Wu, and Y. Zhuang, "Hierarchical recurrent neural encoder for video representation with application to captioning, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 1029-1038.
24. J. Zhang and Y. Peng. (2016). "SSDH: Semi-supervised deep hashing for large scale image retrieval. " [Online]. Available: https:// arxiv. org/abs/1607. 08477
25. C. Ma, J.-B. Huang, X. Yang, and M.-H. Yang, "Hierarchical convolutional features for visual tracking, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 3074-3082.
26. H. Yu, J. Wang, Z. Huang, Y. Yang, and W. Xu, "Video paragraph captioning using hierarchical recurrent neural networks, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 4584-4593.
27. I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks, " in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 3104-3112.
28. A. Sordoni, Y. Bengio, H. Vahabi, C. Lioma, J. G. Simonsen, and J.-Y. Nie, "A hierarchical recurrent encoder-decoder for generative context-aware query suggestion, " in Proc. Int. Conf. Inf. Knowl. Manage., 2015, pp. 553-562.
29. D. Zhang, J. Han, C. Li, J. Wang, and X. Li, "Detection of co-salient objects by looking deep and wide, " Int. J. Comput. Vis., vol. 120, no. 2, pp. 215-232, 2016.
30. D. Zhang, J. Han, J. Han, and L. Shao, "Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining, " IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp. 1163-1176, Jun. 2016.
31. A. S. Razavian, J. Sullivan, A. Maki, and S. Carlsson, "Visual instance retrieval with deep convolutional networks, " ITE Trans. Media Technol. Appl., vol. 4, no. 3, pp. 251-258, 2016.
32. Y. Kalantidis, C. Mellina, and S. Osindero, "Cross-dimensional weighting for aggregated deep convolutional features, " in Proc. Eur. Conf. Comput. Vis., 2015, pp. 685-701.
33. J. Wang, T. Zhang, J. Song, N. Sebe, and H. T. Shen, "A survey on learning to hash, " IEEE Trans. Pattern Anal. Mach. Intell., to be published.
34. L. Zheng, S. Wang, J. Wang, and Q. Tian, "Accurate image search with multi-scale contextual evidences, " Int. J. Comput. Vis., vol. 120, no. 1, pp. 1-13, 2016.
35. L. Zheng, Y. Yang, and Q. Tian, "SIFT meets CNN: A decade survey of instance retrieval, " IEEE Trans. Pattern Anal. Mach. Intell., to be published.
36. B. Kulis and K. Grauman, "Kernelized locality-sensitive hashing, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 6, pp. 1092-1104, Jun. 2012.
37. Z. Zhang, Y. Chen, and V. Saligrama. (2016). "Efficient training of very deep neural networks for supervised hashing. " [Online]. Available: https://arxiv. org/abs/1511. 04524
38. H. Liu, R. Wang, S. Shan, and X. Chen, "Deep supervised hashing for fast image retrieval, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 2064-2072.
39. G. F. Montúfar, R. Pascanu, K. Cho, and Y. Bengio, "On the number of linear regions of deep neural networks, " in Proc. Int. Conf. Neural Inf. Process. Syst., 2014, pp. 2924-2932.
40. G. F. Montúfar, R. Pascanu, K. Cho, and Y. Bengio, "On the number of linear regions of deep neural networks, " in Proc. Adv. Neural Inf. Process. Syst., 2014, vol. 39. no. 1, pp. 122-123.
41. Y. Kim, Y. Jernite, D. Sontag, and A. M. Rush. (2015). "Characteraware neural language models. " [Online]. Available: https://arxiv. org/ abs/1508. 06615
42. Z. Xia, X. Feng, J. Peng, and A. Hadid. (2016). "Unsupervised deep hashing for large-scale visual search. " [Online]. Available: https://arxiv. org/abs/1602. 00206
43. V. E. Liong, J. Lu, G. Wang, P. Moulin, and J. Zhou, "Deep hashing for compact binary codes learning, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 2475-2483.
44. T.-T. Do, A.-Z. Doan, and N.-M. Cheung. (2015). "Discrete hashing with deep neural network. " [Online]. Available: https://arxiv. org/ abs/1508. 07148
45. T.-T. Do, A.-D. Doan, and N.-M. Cheung, "Learning to hash with binary deep neural network, " in Proc. ECCV, 2016, pp. 219-234.
46. W. J. Li, S. Wang, and W. C. Kang, "Feature learning based deep supervised hashing with pairwise labels, " in Proc. Int. Joint Conf. Artif. Intell., 2016, pp. 1711-1717.
47. H. Lai, Y. Pan, Y. Liu, and S. Yan, "Simultaneous feature learning and hash coding with deep neural networks, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 3270-3278.
48. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks, " in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
49. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, " in Proc. Int. Conf. Learn. Represent., 2014, pp. 1-14.
50. P. Weinzaepfel, J. Revaud, Z. Harchaoui, and C. Schmid, "DeepFlow: Large displacement optical flow with deep matching, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 1385-1392.
51. A. Graves, Supervised Sequence Labelling With Recurrent Neural Networks, vol. 385. Heidelberg, Germany: Springer, 2012.
52. W.-C. Kang, W.-J. Li, and Z.-H. Zhou, "Column sampling based discrete supervised hashing, " in Proc. AAAI, 2016, pp. 1230-1236.
53. H. Zhu, M. Long, J. Wang, and Y. Cao, "Deep hashing network for efficient similarity retrieval, " in Proc. AAAI, 2016, pp. 2415-2421.
54. H.-F. Yang, K. Lin, and C.-S. Chen. (2015). "Supervised learning of semantics-preserving hash via deep convolutional neural networks. " Available: https://arxiv. org/abs/1507. 00101
55. A. Hyvärinen, J. Hurri, and P. O. Hoyer, Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, vol. 39. Berlin, Germany: Springer, 2009.
56. T. Tieleman and G. E. Hinton, "Neural networks for machine learning, " Coursera, Mountain View, CA, USA, Tech. Rep., 2012.
57. Y. Lecun and C. Cortes. (1998). The MNIST Database of Handwritten Digits. [online] Available: http://vann. lecun. com/exdb/mnist/
58. G. Irie, Z. Li, X.-M. Wu, and S.-F. Chang, "Locally linear hashing for extracting non-linear manifolds, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 2123-2130.
59. A. Krizhevsky, "Learning multiple layers of features from tiny images, " Dept. Comput. Sci., Univ. Toronto, Toronto, ON, Canada, Tech. Rep., 2009.
60. L. Wolf, T. Hassner, and I. Maoz, "Face recognition in unconstrained videos with matched background similarity, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 529-534.
61. D. Song, W. Liu, R. Ji, D. A. Meyer, and J. R. Smith, "Top rank supervised binary coding for visual search, " in Proc. Int. Conf. Comput. Vis., 2015, pp. 1922-1930.
62. J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, "SUN database: Large-scale scene recognition from abbey to zoo, " in Proc. Comput. Vis. Pattern Recognit., 2010, pp. 3485-3492.
63. K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. (2014). "Return of the devil in the details: Delving deep into convolutional nets. " Available: https://arxiv. org/abs/1405. 3531
64. J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 248-255.
65. A. Vedaldi and K. Lenc, "MatConvNet: Convolutional neural networks for MATLAB, " in Proc. ACM Int. Conf. Multimedia, 2015, pp. 689-692.
66. Q. V. Le, N. Jaitly, and G. E. Hinton. (2015). "A simple way to initialize recurrent networks of rectified linear units. " [Online]. Available: https://arxiv. org/abs/1504. 00941
67. A. Graves. (2013). "Generating sequences with recurrent neural networks. " Available: https://arxiv. org/abs/1308. 0850
68. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. (2017). "Empirical evaluation of gated recurrent neural networks on sequence modeling. " Available: https://arxiv. org/abs/1412. 3555
69. K. Greff, R. K. Srivastava, J. Koutnìk, B. R. Steunebrink, and J. Schmidhuber, "LSTM: A search space odyssey, " IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 10, pp. 2222-2232, Oct. 2017.
70. M. Norouzi and D. J. Fleet, "Minimal loss hashing for compact binary codes, " in Proc. Int. Conf. Mach. Learn., 2011, pp. 353-360.
71. W. Kong and W.-J. Li, "Isotropic hashing, " in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1655-1663.
72. X. Li, G. Lin, C. Shen, A. van den Hengel, and A. Dick, "Learning hash functions using column generation, " in Proc. ICML, 2013, pp. 142-150.