Deep Hashing for Scalable Image Search

IEEE Transactions on Image Processing

Volumn 26, Issue 5, 2017, Pages 2352-2367

  Lu, Jiwen ^a   Liong, Venice Erin ^b   Zhou, Jie ^a  

^a TSINGHUA UNIVERSITY   (China)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

deep learning; fast similarity search; hashing; multi label learning; Scalable image search

Indexed keywords

BINARY CODES; CODES (SYMBOLS); DEEP LEARNING; DEEP NEURAL NETWORKS; IMAGE RETRIEVAL; LEARNING SYSTEMS; LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS;

BINARY FEATURE VECTOR; HASHING; INTRA-CLASS VARIATION; MULTI-LABEL LEARNING; NON-LINEAR RELATIONSHIPS; NON-LINEAR TRANSFORMATIONS; SCALABLE IMAGE; SIMILARITY SEARCH;

BINS;

ARTICLE; DEEP NEURAL NETWORK; LEARNING;

EID: 85018468901     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2017.2678163     Document Type: Article

References (86)

1. A. Andoni and P. Indyk, "Near-optimal hashing algorithms for approximatenearest neighbor in high dimensions, " in Proc. FOCS, 2006, pp. 459-468.
2. Y. Bengio, "Learning deep architectures for AI, " Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009.
3. J. L. Bentley, "Multidimensional binary search trees used for associativesearching, " Commun. ACM, vol. 18, no. 9, pp. 509-517, 1975.
4. T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo., Y.-T. Zheng, "NUS-WIDE: A real-world Web image database from NationalUniversity of Singapore, " in Proc. ACM ICIVR, 2009, p. 48.
5. O. Chum, J. Philbin., A. Zisserman, "Near duplicate image detection:Min-hash and TF-IDF weighting, " in Proc. BMVC, 2008, pp. 812-815.
6. M. Datar, N. Immorlica, P. Indyk., V. S. Mirrokni, "Locality-sensitivehashing scheme based on p-stable distributions, " in Proc. ACM Comput. Geometry, 2004, pp. 253-262.
7. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li., L. Fei-Fei, "ImageNet:A large-scale hierarchical image database, " in Proc. CVPR, Jun. 2009, pp. 248-255.
8. M. Douze, H. Jégou., F. Perronnin, "Polysemous codes, " in Proc. ECCV, 2016, pp. 785-801.
9. W. Feng, B. Jia., M. Zhu, "Deep hash: Semantic similarity preservedhash scheme, " Electron. Lett., vol. 50, no. 19, pp. 1347-1349, Sep. 2014.
10. R. Girshick, J. Donahue, T. Darrell., J. Malik, "Rich featurehierarchies for accurate object detection and semantic segmentation, "in Proc. CVPR, 2014, pp. 580-587.
11. X. Glorot and Y. Bengio, "Understanding the difficulty of training deepfeedforward neural networks, " in Proc. AISTATS, 2011, pp. 249-256.
12. Y. Gong, S. Kumar, V. Verma., S. Lazebnik, "Angular quantizationbasedbinary codes for fast similarity search, " in Proc. NIPS, 2012, pp. 1196-1204.
13. Y. Gong and S. Lazebnik, "Iterative quantization: A procrusteanapproach to learning binary codes, " in Proc. CVPR, Jun. 2011, pp. 817-824.
14. J. He, W. Liu., S.-F. Chang, "Scalable similarity search withoptimized kernel hashing, " in Proc. KDD, 2010, pp. 1129-1138.
15. K. He, F. Wen., J. Sun, "K-means hashing: An affinity-preservingquantization method for learning binary compact codes, " in Proc. CVPR, Jun. 2013, pp. 2938-2945.
16. J.-P. Heo, Y. Lee, J. He, S.-F. Chang., S.-E. Yoon, "Sphericalhashing, " in Proc. CVPR, Jun. 2012, pp. 2957-2964.
17. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality ofdata with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, 2006.
18. H. Hu, "Orthogonal neighborhood preserving discriminant analysis forface recognition, " Pattern Recognit., vol. 41, no. 6, pp. 2045-2054, Jun. 2008.
19. M. J. Huiskes and M. S. Lew, "The MIR Flickr retrieval evaluation, " inProc. ACM MIR, 2008, pp. 39-43.
20. P. Indyk and R. Motwani, "Approximate nearest neighbors: Towardsremoving the curse of dimensionality, " in Proc. ACM Symp. TheoryComput., 1998, pp. 604-613.
21. G. Irie, Z. Li, X.-M. Wu., S.-F. Chang, "Locally linear hashingfor extracting non-linear manifolds, " in Proc. CVPR, Jun. 2014, pp. 2123-2130.
22. P. Jain, B. Kulis., K. Grauman, "Fast image search for learnedmetrics, " in Proc. CVPR, Jun. 2008, pp. 1-8.
23. K. Järvelin and J. Kekälaïnen, "Ir evaluation methods for retrievinghighly relevant documents, " in Proc. ACM SIGIR, 2000, pp. 41-48.
24. H. Jegou, M. Douze., C. Schmid, "Hamming embedding and weakgeometric consistency for large scale image search, " in Proc. ECCV, 2008, pp. 304-317.
25. H. Jegou, M. Douze., C. Schmid, "Product quantization for nearestneighbor search, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 117-128, Jan. 2011.
26. H. Jégou, F. Perronnin, M. Douze, J. Sánchez, P. Pérez., C. Schmid, "Aggregating local image descriptors into compact codes, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 9, pp. 1704-1716, Sep. 2012.
27. J. Ji, J. Li, S. Yan, B. Zhang., Q. Tian, "Super-bit locality-sensitivehashing, " in Proc. NIPS, 2012, pp. 108-116.
28. S. Ji, W. Xu, M. Yang., K. Yu, "3D convolutional neural networksfor human action recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 221-231, Jan. 2013.
29. A. Krizhevsky and G. Hinton, "Learning multiple layers of features fromtiny images, " Univ. Toronto, Toronto, ON, Canada, Tech. Rep. 1, 2009.
30. A. Krizhevsky, I. Sutskever., G. E. Hinton, "ImageNet classificationwith deep convolutional neural networks, " in Proc. NIPS, 2012, pp. 1097-1105.
31. B. Kulis and K. Grauman, "Kernelized locality-sensitive hashing forscalable image search, " in Proc. ICCV, 2009, pp. 2130-2137.
32. B. Kulis, P. Jain., K. Grauman, "Fast similarity search for learnedmetrics, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 12, pp. 2143-2157, Dec. 2009.
33. H. Lai, Y. Pan, Y. Liu., S. Yan, "Simultaneous feature learningand hash coding with deep neural networks, " in Proc. CVPR, 2015, pp. 3270-3278.
34. Q. V. Le, W. Y. Zou, S. Y. Yeung., A. Y. Ng, "Learning hierarchicalinvariant spatio-temporal features for action recognition with independentsubspace analysis, " in Proc. CVPR, Jun. 2011, pp. 3361-3368.
35. Y. LeCun and C. Cortes, "The MNIST database of handwritten digits, "New York Univ., New York, NY, USA, Tech. Rep. 1, 1998.
36. H. Lee, R. Grosse, R. Ranganath., A. Y. Ng, "Convolutionaldeep belief networks for scalable unsupervised learning of hierarchicalrepresentations, " in Proc. ICML, 2009, pp. 609-616.
37. G. Lin, C. Shen, Q. Shi, A. van den Hengel., D. Suter, "Fastsupervised hashing with decision trees for high-dimensional data, " inProc. CVPR, 2014, pp. 1963-1970.
38. K. Lin, H.-F. Yang, J.-H. Hsiao., C.-S. Chen, "Deep learning ofbinary hash codes for fast image retrieval, " in Proc. CVPRW, Jun. 2015, pp. 27-35.
39. H. Liu, R. Wang, S. Shan., X. Chen, "Deep supervised hashing forfast image retrieval, " in Proc. CVPR, Jun. 2015, pp. 2064-2072.
40. W. Liu, J. Wang, R. Ji, Y.-G. Jiang., S.-F. Chang, "Supervisedhashing with kernels, " in Proc. CVPR, Jun. 2012, pp. 2074-2081.
41. W. Liu, J. Wang, S. Kumar., S.-F. Chang, "Hashing with graphs, " inProc. ICML, 2011, pp. 1-8.
42. X. Liu, J. He, B. Lang., S.-F. Chang, "Hash bit selection: A unifiedsolution for selection problems in hashing, " in Proc. CVPR, Jun. 2013, pp. 1570-1577.
43. Y. Liu, J. Shao, J. Xiao, F. Wu., Y. Zhuang, "Hypergraph SpectralHashing for image retrieval with heterogeneous social contexts, " Neurocomputing, vol. 119, pp. 49-58, Nov. 2013.
44. J. Lu, V. E. Liong, X. Zhou., J. Zhou, "Learning compact binaryface descriptor for face recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 10, pp. 2041-2056, Oct. 2015.
45. J. Masci, M. M. Bronstein, A. M. Bronstein., J. Schmidhuber, "Multimodal similarity-preserving hashing, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 4, pp. 824-830, Apr. 2014.
46. J. Masci, M. M. Bronstein, A. M. Bronstein., J. Schmidhuber, "Fastapproximate nearest neighbors with automatic algorithm configuration, "VISAPP, vol. 2, no. 1, pp. 331-340, 2009.
47. M. Muja and D. G. Lowe, "Scalable nearest neighbor algorithms forhigh dimensional data, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 11, pp. 2227-2240, Nov. 2014.
48. M. Norouzi and D. M. Blei, "Minimal loss hashing for compact binarycodes, " in Proc. ICML, 2011, pp. 353-360.
49. M. Norouzi and D. J. Fleet, "Cartesian K-means, " in Proc. CVPR, Jun. 2013, pp. 3017-3024.
50. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holisticrepresentation of the spatial envelope, " Int. J. Comput. Vis., vol. 42, no. 3, pp. 145-175, 2001.
51. M. Raginsky and S. Lazebnik, "Locality-sensitive binary codes fromshift-invariant kernels, " in Proc. NIPS, 2009, pp. 1509-1517.
52. A. Rahimi and B. Recht, "Random features for large-scale kernelmachines, " in Proc. NIPS, 2007, pp. 1177-1184.
53. M. Ranzato, F. J. Huang, Y.-L. Boureau., Y. LeCun, "Unsupervisedlearning of invariant feature hierarchies with applications to objectrecognition, " in Proc. CVPR, Jun. 2007, pp. 1-8.
54. M. Rastegari, A. Farhadi., D. Forsyth, "Attribute discoveryvia predictable discriminative binary codes, " in Proc. ECCV, 2012, pp. 876-889.
55. A. Sablayrolles, M. Douze, H. Jégou., N. Usunier. (2016). "How should we evaluate supervised hashing?" [Online]. Available:https://arxiv. org/abs/1609. 06753
56. R. Salakhutdinov and G. Hinton, "Semantic hashing, " Int. J. Approx. Reasoning, vol. 50, no. 7, pp. 969-978, Jul. 2009.
57. G. Shakhnarovich, T. Darrell., P. Indyk, "Nearest-neighbor methodsin learning and vision, " IEEE Trans. Neural Netw., vol. 19, no. 2, p. 377, Feb. 2008.
58. J. Shao, F. Wu, C. Ouyang., X. Zhang, "Sparse spectral hashing, "Pattern Recognit. Lett., vol. 33, no. 3, pp. 271-277, Feb. 2012.
59. F. Shen, C. Shen, W. Liu., H. T. Shen, "Supervised discrete hashing, "in Proc. CVPR, Jun. 2015, pp. 37-45.
60. F. Shen, C. Shen, Q. Shi, A. van den Hengel., Z. Tang, "Inductivehashing on manifolds, " in Proc. CVPR, 2013, pp. 1562-1569.
61. C. Silpa-Anan and R. Hartley, "Optimised KD-trees for fast imagedescriptor matching, " in Proc. CVPR, Jun. 2008, pp. 1-8.
62. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever., R. Salakhutdinov, "Dropout: A simple way to prevent neural networksfrom overfitting, " J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929-1958, 2014.
63. N. Srivastava and R. R. Salakhutdinov, "Multimodal learning with deepBoltzmann machines, " in Proc. NIPS, 2012, pp. 2222-2230.
64. C. Strecha, A. M. Bronstein, M. M. Bronstein., P. Fua, "LDAHash:Improved matching with smaller descriptors, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 66-78, Jan. 2012.
65. C. Szegedy, A. Toshev., D. Erhan, "Deep neural networks for objectdetection, " in Proc. NIPS, 2013, pp. 2553-2561.
66. Y. Taigman, M. Yang, M. Ranzato., L. Wolf, "DeepFace: Closing thegap to human-level performance in face verification, " in Proc. CVPR, Jun. 2014, pp. 1701-1708.
67. G. W. Taylor, R. Fergus, Y. LeCun., C. Bregler, "Convolutionallearning of spatio-temporal features, " in Proc. ECCV, 2010, pp. 140-153.
68. A. Torralba, R. Fergus., W. T. Freeman, "80 million tiny images:A large data set for nonparametric object and scene recognition, " IEEETrans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1958-1970, Nov. 2008.
69. A. Torralba, R. Fergus., Y. Weiss, "Small codes and large imagedatabases for recognition, " in Proc. CVPR, Jun. 2008, pp. 1-8.
70. A. Vedaldi., K. Lenc, "Matconvnet: Convolutional neural networksfor MATLAB, " in Proc. ACM MM, Oct. 2015, pp. 689-692.
71. D. Wang, X. Gao, X. Wang., L. He, "Semantic topic multimodalhashing for cross-media retrieval, " in Proc. IJCAI, 2015, pp. 3890-3896.
72. D. Wang, X. Gao, X. Wang, L. He., B. Yuan, "Multimodal discriminativebinary embedding for large-scale cross-modal retrieval, " IEEETrans. Image Process., vol. 25, no. 10, pp. 4540-4554, Oct. 2016.
73. H. Wang, C. Ding., H. Huang, "Multi-label linear discriminantanalysis, " in Proc. ECCV, 2010, pp. 126-139.
74. J. Wang, S. Kumar., S.-F. Chang, "Semi-supervised hashing for largescalesearch, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 12, pp. 2393-2406, Dec. 2012.
75. J. Wang, W. Liu, A. X. Sun., Y.-G. Jiang, "Learning hash codes withlistwise supervision, " in Proc. CVPR, 2013, pp. 3032-3039.
76. J. Wang, H. T. Shen, J. Song., J. Ji. (2014). "Hashing for similaritysearch: A survey. " [Online]. Available: Https://arxiv. org/abs/1408. 2927
77. J. Wang et al., "Learning fine-grained image similarity with deepranking, " in Proc. CVPR, 2014, pp. 1386-1393.
78. J. Wang, J. Wang, N. Yu., S. Li, "Order preserving hashingfor approximate nearest neighbor search, " in Proc. ACM MM, 2013, pp. 133-142.
79. N. Wang and D.-Y. Yeung, "Learning a deep compact image representationfor visual tracking, " in Proc. NIPS, 2013, pp. 809-817.
80. X. Wang, T. Zhang, G.-J. Qi, J. Tang., J. Wang, "Supervisedquantization for similarity search, " in Proc. CVPR, Jun. 2016, pp. 2018-2026.
81. Y. Weiss, A. Torralba., R. Fergus, "Spectral hashing, " in Proc. NIPS, 2008, pp. 1753-1760.
82. R. Xia, Y. Pan, H. Lai, C. Liu., S. Yan, "Supervised hashing forimage retrieval via image representation learning, " in Proc. AAAI, 2014, pp. 2156-2162.
83. J. Xiao, J. Hays, K. A. Ehinger, A. Oliva., A. Torralba, "SUNdatabase: Large-scale scene recognition from abbey to zoo, " in Proc. CVPR, Jun. 2010, pp. 3485-3492.
84. T. Yao, F. Long, T. Mei., Y. Rui, "Deep semantic-preservingand ranking-based hashing for image retrieval, " in Proc. IJCAI, 2016, pp. 3931-3937.
85. T. Zhang, C. Du., J. Wang, "Composite quantization for approximatenearest neighbor search, " in Proc. ICML, 2014, pp. 838-846.
86. F. Zhao, Y. Huang, L. Wang., T. Tan, "Deep semantic rankingbased hashing for multi-label image retrieval, " in Proc. CVPR, 2015, pp. 1556-1564.