A practical guide to CNNs and Fisher Vectors for image instance retrieval

Signal Processing

Volumn 128, Issue , 2016, Pages 426-439

  Chandrasekhar, Vijay ^a   Lin, Jie ^a   Morère, Olivier ^a,b   Goh, Hanlin ^a   Veillard, Antoine ^b  

^a INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

^b SORBONNE UNIVERSITÉ   (France)

Author keywords

Convolutional neural networks; Fisher Vectors; Image instance retrieval

Indexed keywords

COMPUTER VISION; CONVOLUTION; IMAGE CLASSIFICATION; IMAGE PROCESSING; IMAGE RETRIEVAL; MATHEMATICAL TRANSFORMATIONS; NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORK; FISHER VECTORS; GEOMETRICAL TRANSFORMATION; GLOBAL DESCRIPTORS; IMAGE DESCRIPTORS; INSTANCE RETRIEVAL; MULTIPLE CRITERIA; PERFORMANCE GAIN;

SEARCH ENGINES;

EID: 84971592097     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2016.05.021     Document Type: Article

References (52)

1. L.-Y. Duan, R. Ji, Z. Chen, T. Huang, and W. Gao Towards mobile document image retrieval for digital library IEEE Trans. Multimed. 16 2 2014 346 359
2. D.M. Chen, and B. Girod A hybrid mobile visual search system with compact global signatures IEEE Trans. Multimed. 17 7 2015 1019 1030
3. R. Ji, L.-Y. Duan, J. Chen, L. Xie, H. Yao, and W. Gao Learning to distribute vocabulary indexing for scalable visual search IEEE Trans. Multimed. 15 1 2013 153 166
4. L.-Y. Duan, J. Lin, Z. Wang, T. Huang, and W. Gao Weighted component hashing of binary aggregated descriptors for fast visual search IEEE Trans. Multimed. 17 6 2015 828 842
5. Y.-G. Jiang, J. Wang, X. Xue, and S.-F. Chang Query-adaptive image search with hash codes IEEE Trans. Multimed. 15 2 2013 442 453
6. F. Perronnin, Y. Liu, J. Sánchez, H. Poirier, Large-scale image retrieval with compressed Fisher vectors, in: Computer Vision and Pattern Recognition (CVPR), IEEE, San Francisco, CA, 2010, pp. 3384-3391.
7. J. Sánchez, F. Perronnin, T. Mensink, and J. Verbeek Image classification with the fisher vector: theory and practice Int. J. Comput. Vis. 105 3 2013 222 245
8. D. Lowe Distinctive image features from scale-invariant keypoints Int. J. Comput. Vis. 60 2 2004 91 110
9. A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in: Neural Information Processing Systems (NIPS), 2012, pp. 1-9.
10. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in: International Conference on Learning Representations (ICLR), 2015.
11. Y. Taigman, M. Yang, M. Ranzato, L. Wolf, Deepface: closing the gap to human-level performance in face verification, in: Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1701-1708.
12. Y. Sun, X. Wang, X. Tang, Deep learning face representation from predicting 10,000 classes, in: Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1891-1898.
13. W. Ouyang, X. Wang, Joint deep learning for pedestrian detection, in: International Conference on Computer Vision (ICCV), 2013, pp. 2056-2063.
14. A. Toshev, C. Szegedy, Deeppose: human pose estimation via deep neural networks, in: Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1653-1660.
15. A. Babenko, A. Slesarev, A. Chigorin, V. Lempitsky, Neural Codes for Image Retrieval, in: European Conference on Computer Vision (ECCV), Springer, Cham, 2014, pp. 584-599.
16. V. Chandrasekhar, J. Lin, O. Morère, A. Veillard, H. Goh, Compact global descriptors for visual search, in: Data Compression Conference (DCC), IEEE, Snowbird, UT, 2015, pp. 333-342.
17. G. Tolias, Y. Avrithis, H. Jégou, To aggregate or not to aggregate: selective match kernels for image search, in: International Conference on Computer Vision (ICCV), 2013, pp. 1401-1408.
18. D. Chen, S. Tsai, V. Chandrasekhar, G. Takacs, R. Vedantham, R. Grzeszczuk, and B. Girod Residual enhanced visual vector as a compact signature for mobile visual search Signal Process. 93 8 2013 2316 2327
19. J. Lin, L.-Y. Duan, T. Huang, W. Gao, Robust Fisher codes for large scale image retrieval, in: International Conference on Acoustics and Signal Processing (ICASSP), 2013.
20. H. Jégou, A. Zisserman, Triangulation embedding and democratic aggregation for image search, in: Computer Vision and Pattern Recognition (CVPR), IEEE, Columbus, OH, 2014, pp. 3310-3317.
21. K. Mikolajczyk, Software for Computing Hessian-Affine Interest Points and SIFT Descriptor, 2010.
22. A. Razavian, H. Azizpour, J. Sullivan, S. Carlsson, Cnn features off-the-shelf: an astounding baseline for recognition, in: Computer Vision and Pattern Recognition Workshop (CVPR), 2014, pp. 806-813.
23. H. Jégou, M. Douze, C. Schmid, Hamming embedding and weak geometric consistency for large scale image search, in: A.Z. David Forsyth, Philip Torr (Eds.), European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, vol. I, Springer, Marseille, France, 2008, pp. 304-317.
24. D. Nister, H. Stewenius, Scalable recognition with a vocabulary tree, in: Computer Vision and Pattern Recognition (CVPR), vol. 2, IEEE, Washington, DC, 2006, pp. 2161-2168.
25. J. Philbin, O. Chum, M. Isard, J. Sivic, A. Zisserman, Object retrieval with large vocabularies and fast spatial matching, in: Computer Vision and Pattern Recognition (CVPR), IEEE, Minneapolis, MN, 2007, pp. 1-8.
26. V.R. Chandrasekhar, D.M. Chen, S.S. Tsai, N.-M. Cheung, H. Chen, G. Takacs, Y. Reznik, R. Vedantham, R. Grzeszczuk, J. Bach, et al., The Stanford mobile visual search data set, in: Multimedia Systems (MMSys), ACM, New York, NY, 2011, pp. 117-122.
27. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell, Caffe: convolutional architecture for fast feature embedding, in: Conference on Multimedia (ACMMM), ACM, New York, NY, 2014, pp. 675-678.
28. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1-9.
29. R. Wu, S. Yan, Y. Shan, Q. Dang, G. Sun, Deep image: scaling up image recognition, arXiv preprint arXiv:1501.02876.
30. T. Chilimbi, Y. Suzue, J. Apacible, K. Kalyanaraman, Project adam: building an efficient and scalable deep learning training system, in: Operating Systems Design and Implementation (OSDI), 2014, pp. 571-582.
31. Researcher, MPEG. Compact Descriptors for Visual Search (CDVS) Benchmark. Stanford Digital Repository, 2014
32. A. Vedaldi, B. Fulkerson, Vlfeat: an open and portable library of computer vision algorithms, in: Conference on Multimedia (ACMMM), ACM, Firenze, Italy, 2010, pp. 1469-1472.
33. G. Francini, S. Lepsøy, and M. Balestri Selection of local features for visual search Signal Process.: Image Commun. 28 4 2013 311 322
34. H. Jégou, F. Perronnin, M. Douze, J. Sanchez, P. Perez, and C. Schmid Aggregating local image descriptors into compact codes IEEE Pattern Anal. Mach. Intell. 34 9 2012 1704 1716
35. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva Learning deep features for scene recognition using places database Neural Information Processing Systems (NIPS) 2014 Montreal Canada 487 495
36. J. Xiao, J. Hays, K.A. Ehinger, A. Oliva, A. Torralba, Sun database: Large-scale scene recognition from abbey to zoo, in: Computer Vision and Pattern Recognition (CVPR), IEEE, San Francisco, CA, 2010, pp. 3485-3492.
37. A. Quattoni, A. Torralba, Recognizing indoor scenes, in: Computer Vision and Pattern Recognition (CVPR), IEEE, Miami, FL, 2009, pp. 413-420.
38. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: a large-scale hierarchical image database, in: Computer Vision and Pattern Recognition (CVPR), IEEE, Miami, FL, 2009, pp. 248-255.
39. F. Yang, Z. Jiang, L.S. Davis, Submodular reranking with multiple feature modalities for image retrieval, in: Asian Conference Computer Vision (ACCV), Springer, Singapore, 2014, pp. 19-34.
40. G. Takacs, V. Chandrasekhar, S. Tsai, D. Chen, R. Grzeszczuk, B. Girod, Unified real-time tracking and recognition with rotation-invariant fast features, in: Computer Vision and Pattern Recognition (CVPR), IEEE, San Francisco, CA, 2010, pp. 934-941.
41. B. Thomee, D.A. Shamma, G. Friedland, B. Elizalde, K. Ni, D. Poland, D. Borth, L. Li, YFCC100M: the new data in multimedia research», Communications of the ACM, 59(2), pp. 64-73, 2016.
42. R. Collobert, K. Kavukcuoglu, C. Farabet, Torch7: a Matlab-like environment for machine learning, in: Neural Information Processing Systems Workshop (NIPS), 2011.
43. S. Winder, G. Hua, M. Brown, Picking the best daisy, in: Computer Vision and Pattern Recognition (CVPR), IEEE, Miami, FL, 2009, pp. 178-185.
44. CDVS Patches, (http://blackhole1.stanford.edu/vijayc/cdvs-patches.tar), 2013.
45. T. Tuytelaars, Dense interest points, in: Computer Vision and Pattern Recognition (CVPR), IEEE, 2010, pp. 2281-2288.
46. A. Iscen, G. Tolias, P.-H. Gosselin, and H. Jégou A comparison of dense region detectors for image search and fine-grained classification IEEE Trans. Image Process. 24 8 2015 2369 2381
47. Y. Gong, L. Wang, R. Guo, S. Lazebnik, Multi-scale orderless pooling of deep convolutional activation features, in: European Conference on Computer Vision (ECCV), Springer, Zurich, Switzerland, 2014, pp. 392-407.
48. Z. Xu, Y. Yang, A.G. Hauptmann, A discriminative CNN video representation for event detection, in: Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1798-1807.
49. H. Jégou, M. Douze, C. Schmid, P. Pérez, Aggregating local descriptors into a compact image representation, in: Computer Vision and Pattern Recognition (CVPR), IEEE, San Francisco, CA, 2010, pp. 3304-3311.
50. A. Gordoa, J.A. Rodríguez-Serrano, F. Perronnin, E. Valveny, Leveraging category-level labels for instance-level image retrieval, in: Computer Vision and Pattern Recognition (CVPR), IEEE, Providence, RI, 2012, pp. 3045-3052.
51. R. Arandjelovic, A. Zisserman, All about VLAD, in: Computer Vision and Pattern Recognition (CVPR), 2013, pp. 1578-1585.
52. T. Ge, Q. Ke, J. Sun, Sparse-coded features for image retrieval, in: British Machine Vision Conference (BMVC), 2013.