NetVLAD: CNN architecture for weakly supervised place recognition

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 5297-5307

  Arandjelovic, Relja ^a   Gronat, Petr ^a   Torii, Akihiko ^b   Pajdla, Tomas ^c   Sivic, Josef ^a  

^a INRIA   (France)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^c CZECH TECHNICAL UNIVERSITY IN PRAGUE   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; IMAGE RETRIEVAL; NETWORK ARCHITECTURE; NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORK; IMAGE REPRESENTATIONS; OVER CURRENT; PLACE RECOGNITION; PROPOSED ARCHITECTURES; STATE OF THE ART; TRAINING PROCEDURES; VECTOR OF LOCALLY AGGREGATED DESCRIPTORS;

PATTERN RECOGNITION;

EID: 84986296991     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.572     Document Type: Conference Paper

References (89)

1. Project webpage (code/networks). http://www.di. ens. fr/willow/research/netvlad/.
2. Supplementary material (appendix) for the paper. http://arxiv. org/abs/1511. 07247.
3. R. Arandjelovíc and A. Zisserman. Three things everyone should know to improve object retrieval. In Proc. CVPR, 2012.
4. R. Arandjelovíc and A. Zisserman. All about VLAD. In Proc. CVPR, 2013.
5. R. Arandjelovíc and A. Zisserman. DisLocation: Scalable descriptor distinctiveness for location recognition. In Proc. ACCV, 2014.
6. M. Aubry, B. C. Russell, and J. Sivic. Painting-to-3D model alignment via discriminative visual elements. ACM Transactions on Graphics (TOG), 33 (2): 14, 2014.
7. H. Azizpour, A. Razavian, J. Sullivan, A. Maki, and S. Carlsson. Factors of transferability from a generic ConvNet representation. CoRR, abs/1406. 5774, 2014.
8. A. Babenko and V. Lempitsky. Aggregating local deep features for image retrieval. In Proc. ICCV, 2015.
9. A. Babenko, A. Slesarev, A. Chigorin, and V. Lempitsky. Neural codes for image retrieval. In Proc. ECCV, 2014.
10. S. Cao and N. Snavely. Graph-based discriminative learning for location recognition. In Proc. CVPR, 2013.
11. D. M. Chen, G. Baatz, K. Koeser, S. S. Tsai, R. Vedantham, T. Pylvanainen, K. Roimela, X. Chen, J. Bach, M. Pollefeys, B. Girod, and R. Grzeszczuk. City-scale landmark identification on mobile devices. In Proc. CVPR, 2011.
12. O. Chum, A. Mikulik, M. Perdoch, and J. Matas. Total recall II: Query expansion revisited. In Proc. CVPR, 2011.
13. O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman. Total recall: Automatic query expansion with a generative feature model for object retrieval. In Proc. ICCV, 2007.
14. M. Cimpoi, S. Maji, and A. Vedaldi. Deep filter banks for texture recognition and segmentation. In Proc. CVPR, 2015.
15. G. Csurka, C. Bray, C. Dance, and L. Fan. Visual categorization with bags of keypoints. In Workshop on Statistical Learning in Computer Vision, ECCV, pages 1-22, 2004.
16. M. Cummins and P. Newman. FAB-MAP: Probabilistic localization and mapping in the space of appearance. The International Journal of Robotics Research, 2008.
17. M. Cummins and P. Newman. Highly scalable appearanceonly SLAM-FAB-MAP 2. 0. In RSS, 2009.
18. J. Delhumeau, P.-H. Gosselin, H. Jégou, and P. Pérez. Revisiting the VLAD image representation. In Proc. ACMM, 2013.
19. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proc. CVPR, 2009.
20. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. DeCAF: A deep convolutional activation feature for generic visual recognition. CoRR, abs/1310. 1531, 2013.
21. J. Foulds and E. Frank. A review of multi-instance learning assumptions. The Knowledge Engineering Review, 25 (01): 1-25, 2010.
22. R. B. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proc. CVPR, 2014.
23. Y. Gong, L. Wang, R. Guo, and S. Lazebnik. Multi-scale orderless pooling of deep convolutional activation features. In Proc. ECCV, 2014.
24. P. Gronat, G. Obozinski, J. Sivic, and T. Pajdla. Learning and calibrating per-location classifiers for visual place recognition. In Proc. CVPR, 2013.
25. H. Jégou and O. Chum. Negative evidences and cooccurrences in image retrieval: The benefit of PCA and whitening. In Proc. ECCV, 2012.
26. H. Jégou, M. Douze, and C. Schmid. Hamming embedding and weak geometric consistency for large scale image search. In Proc. ECCV, pages 304-317, 2008.
27. H. Jégou, M. Douze, and C. Schmid. On the burstiness of visual elements. In Proc. CVPR, Jun 2009.
28. H. Jégou, M. Douze, and C. Schmid. Product quantization for nearest neighbor search. IEEE PAMI, 2011.
29. H. Jégou, M. Douze, C. Schmid, and P. Pérez. Aggregating local descriptors into a compact image representation. In Proc. CVPR, 2010.
30. H. Jégou, H. Harzallah, and C. Schmid. A contextual dissimilarity measure for accurate and efficient image search. In Proc. CVPR, 2007.
31. H. Jégou, F. Perronnin, M. Douze, J. Sánchez, P. Pérez, and C. Schmid. Aggregating local images descriptors into compact codes. IEEE PAMI, 2012.
32. H. Jégou and A. Zisserman. Triangulation embedding and democratic aggregation for image search. In Proc. CVPR, 2014.
33. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proc. CVPR, 2015.
34. A. Kendall, M. Grimes, and R. Cipolla. PoseNet: A convolutional network for real-time 6-DOF camera relocalization. In Proc. ICCV, 2015.
35. J. Knopp, J. Sivic, and T. Pajdla. Avoiding confusing features in place recognition. In Proc. ECCV, 2010.
36. D. Kotzias, M. Denil, P. Blunsom, and N. de Freitas. Deep multi-instance transfer learning. CoRR, abs/1411. 3128, 2014.
37. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, pages 1106-1114, 2012.
38. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1 (4): 541-551, 1989.
39. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86 (11): 2278-2324, 1998.
40. Y. Li, N. Snavely, D. Huttenlocher, and P. Fua. Worldwide pose estimation using 3D point clouds. In Proc. ECCV, 2012.
41. T.-Y. Lin, Y. Cui, S. Belongie, and J. Hays. Learning deep representations for ground-to-aerial geolocalization. In Proc. CVPR, 2015.
42. T.-Y. Lin, A. RoyChowdhury, and S. Maji. Bilinear CNN models for fine-grained visual recognition. In Proc. ICCV, 2015.
43. D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60 (2): 91-110, 2004.
44. W. Maddern and S. Vidas. Towards robust night and day place recognition using visible and thermal imaging. In Proc. Intl. Conf. on Robotics and Automation, 2014.
45. A. Makadia. Feature tracking for wide-baseline image retrieval. In Proc. ECCV, 2010.
46. C. McManus, W. Churchill, W. Maddern, A. Stewart, and P. Newman. Shady dealings: Robust, long-term visual localisation using illumination invariance. In Proc. Intl. Conf. on Robotics and Automation, 2014.
47. S. Middelberg, T. Sattler, O. Untzelmann, and L. Kobbelt. Scalable 6-DOF localization on mobile devices. In Proc. ECCV, 2014.
48. A. Mikulik, M. Perdoch, O. Chum, and J. Matas. Learning a fine vocabulary. In Proc. ECCV, 2010.
49. M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and transferring mid-level image representations using convolutional neural networks. In Proc. CVPR, 2014.
50. M. Paulin, M. Douze, Z. Harchaoui, J. Mairal, F. Perronnin, and C. Schmid. Local convolutional features with unsupervised training for image retrieval. In Proc. ICCV, 2015.
51. F. Perronnin and D. Dance. Fisher kernels on visual vocabularies for image categorization. In Proc. CVPR, 2007.
52. F. Perronnin, Y. Liu, J. Sánchez, and H. Poirier. Large-scale image retrieval with compressed fisher vectors. In Proc. CVPR, 2010.
53. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In Proc. CVPR, 2007.
54. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Lost in quantization: Improving particular object retrieval in large scale image databases. In Proc. CVPR, 2008.
55. J. Philbin, M. Isard, J. Sivic, and A. Zisserman. Descriptor learning for efficient retrieval. In Proc. ECCV, 2010.
56. D. Qin, X. Chen, M. Guillaumin, and L. V. Gool. Quantized kernel learning for feature matching. In NIPS, 2014.
57. D. Qin, Y. Chen, M. Guillaumin, and L. V. Gool. Learning to rank bag-of-word histograms for large-scale object retrieval. In Proc. BMVC., 2014.
58. D. Qin, S. Gammeter, L. Bossard, T. Quack, and L. Van Gool. Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In Proc. CVPR, 2011.
59. D. Qin, C. Wengert, and L. V. Gool. Query adaptive similarity for large scale object retrieval. In Proc. CVPR, 2013.
60. A. S. Razavian, H. Azizpour, J. Sullivan, and S. Carlsson. CNN features off-the-shelf: An astounding baseline for recognition. CoRR, abs/1403. 6382, 2014.
61. A. S. Razavian, J. Sullivan, A. Maki, and S. Carlsson. A baseline for visual instance retrieval with deep convolutional networks. In Proc. ICLR, 2015.
62. T. Sattler, M. Havlena, F. Radenovíc, K. Schindler, and M. Pollefeys. Hyperpoints and fine vocabularies for largescale location recognition. In Proc. ICCV, 2015.
63. T. Sattler, B. Leibe, and L. Kobbelt. Fast image-based localization using direct 2D-to-3D matching. In Proc. ICCV, 2011.
64. T. Sattler, T. Weyand, B. Leibe, and L. Kobbelt. Image retrieval for image-based localization revisited. In Proc. BMVC., 2012.
65. G. Schindler, M. Brown, and R. Szeliski. City-scale location recognition. In Proc. CVPR, 2007.
66. F. Schroff, D. Kalenichenko, and J. Philbin. FaceNet: A unified embedding for face recognition and clustering. In Proc. CVPR, 2015.
67. M. Schultz and T. Joachims. Learning a distance metric from relative comparisons. In NIPS, 2004.
68. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. OverFeat: Integrated recognition, localization and detection using convolutional networks. CoRR, abs/1312. 6229, 2013.
69. E. Simo-Serra, E. Trulls, L. Ferraz, I. Kokkinos, and F. Moreno-Noguer. Fracking deep convolutional image descriptors. CoRR, abs/1412. 6537, 2014.
70. K. Simonyan, A. Vedaldi, and A. Zisserman. Descriptor learning using convex optimisation. In Proc. ECCV, 2012.
71. K. Simonyan, A. Vedaldi, and A. Zisserman. Deep Fisher networks for large-scale image classification. In NIPS, 2013.
72. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In Proc. ICLR, 2015.
73. J. Sivic and A. Zisserman. Video Google: A text retrieval approach to object matching in videos. In Proc. ICCV, volume 2, pages 1470-1477, 2003.
74. N. Sunderhauf, S. Shirazi, A. Jacobson, E. Pepperell, F. Dayoub, B. Upcroft, and M. Milford. Place recognition with ConvNet landmarks: Viewpoint-robust, condition-robust, training-free. In Robotics: Science and Systems, 2015.
75. V. Sydorov, M. Sakurada, and C. Lampert. Deep fisher kernels-end to end learning of the fisher kernel GMM parameters. In Proc. CVPR, 2014.
76. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proc. CVPR, 2014.
77. G. Tolias, Y. Avrithis, and H. Jégou. To aggregate or not to aggregate: Selective match kernels for image search. In Proc. ICCV, 2013.
78. G. Tolias and H. Jégou. Visual query expansion with or without geometry: refining local descriptors by feature aggregation. Pattern Recognition, 2014.
79. A. Torii, R. Arandjelovíc, J. Sivic, M. Okutomi, and T. Pajdla. 24/7 place recognition by view synthesis. In Proc. CVPR, 2015.
80. A. Torii, J. Sivic, T. Pajdla, and M. Okutomi. Visual place recognition with repetitive structures. In Proc. CVPR, 2013.
81. T. Turcot and D. G. Lowe. Better matching with fewer features: The selection of useful features in large database recognition problems. In ICCV Workshop on Emergent Issues in Large Amounts of Visual Data (WS-LAVD), 2009.
82. T. Tuytelaars and K. Mikolajczyk. Local invariant feature detectors: A survey. Foundations and Trends R - in Computer Graphics and Vision, 3 (3): 177-280, 2008. 1
83. P. Viola, J. C. Platt, and C. Zhang. Multiple instance boosting for object detection. In NIPS, 2005.
84. J. Wang, Y. Song, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, and Y. Wu. Learning fine-grained image similarity with deep ranking. In Proc. CVPR, 2014.
85. K. Q. Weinberger, J. Blitzer, and L. Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS, 2006.
86. S. Winder, G. Hua, and M. Brown. Picking the best DAISY. In Proc. CVPR, pages 178-185, 2009.
87. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In Proc. ECCV, 2014.
88. J. Zepeda and P. Pérez. Exemplar SVMs as visual feature encoders. In Proc. CVPR, 2015.
89. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In NIPS, 2014.