LG-CNN: From local parts to global discrimination for fine-grained recognition

Pattern Recognition

Volumn 71, Issue , 2017, Pages 118-131

  Xie, Guo Sen ^a   Zhang, Xu Yao ^a   Yang, Wenhan ^c   Xu, Mingliang ^d   Yan, Shuicheng ^e   Liu, Cheng Lin ^a,b  

^a INSTITUTE OF AUTOMATION   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c PEKING UNIVERSITY   (China)

^d ZHENGZHOU UNIVERSITY   (China)

^e NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Bilinear pooling; Convolutional neural networks; Fine grained recognition; Global discrimination; Local parts

Indexed keywords

CONVOLUTION; DISCRIMINATORS; NEURAL NETWORKS; OBJECT RECOGNITION;

BILINEAR POOLING; CONVOLUTIONAL NEURAL NETWORK; FINE GRAINED; GLOBAL DISCRIMINATION; LOCAL PARTS;

NETWORK ARCHITECTURE;

EID: 85023175472     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2017.06.002     Document Type: Article

References (69)

1. He, K., Zhang, X., Ren, S., Sun, J., Delving deep into rectifiers: surpassing human-level performance on imagenet classification. International Conference on Computer Vision (ICCV), 2015, IEEE, 1026–1034.
2. Lin, T.-Y., RoyChowdhury, A., Maji, S., Bilinear cnn models for fine-grained visual recognition. International Conference on Computer Vision (ICCV), 2015, IEEE, 1449–1457.
3. Jaderberg, M., Simonyan, K., Zisserman, A., et al. Spatial transformer networks. Advances in Neural Information Processing Systems, 2015, 2017–2025.
4. Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C., Visual categorization with bags of keypoints. ECCV Workshop on Statistical Learning in Computer Vision, 1, 2004, Prague, 1–2.
5. Penatti, O.A., Silva, F.B., Valle, E., Gouet-Brunet, V., Torres, R.D.S., Visual word spatial arrangement for image retrieval and classification. Pattern Recognit. 47:2 (2014), 705–720.
6. Lowe, D.G., Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60:2 (2004), 91–110.
7. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L., Speeded-up robust features (surf). Comput. Vision Image Understanding 110:3 (2008), 346–359.
8. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y., Locality-constrained linear coding for image classification. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, IEEE, 3360–3367.
9. Huang, Y., Wu, Z., Wang, L., Tan, T., Feature coding in image classification: a comprehensive study. IEEE Trans. Pattern Anal. Mach. Intell. 36:3 (2014), 493–506.
10. Yang, J., Yu, K., Gong, Y., Huang, T., Linear spatial pyramid matching using sparse coding for image classification. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, IEEE, 1794–1801.
11. Liu, L., Wang, L., Liu, X., In defense of soft-assignment coding. International Conference on Computer Vision (ICCV), 2011, IEEE, 2486–2493.
12. Liu, Y., Zhang, Y.-M., Zhang, X.-Y., Liu, C.-L., Adaptive spatial pooling for image classification. Pattern Recognit. 55 (2016), 58–67.
13. Shen, F., Shen, C., Zhou, X., Yang, Y., Shen, H.T., Face image classification by pooling raw features. Pattern Recognit. 54 (2016), 94–103.
14. Perronnin, F., Sánchez, J., Mensink, T., Improving the Fisher kernel for large-scale image classification. Computer Vision–ECCV 2010, 2010, Springer, 143–156.
15. Gosselin, P.-H., Murray, N., Jégou, H., Perronnin, F., Revisiting the Fisher vector for fine-grained classification. Pattern Recognit. Lett. 49 (2014), 92–98.
16. Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J., Image classification with the Fisher vector: theory and practice. Int. J. Comput. Vision 105:3 (2013), 222–245.
17. Li, Z., Gavves, E., Sande, K.E.A.V.D., Snoek, C.G.M., Smeulders, A.W.M., Codemaps - segment, classify and search objects locally. IEEE International Conference on Computer Vision, 2013, 2136–2143.
18. Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 1097–1105.
19. Simonyan, K., Zisserman, A., Very deep convolutional networks for large-scale image recognition. ICLR, 2015.
20. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, T. Darrell, Decaf: a deep convolutional activation feature for generic visual recognition, arXiv preprint arXiv:1310.1531 (2013).
21. He, K., Zhang, X., Ren, S., Sun, J., Spatial pyramid pooling in deep convolutional networks for visual recognition. Computer Vision–ECCV 2014, 2014, Springer, 346–361.
22. K. Chatfield, K. Simonyan, A. Vedaldi, A. Zisserman, Return of the devil in the details: delving deep into convolutional nets, arXiv preprint arXiv:1405.3531 (2014).
23. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A., Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, IEEE, 1–9.
24. Barat, C., Ducottet, C., String representations and distances in deep convolutional neural networks for image classification. Pattern Recognit. 54 (2016), 104–115.
25. Nogueira, K., Penatti, O.A., Santos, J.A.d., Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recognition, 2016 published online.
26. Ijjina, E.P., Chalavadi, K.M., Human action recognition using genetic algorithms and convolutional neural networks. Pattern Recognition, 2016 published online.
27. Girshick, R., Donahue, J., Darrell, T., Malik, J., Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, IEEE, 580–587.
28. Girshick, R., Fast r-cnn. International Conference on Computer Vision (ICCV), 2015, IEEE, 1440–1448.
29. Ren, S., He, K., Girshick, R., Sun, J., Faster r-cnn: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 2015, 91–99.
30. Gidaris, S., Komodakis, N., Object detection via a multi-region and semantic segmentation-aware cnn model. International Conference on Computer Vision (ICCV), 2015, IEEE, 1134–1142.
31. Ribeiro, D., Nascimento, J.C., Bernardino, A., Carneiro, G., Improving the performance of pedestrian detectors using convolutional learning. Pattern Recognition, 2016.
32. Long, J., Shelhamer, E., Darrell, T., Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, IEEE, 3431–3440.
33. Hong, S., Noh, H., Han, B., Decoupled deep neural network for semi-supervised semantic segmentation. Advances in Neural Information Processing Systems, 2015, 1495–1503.
34. Noh, H., Hong, S., Han, B., Learning deconvolution network for semantic segmentation. International Conference on Computer Vision (ICCV), 2015, IEEE, 1520–1528.
35. Zhao, F., Huang, Y., Wang, L., Tan, T., Deep semantic ranking based hashing for multi-label image retrieval. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, IEEE, 1556–1564.
36. Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., Perona, P., Caltech-UCSD Birds 200. Technical Report, CNS-TR-2010-001, 2010, California Institute of Technology.
37. Maji, S., Kannala, J., Rahtu, E., Blaschko, M., Vedaldi, A., Fine-Grained Visual Classification of Aircraft. Technical Report, 2013.
38. Zhang, N., Farrell, R., Darrell, T., Pose pooling kernels for sub-category recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, IEEE, 3665–3672.
39. K. Simonyan, A. Vedaldi, A. Zisserman, Deep inside convolutional networks: visualising image classification models and saliency maps, arXiv preprint arXiv:1312.6034 (2013).
40. Gulshan, V., Rother, C., Criminisi, A., Blake, A., Zisserman, A., Geodesic star convexity for interactive image segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, IEEE, 3129–3136.
41. Zhang, N., Farrell, R., Iandola, F., Darrell, T., Deformable part descriptors for fine-grained recognition and attribute prediction. International Conference on Computer Vision (ICCV), 2013, IEEE, 729–736.
42. Rubio, J.C., Eigenstetter, A., Ommer, B., Generative regularization with latent topics for discriminative object recognition. Pattern Recognit. 48:12 (2015), 3871–3880.
43. Berg, T., Belhumeur, P., Poof: part-based one-vs.-one features for fine-grained categorization, face verification, and attribute estimation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, 955–962.
44. Göring, C., Rodner, E., Freytag, A., Denzler, J., Nonparametric part transfer for fine-grained recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, IEEE, 2489–2496.
45. Gavves, E., Fernando, B., Snoek, C.G., Smeulders, A.W., Tuytelaars, T., Local alignments for fine-grained categorization. Int. J. Comput. Vision 111:2 (2015), 191–212.
46. Chai, Y., Lempitsky, V., Zisserman, A., Symbiotic segmentation and part localization for fine-grained categorization. International Conference on Computer Vision (ICCV), 2013, 321–328.
47. Zhang, N., Donahue, J., Girshick, R., Darrell, T., Part-based r-cnns for fine-grained category detection. Computer Vision–ECCV 2014, 2014, Springer, 834–849.
48. S. Branson, G. Van Horn, S. Belongie, P. Perona, Bird species categorization using pose normalized deep convolutional nets, arXiv preprint arXiv:1406.2952 (2014).
49. Lin, D., Shen, X., Lu, C., Jia, J., Deep lac: deep localization, alignment and classification for fine-grained recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, 1666–1674.
50. Krause, J., Jin, H., Yang, J., Fei-Fei, L., Fine-grained recognition without part annotations. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, 5546–5555.
51. N. Zhang, E. Shelhamer, Y. Gao, T. Darrell, Fine-grained pose prediction, normalization, and recognition, arXiv preprint arXiv:1511.07063 (2015).
52. S. Huang, Z. Xu, D. Tao, Y. Zhang, Part-stacked cnn for fine-grained visual categorization, arXiv preprint arXiv:1512.08086 (2015).
53. Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z., The application of two-level attention models in deep convolutional neural network for fine-grained image classification. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, IEEE, 842–850.
54. Wang, D., Shen, Z., Shao, J., Zhang, W., Xue, X., Zhang, Z., Multiple granularity descriptors for fine-grained categorization. IEEE International Conference on Computer Vision (ICCV), 2015, 2399–2406.
55. Simon, M., Rodner, E., Neural activation constellations: unsupervised part model discovery with convolutional networks. International Conference on Computer Vision (ICCV), 2015, IEEE, 1143–1151.
56. Y. Gao, O. Beijbom, N. Zhang, T. Darrell, Compact bilinear pooling, arXiv preprint arXiv:1511.06062 (2015).
57. Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S., Salient object detection: a discriminative regional feature integration approach. IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2013, 2083–2090.
58. Shi, J., Yan, Q., Xu, L., Jia, J., Hierarchical image saliency detection on extended cssd. IEEE Trans. Pattern Anal. Mach. Intell. 38:4 (2016), 717–729.
59. Xie, G., Zhang, X., Yan, S., Liu, C., Hybrid CNN and dictionary-based models for scene recognition and domain adaptation. CoRR, abs/1601.07977, 2016.
60. Yoo, D., Park, S., Lee, J.-Y., Kweon, I.S., Fisher kernel for deep neural activations. Proceedings of CVPR Workshops, 2015.
61. Nilsback, M.-E., Zisserman, A., Automated flower classification over a large number of classes. Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing, 2008.
62. Parkhi, O.M., Vedaldi, A., Zisserman, A., Jawahar, C., Cats and dogs. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, IEEE, 3498–3505.
63. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T., Caffe: convolutional architecture for fast feature embedding. ACM International Conference on Multimedia, 2014, 675–678.
64. Perronnin, F., Dance, C., Fisher kernels on visual vocabularies for image categorization. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007, 1–8.
65. Angelova, A., Zhu, S., Efficient object detection and segmentation for fine-grained recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, 811–818.
66. Murray, N., Perronnin, F., Generalized max pooling. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, IEEE, 2473–2480.
67. Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S., Cnn features off-the-shelf: an astounding baseline for recognition. Proceedings of CVPR Workshops, 2014, 512–519.
68. Azizpour, H., Razavian, A., Sullivan, J., Maki, A., Carlsson, S., From generic to specific deep representations for visual recognition. Proceedings of CVPR Workshops, 2015, 36–45.
69. Fei-Fei, L., Fergus, R., Perona, P., Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. Comput. Vision Image Understanding 106:1 (2007), 59–70.