Free-form region description with second-order pooling

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 37, Issue 6, 2015, Pages 1177-1189

  Carreira, João ^a,b   Caseiro, Rui ^b   Batista, Jorge ^b   Sminchisescu, Cristian ^c,d  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF COIMBRA   (Portugal)

^c LUND UNIVERSITY   (Sweden)

^d INSTITUTE OF MATHEMATICS   (Romania)

Author keywords

differential geometry; image descriptors; pooling; Recognition; regression; second order statistics; segmentation

Indexed keywords

EXTRACTION; FEATURE EXTRACTION; GEOMETRY; IMAGE CLASSIFICATION; IMAGE CODING; IMAGE PROCESSING; IMAGE SEGMENTATION; SEMANTICS;

DIFFERENTIAL GEOMETRY; IMAGE DESCRIPTORS; POOLING; RECOGNITION; REGRESSION; SECOND ORDER STATISTICS;

OBJECT DETECTION;

EID: 84929223025     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2014.2361137     Document Type: Article

References (69)

1. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, "The Pascal visual object classes (voc) challenge," Int. J. Comput. Vis., vol. 88, 2010, pp. 303-338.
2. J. Carreira, F. Li, and C. Sminchisescu, "Object recognition by sequential figure-ground ranking," Int. J. Comput. Vis., vol. 98, 2012, pp. 243-262.
3. A. Ion, J. Carreira, and C. Sminchisescu, "Probabilistic joint segmentation and labeling," in Proc. Adv. Neural Inf. Process. Syst., 2011, pp. 40-57.
4. P. Arbelaez, B. Hariharan, C. Gu, S. Gupta, L. Bourdev, and J. Malik, "Semantic segmentation using regions and parts," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2012, pp. 3378-3385.
5. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2014, pp. 580-587.
6. J. Sivic and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos," in Proc. IEEE Int. Conf. Comput. Vis., 2003, pp. 1470-1477.
7. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2006, pp. 2169-2178.
8. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2005, pp. 886-893.
9. Y. Boureau, J. Ponce, and Y. LeCun, "A theoretical analysis of feature pooling in vision algorithms," in Proc. Int. Conf. Mach. Learn., 2010, pp. 111-118.
10. Y. Boureau, N. Le Roux, F. Bach, J. Ponce, and Y. LeCun, "Ask the locals: Multi-way local pooling for image recognition," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 2651-2658.
11. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks." in Proc. NIPS, vol. 1, no. 2, 2012, p. 4.
12. V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, "Geometric means in a novel vector space structure on symmetric positivedefinite matrices," in Proc. SIAM J. Math. Anal. Appl., 2006, pp. 328-347.
13. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, Nov. 2004, pp. 91-110.
14. T. Sebastian, P. Klein, and B. Kimia, "Recognition of shapes by editing their shock graphs," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 5, pp. 550-571, May 2004.
15. S. Belongie, J. Malik, and J. Puzicha, "Shape context: A new descriptor for shape matching and object recognition," in Proc. Adv. Neural Inf. Process. Syst., 2000, pp. 509-522.
16. L. Latecki, R. Lakamper, and T. Eckhardt, "Shape descriptors for non-rigid shapes with a single closed contour," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2000, pp. 424-429.
17. R. Fergus, P. Perona, and A. Zisserman, "Weakly supervised scale-invariant learning of models for visual recognition," Int. J. Comput. Vis., vol. 71, pp. 273-303, 2007.
18. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
19. K. Grauman and T. Darrell, "The pyramid match kernel: Discriminative classification with sets of image features," in Proc. IEEE Int. Conf. Comput. Vis., 2005, pp. 1458-1465.
20. T. Tuytelaars, M. Fritz, K. Saenko, and T. Darrell, "The nbnn kernel," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 1824-1831.
21. J. Farquhar, S. Szedmak, H. Meng, and J. Shawe-Taylor, "Improving "bag-of-keypoints" image categorisation: Generative models and pdf-kernels," 2005.
22. L. Bo and C. Sminchisescu, "Efficient Match Kernel between Sets of Features for Visual Recognition," in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 135-143.
23. T. Joachims, "Training linear SVMs in linear time," in Proc. ACM Conf. Knowledge Discov. Data Min., 2006, pp. 217-226.
24. B. Hariharan, J. Malik, and D. Ramanan, "Discriminative decorrelation for clustering and classification," in Proc. Eur. Conf. Comput. Vis.., 2012, pp. 459-472.
25. K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman, "The devil is in the details: An evaluation of recent feature encoding methods," in Proc. British Mach. Vis. Conf., 2011, pp. 1-12.
26. M.-K. Hu, "Visual pattern recognition by moment invariants," IRE Trans. Inf. Theory, vol. 8, no. 2, pp. 179-187, 1962.
27. O. Tuzel, F. Porikli, and P. Meer, "Pedestrian detection via classification on Riemannian manifolds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 10, pp. 1713-1727, Oct. 2008.
28. R. Caseiro, J. Henriques, P. Martins, and J. Batista, "A nonparametric Riemannian framework on tensor field with application to foreground segmentation," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 1-8.
29. H. Nakayama, T. Harada, and Y. Kuniyoshi, "Global Gaussian approach for scene categorization using information geometry," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2336-2343.
30. F. Perronnin, J. Sánchez, and T. Mensink, "Improving the Fisher kernel for large-scale image classification," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 143-156.
31. D. Picard and P.-H. Gosselin, "Improving image similarity with vectors of locally aggregated tensors," in Proc. 18th IEEE Int. Conf. Image Process., 2011, pp. 669-672.
32. J. Shotton, J. Winn, C. Rother, and A. Criminisi, "Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context," Int. J. Comput. Vis., vol. 81, pp. 2-23, 2009.
33. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling," IEEE Trans. Pattern Anal. Mach. Intell., no. 8, pp. 1915-1929, 2013.
34. B. Fulkerson, A. Vedaldi, and S. Soatto, "Class segmentation and object localization with superpixel neighborhoods," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 670-677.
35. X. Boix, J. M. Gonfaus, J. van de Weijer, A. D. Bagdanov, J. Serrat, and J. Gonzalez, "Harmony potentials: Fusing global and local scale for semantic image segmentation," Int. J. Comput. Vis., vol. 91, pp. 83-102, 2012.
36. X. Ren, L. Bo, and D. Fox, "Rgb-(d) scene labeling: Features and algorithms," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2012, pp. 2759-2766.
37. T. Malisiewicz and A. Efros, "Recognition by association via learning per-exemplar distances," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
38. C. Gu, J. J. Lim, P. Arbeláez, and J. Malik, "Recognition using regions," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1030-1037.
39. J. Tighe and S. Lazebnik, "Superparsing-scalable nonparametric image parsing with superpixels," Int. J. Comput. Vis., vol. 101, no. 2, pp. 329-349, 2013.
40. L. Ladicky, P. Sturgess, K. Alaharia, C. Russel, and P. H. Torr, "What, where & how many? Combining object detectors and CRFs," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 424-437.
41. W. Xia, C. Domokos, J. Dong, L.-F. Cheong, and S. Yan, "Semantic segmentation without annotating segments," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 2176-2183.
42. M. P. Carmo, Riemannian Geometry. Cambridge, MA, USA: Birkhauser, 1992.
43. X. Pennec, P. Fillard, and N. Ayache, "A Riemannian framework for tensor computing," Int. J. Comput. Vis., vol. 66, pp. 41-66, 2006.
44. B. Caputo and L. Jie, "A performance evaluation of exact and approximate match kernels for object recognition," Electron. Lett. Comput. Vis. Image Anal., vol. 8, pp. 15-26, 2010.
45. P. I. Davies and N. J. Higham, "A Schur-Parlett algorithm for computing matrix functions," SIAM J. Matrix Anal. Appl., vol. 25, pp. 464-485, 2003.
46. A. Vedaldi, and A. Zisserman, "Efficient additive kernels via explicit feature maps," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 480-492, Mar. 2011.
47. O. Boiman, E. Shechtman, and M. Irani, "In defense of nearestneighbor based image classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
48. P. Koniusz and K. Mikolajczyk, "Spatial coordinate coding to reduce histogram representations, dominant angle and colour pyramid match," in Proc. 18th IEEE Int. Conf., 2011, pp. 661-664.
49. J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu, "Semantic segmentation with second-order pooling," in Proc. Eur. Conf. Comput. Vis., 2012, pp. 430-443.
50. J. Sánchez, F. Perronnin, and T. De Campos, "Modeling the spatial layout of images beyond spatial pyramids," Pattern Recognit. Lett., vol. 33, no. 16, pp. 2216-2223, 2012.
51. J. Van De Weijer and C. Schmid, "Coloring local feature extraction," in Proc. Eur. Conf. Comput. Vis., Springer, 2006, pp. 334-348.
52. T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution grayscale and rotation invariant texture classification with local binary patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971-987, Jul. 2002.
53. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2001, pp. I-511-I-518.
54. J. Carreira and C. Sminchisescu, "CPMC: Automatic object segmentation using constrained parametric min-cuts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 7, pp. 1312-1328, Jul. 2012.
55. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "From contours to regions: An empirical evaluation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 2294-2301.
56. B. Hariharan, P. Arbelaez, L. Bourdev, S. Maji, and J. Malik, "Semantic contours from inverse detectors," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 991-998.
57. A. Vedaldi and B. Fulkerson. (2008). VLFeat: An open and portable library of computer vision algorithms. [Online]. Available: http://vlfeat.org/
58. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, pp. 1-27, 2011.
59. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, "Locality-constrained linear coding for image classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2010, pp. 3360-3367.
60. L. Fei-Fei, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories," in Proc. Comput. Vis. Image Understanding, 2007, p. 178.
61. O. Duchenne, A. Joulin, and J. Ponce, "A graph-matching kernel for object categorization," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 1792-1799.
62. A. Ion, J. Carreira, and C. Sminchisescu, "Probabilistic image segmentation and labeling by figure-ground composition," Int. J. Comput. Vis., vol. 107, pp. 40-57, 2014.
63. I. Jolliffe, Principal Component Analysis. New York, NY, USA: Springer Verlag, 1986.
64. M. Maire, S. X. Yu, and P. Perona, "Object detection and segmentation from joint embedding of parts and pixels," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 2142-2149.
65. P. Yadollahpour, D. Batra, and G. Shakhnarovich, "Discriminative re-ranking of diverse segmentations," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2013, pp. 1923-1930.
66. Z. Li, E. Gavves, K. E. van de Sande, and C. G. Snoek, "Codemaps segment, classify and search objects locally," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2136-2143.
67. F. Li, J. Carreira, G. Lebanon, and C. Sminchisescu, "Composite statistical inference for semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 3302-3309.
68. S. Fidler, R. Mottaghi, A. Yuille, and R. Urtasun, "Bottom-up segmentation for top-down detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 3294-3301.
69. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. (2011). The PASCAL Visual Object Classes Challenge 2011 (VOC2011) Results. [Online]. Available: http://pascal-network. org/challenges/VOC/voc2011/workshop/index.html