Multi-scale context for scene labeling via flexible segmentation graph

Pattern Recognition

Volumn 59, Issue , 2016, Pages 312-324

  Zhou, Quan ^a   Zheng, Baoyu ^a   Zhu, Weiping ^b   Jan Latecki, Longin ^c  

^a NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b Gina Cody School of Engineering and Computer Science   (Canada)

^c Temple University   (United States)

Author keywords

Classification; Feature extraction; Flexible segmentation graph; Multi scale context; Scene labeling; Semantic segmentation

Indexed keywords

CLASSIFICATION (OF INFORMATION); FEATURE EXTRACTION; HIERARCHICAL SYSTEMS; IMAGE PROCESSING; IMAGE SEGMENTATION; SEMANTICS;

CONDITIONAL RANDOM FIELDS(CRFS); CONTEXTUAL INFORMATION; DISCRIMINATIVE CLASSIFIERS; FLEXIBLE SEGMENTATION GRAPH; IMAGE PROCESSING AND COMPUTER VISION; MULTI-SCALE CONTEXT; SEMANTIC SEGMENTATION; SPATIAL RELATIONSHIPS;

COMPUTER VISION;

EID: 84975686260     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2016.03.023     Document Type: Article

References (61)

1. [1] Y. Yang, S. Hallman, D. Ramanan, C. Fowlkes, Layered object detection for multi-class segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2010, pp. 3113–3120.
2. [2] Lin, L., Wang, X., Yang, W., Lai, J., Discriminatively trained and-or graph models for object shape detection. IEEE Trans. Pattern Anal. Mach. Intell. 37:5 (2015), 959–972.
3. [3] Uijlings, J.R., van de Sande, K.E., Gevers, T., Smeulders, A.W., Selective search for object recognition. Int. J. Comput. Vis. 104:2 (2013), 154–171.
4. [4] Wang, K., Lin, L., Lu, J., Li, C., Shi, K., Pisa: Pixelwise image saliency by aggregating complementary appearance contrast measures with edge-preserving coherence. IEEE Trans. Image Process. 24:10 (2015), 3019–3033.
5. [5] D. Hoiem, A. Efros, M. Hebert, Closing the loop in scene interpretation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
6. [6] Lin, L., Zhang, R., Duan, X., Adaptive scene category discovery with generative learning and compositional sampling. IEEE Trans. Circuits Syst. Video Technol. 25:2 (2015), 251–260.
7. [7] Lerouge, J., Herault, R., Chatelain, C., Jardin, F., Modzelewski, R., Ioda: an input/output deep architecture for image labeling. Pattern Recognit. 48:9 (2015), 2847–2858.
8. [8] C. Liu, J. Yuen, A. Torralba, Nonparametric scene parsing: label transfer via dense scene alignment, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 1972–1979.
9. [9] Liu, C., Yuen, J., Torralba, A., Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell. 33:12 (2011), 2368–2382.
10. [10] Ma, J., Zhou, H., Zhao, J., Tian, J., Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans. Geosci. Remote Sens. 53:12 (2015), 6469–6481.
11. [11] Ma, J., Zhao, J., Yuille, A.L., Non-rigid point set registration by preserving global and local structures. IEEE Trans. Image Process. 25:1 (2016), 53–64.
12. [12] B. Shuai, G. Wang, Z. Zuo, B. Wang, L. Zhao, Integrating parametric and non-parametric models for scene labeling, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 4249–4258.
13. [13] A. Roy, S. Todorovic, Scene labeling using beam search under mutex constraints, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1178–1185.
14. [14] A. Sharma, O. Tuzel, D.W. Jacobs, Deep hierarchical parsing for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 530–538.
15. [15] J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–8.
16. [16] F. Li, J. Carreira, G. Lebanon, C. Sminchisescu, Composite statistical inference for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 4249–4258.
17. [17] Y. Liu, J. Liu, Z. Li, J. Tang, H. Lu, Weakly-supervised dual clustering for image semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 2075–2082.
18. [18] Borenstein, E., Ullman, S., Combined top-down/bottom-up segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 30:12 (2008), 2109–2125.
19. [19] J. Xu, A.G. Schwing, R. Urtasun, Learning to segment under various forms of weak supervision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3781–3790.
20. [20] R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, A. Yuille, The role of context for object detection and semantic segmentation in the wild, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1–8.
21. [21] Farabet, C., Couprie, C., Najman, L., LeCun, Y., Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1915–1929.
22. [22] J. Yang, B. Price, S. Cohen, M.-H. Yang, Context driven scene parsing with attention to rare classes, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3294–3301.
23. [23] Zhou, Q., Zhu, J., Liu, W., Learning dynamic hybrid Markov random field for image labeling. IEEE Trans. Image Process. 22:6 (2013), 2219–2232.
24. [24] Belongie, S., Malik, J., Puzicha, J., Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24:24 (2002), 509–522.
25. [25] Geman, S., Geman, D., Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6:6 (1984), 721–741.
26. [26] D. Munoz, J.A. Bagnell, M. Hebert, Stacked hierarchical labeling, in: Proceedings of the Europe Conference on Computer Vision, 2010, pp. 57–70.
27. [27] Mensink, T., Verbeek, J., Csurka, G., Tree-structured CRF models for interactive image labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35:2 (2013), 476–489.
28. [28] Comaniciu, D., Meer, P., Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24:5 (2002), 603–619.
29. [29] Felzenszwalb, P.F., Huttenlocher, D.P., Efficient graph-based image segmentation. Int. J. Comput. Vis. 59:2 (2004), 167–181.
30. [30] J.J. Lim, P. Arbeláez, C. Gu, J. Malik, Context by region ancestry, in: Proceedings of the IEEE Conference on Computer Vision, 2009, pp. 1978–1985.
31. [31] C. Pantofaru, C. Schmid, M. Hebert, Object recognition by integrating multiple image segmentations, in: Proceedings of the Europe Conference on Computer Vision, 2008, pp. 481–494.
32. [32] Q. Zhou, C. Yan, Y. Zhu, X. Bai, W. Liu, Image labeling by multiple segmentation, in: Proceedings of the IEEE Conference on Image Processing, 2011, pp. 3129–3132.
33. [33] Tu, Z.W., Bai, X., Auto-context and its application to high-level vision tasks and 3d brain image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32:10 (2010), 1744–1757.
34. [34] Kumar, S., Hebert, M., Discriminative random fields. Int. J. Comput. Vis. 68:2 (2006), 179–201.
35. [35] Shotton, J., Winn, J.M., Rother, C., Criminisi, A., TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vis. 81:1 (2009), 2–23.
36. [36] C. Galleguillos, A. Rabinovich, S. Belongie, Object categorization using co-occurrence, location and appearance, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
37. [37] Gould, S., Rodgers, J., Cohen, D., Elidan, G., Koller, D., Multi-class segmentation with relative location prior. Int. J. Comput. Vis. 80:3 (2008), 1239–1253.
38. [38] P. Yadollahpour, D. Batra, G. Shakhnarovich, Discriminative re-ranking of diverse segmentations, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 1923–1930.
39. [39] L. Ladicky, P. Sturgess, K. Alahari, C. Russell, P. Torr, What, where and how many? Combining object detectors and CRFs, in: Proceedings of the Europe Conference on Computer Vision, 2010, pp. 424–437.
40. [40] S. Gould, T. Gao, D. Koller, Region-based segmentation and object detection, in: Proceedings of the Conference on Neural Information Processing Systems, 2009, pp. 655–663.
41. [41] J. Yao, S. Fidler, R. Urtasun, Describing the scene as a whole: joint object detection, scene classification and semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 702–709.
42. [42] Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S., Associative hierarchical random fields. IEEE Trans. Pattern Anal. Mach. Intell. 36:6 (2014), 1056–1077.
43. [43] V. Lempitsky, A. Vedaldi, A. Zisserman, Pylon model for semantic segmentation, in: Proceedings of the Conference on Neural Information Processing Systems, 2011.
44. [44] J. Zhu, T. Wu, J. Zhu, X. Yang, W. Zhang, Learning reconfigurable scene representation by tangram model, in: Proceedings of the IEEE Workshop on the Applications of Computer Vision, 2012, pp. 449–456.
45. [45] Zhu, L., Chen, Y., Lin, Y., Lin, C., Yuille, A., Recursive segmentation and recognition templates for image parsing. IEEE Trans. Pattern Anal. Mach. Intell. 34:2 (2012), 359–371.
46. [46] H. Caesar, J. Uijlings, V. Ferrari, Joint calibration for semantic segmentation, arXiv preprint arXiv:1507.01581.
47. [47] Liu, F., Lin, G., Shen, C., CRF learning with CNN features for image segmentation. Pattern Recognit. 48:10 (2015), 2983–2992.
48. [48] Mottaghi, R., Fidler, S., Yuille, A., Urtasun, R., Parikh, D., Human–machine CRFs for identifying bottlenecks in scene understanding. Trans. Pattern Anal. Mach. Intell. IEEE Trans. Pattern Anal. Mach. Intell. 38:1 (2016), 74–87.
49. [49] Arbelaez, P., Maire, M., Fowlkes, C., Malik, J., Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33:5 (2011), 898–916.
50. [50] Najman, L., Schmitt, M., Geodesic saliency of watershed contours and hierarchical segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18:12 (1996), 1163–1173.
51. [51] Shi, J., Malik, J., Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22:8 (2000), 888–905.
52. [52] Friedman, J., Hastie, T., Tibshirani, R., Additive logistic regression: a statistical view of boosting. Ann. Stat. 28:2 (2000), 337–407.
53. [53] Barnard, K., Duygulu, P., Forsyth, D., De Freitas, N., Blei, D.M., Jordan, M.I., Matching words and pictures. J. Mach. Learn. Res. 3 (2003), 1107–1135.
54. [54] Zhang, J., Marszałek, M., Lazebnik, S., Schmid, C., Local features and kernels for classification of texture and object categories: a comprehensive study. Int. J. Comput. Vis. 73:2 (2007), 213–238.
55. [55] M.J. Choi, J.J. Lim, A. Torralba, A.S. Willsky, Exploiting hierarchical context on a large database of object categories, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2010, pp. 129–136.
56. [56] Oliva, A., Torralba, A., The role of context in object recognition. Trends Cogn. Sci. 11:12 (2007), 520–527.
57. [57] B. Yao, X. Yang, S.C. Zhu, Introduction to a large scale general purpose groundtruth dataset: methodology, annotation tool, and benchmarks, in: Proceedings of the Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, 2007.
58. [58] Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A., The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88:2 (2010), 303–338.
59. [59] P. Krahenbuhl, V. Koltun, Efficient inference in fully connected CRFs with gaussian edge potentials, in: Proceedings of the Conference on Neural Information Processing Systems, 2011, pp. 109–117.
60. [60] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A.L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected CRFs, arXiv preprint arXiv:1412.7062.
61. [61] S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, P. H. S. Torr, Conditional random fields as recurrent neural networks, in: Proceedings of the IEEE Conference on Computer Vision, 2015, pp. 1529–1537.