Scene parsing using inference Embedded Deep Networks

Pattern Recognition

Volumn 59, Issue , 2016, Pages 188-198

  Bu, Shuhui ^a   Han, Pengcheng ^a   Liu, Zhenbao ^a   Han, Junwei ^a  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

Author keywords

Conditional Random Fields (CRFs); Convolutional Neural Networks (CNNs); Hybrid Features; Inference Embedded Deep Networks (IEDNs)

Indexed keywords

CONVOLUTION; NETWORK ARCHITECTURE; NEURAL NETWORKS;

CONDITIONAL RANDOM FIELDS(CRFS); CONVOLUTIONAL NEURAL NETWORK; GRAPHICAL MODEL; HIERARCHICAL INFORMATION; HYBRID FEATURES; INFERENCE EMBEDDED DEEP NETWORKS (IEDNS); NOVEL NEURAL NETWORK; SPATIAL RELATIONSHIPS;

GRAPHIC METHODS;

EID: 84958191683     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2016.01.027     Document Type: Article

References (62)

1. [1] Shotton, J., Winn, J., Rother, C., Criminisi, A., Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vis. 81:1 (2009), 2–23.
2. [2] Farabet, C., Couprie, C., Najman, L., LeCun, Y., Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1915–1929.
3. [3] J. Yang, K. Yu, Y. Gong, T. Huang, Linear spatial pyramid matching using sparse coding for image classification, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, Miami, FL, USA, 2009, pp. 1794–1801.
4. [4] Bengio, Y., Learning deep architectures for AI. Found. Trends Mach. Learn. 2:1 (2009), 1–127.
5. [5] Han, J., He, S., Qian, X., Wang, D., Guo, L., Liu, T., An object-oriented visual saliency detection framework based on sparse coding representations. IEEE Trans. Circuits Syst. Video Technol. 23:12 (2013), 2009–2021.
6. [6] Bu, S., Han, P., Liu, Z., Li, K., Han, J., Shift-invariant ring feature for 3d shape. Vis. Comput. 30:6–8 (2014), 867–876.
7. ® Comput. Graph. Vis. 6:3–4 (2011), 185–365.
8. [8] Lin, L., Wang, X., Yang, W., Lai, J.-H., Discriminatively trained and-or graph models for object shape detection. IEEE Trans. Pattern Anal. Mach. Intell. 37:5 (2015), 959–972.
9. [9] Oliva, A., Torralba, A., Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42:3 (2001), 145–175.
10. [10] N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, IEEE, San Diego, California, USA, 2005, pp. 886–893.
11. [11] Lowe, D.G., Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60:2 (2004), 91–110.
12. [12] Bay, H., Ess, A., Tuytelaars, T., Van Gool, L., Speeded-up robust features (surf). Comput. Vis. Image Understand. 110:3 (2008), 346–359.
13. [13] Han, J., Chen, C., Shao, L., Hu, X., Liu, T., Learning computational models of video memorability from fmri brain imaging. IEEE Trans. Cybern. 45:8 (2015), 1692–1703.
14. [14] J. Han, D. Zhang, S. Wen, L. Gao, T. Liu, X. Li, Two-stage learning to predict human eye fixations via sdaes, IEEE Trans. Cybern. 46 (2) (2016) 487–498. 10.1109/TCYB.2015.2404432.
15. [15] Q.V. Le, Building high-level features using large scale unsupervised learning, in: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Vancouver, Canada, 2013, pp. 8595–8598.
16. [16] Q.V. Le, W.Y. Zou, S.Y. Yeung, A.Y. Ng, Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis, in: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Colorado Springs, USA, 2011, pp. 3361–3368.
17. [17] H. Lee, R. Grosse, R. Ranganath, A.Y. Ng, Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, in: Proceedings of the 26th Annual International Conference on Machine Learning, ACM, Montreal, Canada, 2009, pp. 609–616.
18. [18] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, T. Darrell, Decaf: a deep convolutional activation feature for generic visual recognition, arXiv preprint arXiv:1310.1531.
19. [19] Bu, S., Liu, Z., Han, J., Wu, J., Ji, R., Learning high-level feature by deep belief networks for 3-d model retrieval and recognition. IEEE Trans. Multimed. 16:8 (2014), 2154–2167.
20. [20] Bu, S., Han, P., Liu, Z., Han, J., Lin, H., Local deep feature learning framework for 3d shape. Comput. Graph. 46 (2015), 117–129.
21. [21] Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., Wu, F., Background prior-based salient object detection via deep reconstruction residual. IEEE Trans. Circuits Syst. Video Technol. 25:8 (2015), 1309–1321.
22. [22] Han, J., Zhang, D., Cheng, G., Guo, L., Ren, J., Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Trans. Geosci. Remote Sens. 53:6 (2015), 3325–3337.
23. [23] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, 2012, pp. 1097–1105.
24. [24] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, Overfeat: Integrated recognition, localization and detection using convolutional networks, in: International Conference on Learning Representations (ICLR 2014), CBLS, 2014.
25. [25] Ding, S., Lin, L., Wang, G., Chao, H., Deep feature learning with relative distance comparison for person re-identification. Pattern Recognit. 48:10 (2015), 2993–3003.
26. [26] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A.L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected crfs, in: International Conference on Learning Representations, 2015.
27. [27] Cortes, C., Vapnik, V., Support-vector networks. Mach. Learn. 20:3 (1995), 273–297.
28. [28] Haykin, S., Network, N., A comprehensive foundation. Neural Netw., 2, 2004.
29. [29] M.D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, in: Computer Vision–ECCV 2014, Springer, Zurich, Switzerland, 2014, pp. 818–833.
30. [30] J. Lafferty, A. McCallum, F.C. Pereira, Conditional random fields: probabilistic models for segmenting and labeling sequence data, 2001.
31. [31] A. Kae, K. Sohn, H. Lee, E. Learned-Miller, Augmenting crfs with Boltzmann machine shape priors for image labeling, in: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2013, pp. 2019–2026.
32. [32] Hinton, G.E., Salakhutdinov, R.R., Reducing the dimensionality of data with neural networks. Science 313:5786 (2006), 504–507.
33. [33] P. Luo, L. Lin, X. Liu, Learning compositional shape models of multiple distance metrics by information projection, IEEE Trans. Neural Netw. Learn. Syst. 10.1109/TNNLS.2015.2440430.
34. [34] C. Russell, P. Kohli, P.H. Torr, et al., Associative hierarchical crfs for object class image segmentation, in: 2009 IEEE 12th International Conference on Computer Vision, IEEE, Miami, FL, USA, 2009, pp. 739–746.
35. [35] S. Gould, R. Fulton, D. Koller, Decomposing a scene into geometric and semantically consistent regions, in: 2009 IEEE 12th International Conference on Computer Vision, IEEE, Kyoto, Japan, 2009, pp. 1–8.
36. [36] J. Tighe, S. Lazebnik, Superparsing: scalable nonparametric image parsing with superpixels, in: Computer Vision–ECCV 2010, Springer, Crete, Greece, 2010, pp. 352–365.
37. [37] Wang, K., Lin, L., Lu, J., Li, C., Shi, K., Pisa: pixelwise image saliency by aggregating complementary appearance contrast measures with edge-preserving coherence. IEEE Trans. Image Process. 24:10 (2015), 3019–3033.
38. [38] P. Krähenbühl, V. Koltun, Efficient inference in fully connected crfs with gaussian edge potentials, in: Advances in Neural Information Processing Systems, 2011, pp. 109–117.
39. [39] R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Columbus, Ohio, 2014, pp. 580–587.
40. [40] R. Socher, C.C. Lin, C. Manning, A.Y. Ng, Parsing natural scenes and natural language with recursive neural networks, in: Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 129–136.
41. [41] H. Li, R. Zhao, X. Wang, Highly efficient forward and backward propagation of convolutional neural networks for pixelwise classification, arXiv preprint arXiv:1412.4526.
42. [42] S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, P. Torr, Conditional random fields as recurrent neural networks, arXiv preprint arXiv:1502.03240.
43. [43] G. Lin, C. Shen, I. Reid, et al., Efficient piecewise training of deep structured models for semantic segmentation, arXiv preprint arXiv:1504.01013.
44. [44] M. Cogswell, X. Lin, S. Purushwalkam, D. Batra, Combining the best of graphical models and convnets for semantic segmentation, arXiv preprint arXiv:1412.4313.
45. [45] F. Liu, G. Lin, C. Shen, Crf learning with cnn features for image segmentation, Pattern Recognit. 48 (10) (2015) 2983-2992. 10.1016/j.patcog.2015.04.019.
46. [46] J.J. Tompson, A. Jain, Y. LeCun, C. Bregler, Joint training of a convolutional network and a graphical model for human pose estimation, in: Advances in Neural Information Processing Systems, 2014, pp. 1799–1807.
47. [47] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S., Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34:11 (2012), 2274–2282.
48. [48] Boykov, Y., Veksler, O., Zabih, R., Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23:11 (2001), 1222–1239.
49. [49] Boykov, Y., Kolmogorov, V., An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26:9 (2004), 1124–1137.
50. [50] Freund, Y., Haussler, D., Computer Research Laboratory. Unsupervised learning of distributions of binary vectors using two layer networks, 1994, University of California, Santa Cruz.
51. [51] Hinton, G.E., Training products of experts by minimizing contrastive divergence. Neural Comput. 14:8 (2002), 1771–1800.
52. [52] C. Liu, J. Yuen, A. Torralba, Nonparametric scene parsing: label transfer via dense scene alignment, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, Miami, FL, USA, 2009, pp. 1972–1979.
53. [53] D. Eigen, R. Fergus, Nonparametric image parsing using adaptive neighbor sets, in: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2012, pp. 2799–2806.
54. [54] G. Singh, J. Kosecka, Nonparametric scene parsing with adaptive feature relevance and semantic context, in: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Columbus, Ohio, USA, 2013, pp. 3151–3157.
55. [55] P.H.O. Pinheiro, R. Collobert, P.H.O. Pinheiro, R. Collobert, Recurrent convolutional neural networks for scene labeling, in: in International Conference on Machine Learning (ICML), 2014.
56. [56] A. Vedaldi, K. Lenc, Matconvnet – convolutional neural networks for matlab, CoRR abs/1412.4564.
57. [57] Liu, C., Yuen, J., Torralba, A., Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell. 33:12 (2011), 2368–2382.
58. [58] Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A., The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111:1 (2015), 98–136.
59. [59] B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji, J. Malik, Semantic contours from inverse detectors, in: 2011 IEEE International Conference on Computer Vision (ICCV), IEEE, Barcelona, Spain, 2011, pp. 991–998.
60. [60] Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A., The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88:2 (2010), 303–338.
61. [61] M. Mostajabi, P. Yadollahpour, G. Shakhnarovich, Feedforward semantic segmentation with zoom-out features, in: IEEE Conference on Computer Vision and Pattern Recognition, 2015. CVPR 2015. IEEE, Boston, USA, 2015, pp. 3376-3385.
62. [62] G. Papandreou, L.-C. Chen, K. Murphy, A.L. Yuille, Weakly- and semi-supervised learning of a dcnn for semantic image segmentation, arXiv preprint arXiv:1502.02734.