Unsupervised Joint Feature Learning and Encoding for RGB-D Scene Labeling

IEEE Transactions on Image Processing

Volumn 24, Issue 11, 2015, Pages 4459-4473

  Wang, Anran ^a   Lu, Jiwen ^b   Cai, Jianfei ^a   Wang, Gang ^c   Cham, Tat Jen ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b TSINGHUA UNIVERSITY   (China)

^c NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

joint feature learning and encoding; multi modality; RGB D scene labeling; unsupervised feature learning

Indexed keywords

BENCHMARKING;

COMPETITIVE PERFORMANCE; DEEP FEATURE LEARNING; FEATURE COMBINATION; FEATURE LEARNING; INTRINSIC CHARACTERISTICS; MULTI MODALITY; STATE-OF-THE-ART METHODS; UNSUPERVISED FEATURE LEARNING;

ENCODING (SYMBOLS);

EID: 84939824028     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2465133     Document Type: Article

References (43)

1. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011.
2. C. Cadena and J. Košecka, "Semantic parsing for priming object detection in RGB-D scenes, " in Proc. 3rd Workshop Semantic Perception, Mapping Exploration, 2013, pp. 1-6.
3. A. Coates, A. Y. Ng, and H. Lee, "An analysis of single-layer networks in unsupervised feature learning, " in Proc. Int. Conf. Artif. Intell. Statist., 2011, pp. 215-223.
4. C. Couprie, C. Farabet, L. Najman, and Y. LeCun. (2013). "Indoor semantic segmentation using depth information." [Online]. Available: http://arxiv.org/abs/1301.3572
5. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection, " in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2005, pp. 886-893.
6. D. Eigen and R. Fergus. (2014). "Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture." [Online]. Available: http://arxiv.org/abs/1411.4734
7. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1915-1929, Aug. 2013.
8. C. Galleguillos, A. Rabinovich, and S. Belongie, "Object categorization using co-occurrence, location and appearance, " in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1-8.
9. S. Gould, R. Fulton, and D. Koller, "Decomposing a scene into geometric and semantically consistent regions, " in Proc. IEEE 12th ICCV, Sep./Oct. 2009, pp. 1-8.
10. D. Grangier, L. Bottou, and R. Collobert, "Deep convolutional networks for scene parsing, " in Proc. ICML Deep Learn. Workshop, 2009, pp. 1-2.
11. S. Gupta, P. Arbelaez, and J. Malik, "Perceptual organization and recognition of indoor scenes from RGB-D images, " in Proc. IEEE Conf. CVPR, Jun. 2013, pp. 564-571.
12. S. Gupta, R. Girshick, P. Arbeláez, and J. Malik, "Learning rich features from RGB-D images for object detection and segmentation, " in Proc. 13th ECCV, 2014, pp. 345-360.
13. X. He, R. S. Zemel, and M. A. Carreira-Perpiñán, "Multiscale conditional random fields for image labeling, " in Proc. IEEE Comput. Soc. Conf. CVPR, Jun/Jul. 2004, pp. II-695-II-702.
14. A. Hermans, G. Floros, and B. Leibe, "Dense 3D semantic mapping of indoor scenes from RGB-D images, " in Proc. IEEE ICRA, May/Jun. 2014, pp. 2631-2638.
15. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets, " Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
16. I.-H. Jhuo, S. Gao, L. Zhuang, D. Lee, and Y. Ma, "Unsupervised feature learning for RGB-D image classification, " in Proc. 12th ACCV, Nov. 2014, pp. 276-289.
17. S. H. Khan, M. Bennamoun, F. Sohel, and R. Togneri, "Geometry driven semantic labeling of indoor scenes, " in Proc. 13th ECCV, 2014, pp. 679-694.
18. H. S. Koppula, A. Anand, T. Joachims, and A. Saxena, "Semantic labeling of 3D point clouds for indoor scenes, " in Proc. Adv. NIPS, 2011, pp. 244-252.
19. A. Kumar, P. Rai, and H. Daumé, III, "Co-regularized multi-view spectral clustering, " in Proc. Adv. NIPS, 2011, pp. 1413-1421.
20. K. Lai, L. Bo, and D. Fox, "Unsupervised feature learning for 3D scene labeling, " in Proc. IEEE ICRA, May/Jun. 2014, pp. 3050-3057.
21. Q. V. Le, A. Karpenko, J. Ngiam, and A. Y. Ng, "ICA with reconstruction cost for efficient overcomplete feature learning, " in Proc. Adv. NIPS, 2011, pp. 1017-1025.
22. Q. V. Le, W. Y. Zou, S. Y. Yeung, and A. Y. Ng, "Learning hierarchical invariant spatio-Temporal features for action recognition with independent subspace analysis, " in Proc. IEEE Conf. CVPR, Jun. 2011, pp. 3361-3368.
23. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms, " in Proc. Adv. NIPS, 2006, pp. 801-808.
24. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, " in Proc. 26th Annu. ICML, 2009, pp. 609-616.
25. V. Lempitsky, A. Vedaldi, and A. Zisserman, "Pylon model for semantic segmentation, " in Proc. Adv. NIPS, 2011, pp. 1485-1493.
26. C. Liu, J. Yuen, and A. Torralba, "SIFT flow: Dense correspondence across scenes and its applications, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 978-994, May 2011.
27. D. G. Lowe, "Object recognition from local scale-invariant features, " in Proc. 7th IEEE ICCV, Sep. 1999, pp. 1150-1157.
28. B. Micusik and J. Kosecka, "Semantic segmentation of street scenes by superpixel co-occurrence and 3D geometry, " in Proc. IEEE 12th ICCV Workshops, Sep./Oct. 2009, pp. 625-632.
29. A. C. Möller and S. Behnke, "Learning depth-sensitive conditional random fields for semantic segmentation of RGB-D images, " in Proc. IEEE ICRA, May/Jun. 2014, pp. 6232-6237.
30. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, "Multimodal deep learning, " in Proc. 28th ICML, 2011, pp. 689-696.
31. D. Pei, H. Liu, Y. Liu, and F. Sun, "Unsupervised multimodal feature learning for semantic image segmentation, " in Proc. IJCNN, Aug. 2013, pp. 1-6.
32. G. Potamianos, C. Neti, J. Luettin, and I. Matthews, "Audio-visual automatic speech recognition: An overview, " in Issues in Visual and Audio-Visual Speech Processing, vol. 22. Cambridge, MA, USA: MIT Press, 2004, p. 23.
33. X. Ren, L. Bo, and D. Fox, "RGB-(D) scene labeling: Features and algorithms, " in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 2759-2766.
34. M. Schimidt. (2005). minFunc. [Online]. Available: http://www.cs.ubc. ca/~schmidtm/Software/minFunc.html
35. J. Shotton, J. Winn, C. Rother, and A. Criminisi, "TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation, " in Proc. 9th ECCV, 2006, pp. 1-15.
36. N. Silberman and R. Fergus, "Indoor scene segmentation using a structured light sensor, " in Proc. IEEE ICCV Workshops, Nov. 2011, pp. 601-608.
37. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor segmentation and support inference from RGBD images, " in Proc. 12th ECCV, 2012, pp. 746-760.
38. R. Socher, B. Huval, B. Bath, C. D. Manning, and A. Y. Ng, "Convolutional-recursive deep learning for 3D object classification, " in Proc. Adv. NIPS, 2012, pp. 665-673.
39. R. Socher, C. C. Lin, C. Manning, and A. Y. Ng, "Parsing natural scenes and natural language with recursive neural networks, " in Proc. 28th ICML, 2011, pp. 129-136.
40. A. Wang, J. Lu, G. Wang, J. Cai, and T.-J. Cham, "Multi-modal unsupervised feature learning for RGB-D scene labeling, " in Proc. 13th ECCV, 2014, pp. 453-467.
41. H. Wang, M. M. Ullah, A. Klaser, I. Laptev, and C. Schmid, "Evaluation of local spatio-Temporal features for action recognition, " in Proc. BMVC, 2009, pp. 124.1-124.11.
42. Y. Xie, J. Ho, and B. Vemuri, "On a nonlinear generalization of sparse coding and dictionary learning, " in Proc. ICML, 2013, pp. 1480-1488.
43. J. Yang, K. Yu, Y. Gong, and T. Huang, "Linear spatial pyramid matching using sparse coding for image classification, " in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 1794-1801.