A Joint Convolutional Neural Networks and Context Transfer for Street Scenes Labeling

IEEE Transactions on Intelligent Transportation Systems

Volumn 19, Issue 5, 2018, Pages 1457-1470

  Wang, Qi ^a,b   Gao, Junyu ^b   Yuan, Yuan ^b  

^a Unmanned System Research Institute   (China)

^b NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

Author keywords

convolutional neural networks; data augmentation; deep learning; label transfer; Scene labeling; street scenes

Indexed keywords

CONVOLUTION; DEEP LEARNING; FEATURE EXTRACTION; LABELING; NEURAL NETWORKS; PERSONNEL TRAINING; PIXELS;

COMPETITIVE PERFORMANCE; COMPUTATIONAL MODEL; CONVOLUTIONAL NEURAL NETWORK; DATA AUGMENTATION; IMAGE REPRESENTATIONS; LOCATION INFORMATION; STREET SCENE UNDERSTANDING; THREE-DIMENSIONAL DISPLAY;

DATA MINING;

EID: 85028507989     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2017.2726546     Document Type: Article

References (41)

1. S. Zhang, C. Bauckhage, and A. B. Cremers, "Efficient pedestrian detection via rectangular features based on a statistical shape model," IEEE Trans. Intell. Transp. Syst., vol. 16, no. 2, pp. 763-775, Apr. 2015.
2. Y. Yuan, D. Wang, and Q.Wang, "Anomaly detection in traffic scenes via spatial-aware motion reconstruction," IEEE Trans. Intell. Transp. Syst., vol. 18, no. 5, pp. 1198-1209, May 2017.
3. T. Chen, R. Wang, B. Dai, D. Liu, and J. Song, "Likelihood-field-modelbased dynamic vehicle detection and tracking for self-driving," IEEE Trans. Intell. Transp. Syst., vol. 17, no. 11, pp. 3142-3158, Nov. 2016.
4. Q. Wang, J. Fang, and Y. Yuan, "Adaptive road detection via contextaware label transfer," Neurocomputing, vol. 158, pp. 174-183, Jun. 2015.
5. Y. Na, Y. Guo, Q. Fu, and Y. Yan, "Cross array and rank-1 MUSIC algorithm for acoustic highway lane detection," IEEE Trans. Intell. Transp. Syst., vol. 17, no. 9, pp. 2502-2514, Sep. 2016.
6. W. A. Wright, "Image labelling with a neural network," in Proc. Alvey Vis. Conf., 1989, pp. 1-6.
7. F. Nie, J. Li, and X. Li, "Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification," in Proc. IJCAI, 2016, pp. 1881-1887.
8. J. Shotton, J. Winn, C. Rother, and A. Criminisi, "TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation," in Proc. Eur. Conf. Comput. Vis., 2006, pp. 1-15.
9. J. Tighe and S. Lazebnik, "Superparsing-Scalable nonparametric image parsing with superpixels," Int. J. Comput. Vis., vol. 101, no. 2, pp. 329-349, 2013.
10. X. Li, B. Zhao, and X. Lu, "A general framework for edited video and raw video summarization," IEEE Trans. Image Process., vol. 26, no. 8, pp. 3652-3664, Aug. 2017.
11. G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, "Segmentation and recognition using structure from motion point clouds," in Proc. Eur. Conf. Comput. Vis., 2008, pp. 44-57.
12. S. Sengupta, E. Greveson, A. Shahrokni, and P. H. S. Torr, "Urban 3D semantic modelling using stereo vision," in Proc. IEEE Int. Conf. Robot. Autom., May 2013, pp. 580-585.
13. C. Liu, J. Yuen, and A. Torralba, "Nonparametric scene parsing: Label transfer via dense scene alignment," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1972-1979.
14. J. Xiao and L. Quan, "Multiple view semantic segmentation for street view images," in Proc. Int. Conf. Comput. Vis., 2009, pp. 686-693.
15. C. Zhang, L. Wang, and R. Yang, "Semantic segmentation of urban scenes using dense depth maps," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 708-721.
16. X. Peng, J. Lu, Z. Yi, and R. Yan. (2014). "Automatic subspace learning via principal coefficients embedding." [Online]. Available: https://arxiv.org/abs/1411.4419
17. D. Grangier, L. Bottou, and R. Collobert, "Deep convolutional networks for scene parsing," in Proc. Int. Conf. Mach. Learn. Deep Learn. Workshop, vol. 3. 2009, pp. 1-8.
18. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1915-1929, Aug. 2013.
19. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 580-587.
20. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1106-1114.
21. B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik, "Simultaneous detection and segmentation," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 297-312.
22. E. Shelhamer, J. Long, and T. Darrell, "Fully convolutional networks for semantic segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 640-651, Apr. 2017.
23. K. Simonyan and A. Zisserman. (2014). "Very deep convolutional networks for large-scale image recognition." [Online]. Available: https://arxiv.org/abs/1409.1556
24. C. Szegedy et al. (2014). "Going deeper with convolutions." [Online]. Available: https://arxiv.org/abs/1409.4842
25. L. Ladický, C. Russell, P. Kohli, and P. H. S. Torr, "Associative hierarchical CRFs for object class image segmentation," in Proc. IEEE Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 739-746.
26. H. Myeong, J. Y. Chang, and K. M. Lee, "Learning object relationships via graph-based context model," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 2727-2734.
27. H. Myeong and K. M. Lee, "Tensor-based high-order semantic relation transfer for semantic scene segmentation," in Proc. Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 3073-3080.
28. J. Yang, B. Price, S. Cohen, and M.-H. Yang, "Context driven scene parsing with attention to rare classes," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 3294-3301.
29. A. Sharma, O. Tuzel, and M.-Y. Liu, "Recursive context propagation network for semantic scene labeling," in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 2447-2455.
30. P. Kontschieder, S. R. Bulo, M. Pelillo, and H. Bischof, "Structured labels in random forests for semantic labelling and object detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 10, pp. 2104-2116, Oct. 2014.
31. X. Peng, S. Xiao, J. Feng, W.-Y. Yau, and Z. Yi, "Deep subspace clustering with sparsity prior," in Proc. 25th Int. Joint Conf. Artif. Intell., 2016, pp. 1925-1931.
32. H. He and E. A. Garcia, "Learning from imbalanced data," IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263-1284, Sep. 2009.
33. A. G. Howard. (2013). "Some improvements on deep convolutional neural network based image classification." [Online]. Available: https://arxiv.org/abs/1312.5402
34. R. Wu, S. Yan, Y. Shan, Q. Dang, and G. Sun. (2015). "Deep image: Scaling up image recognition." [Online]. Available: https://arxiv.org/abs/1501.02876
35. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, "SLIC superpixels compared to state-of-The-art superpixel methods," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274-2282, Nov. 2012.
36. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva, "Learning deep features for scene recognition using places database," in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 487-495.
37. Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
38. V. Kolmogorov and R. Zabin, "What energy functions can be minimized via graph cuts?" IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 2, pp. 147-159, Feb. 2004.
39. Y. Boykov and V. Kolmogorov, "An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, pp. 1124-1137, Sep. 2004.
40. Y. Yang, Z. Li, L. Zhang, C. Murphy, J. V. Hoeve, and H. Jiang, "Local label descriptor for example based semantic image labeling," in Proc. Eur. Conf. Comput. Vis., 2012, pp. 361-375.
41. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, "LabelMe: A database and Web-based tool for image annotation," Int. J. Comput. Vis., vol. 77, nos. 1-3, pp. 157-173, 2008.