STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 39, Issue 11, 2017, Pages 2314-2320

  Wei, Yunchao ^a   Liang, Xiaodan ^b   Chen, Yunpeng ^e   Shen, Xiaohui ^c   Cheng, Ming Ming ^d   Feng, Jiashi ^e   Zhao, Yao ^a   Yan, Shuicheng ^e  

^a BEIJING JIAOTONG UNIVERSITY   (China)

^b SUN YAT SEN UNIVERSITY   (China)

^c ADOBE RESEARCH   (United States)

^d NANKAI UNIVERSITY   (China)

^e NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

convolutional neural network; Semantic segmentation; weakly supervised learning

Indexed keywords

COMPLEX NETWORKS; CONVOLUTION; DEEP NEURAL NETWORKS; IMAGE ENHANCEMENT; NEURAL NETWORKS; OBJECT DETECTION; PIXELS; SEMANTICS;

CLUTTERED BACKGROUNDS; CONVOLUTIONAL NEURAL NETWORK; INITIAL SEGMENTATION; SALIENT OBJECT DETECTION; SEGMENTATION MASKS; SEMANTIC SEGMENTATION; STATE OF THE ART; WEAKLY SUPERVISED LEARNING;

IMAGE SEGMENTATION;

EID: 85032268688     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2016.2636150     Document Type: Article

References (47)

1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. Advances Neural Inf. Process. Syst., 2012, pp. 1097-1105.
2. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Representations, 2015.
3. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," IEEE CVPR, pp. 1-9, 2015.
4. Y. Wei, et al., "HCP: A flexible CNN framework for multi-label image classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 9, pp. 1901-1907, Sep. 2016.
5. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 580-587.
6. R. Girshick, "Fast R-CNN," IEEE ICCV, pp. 1440-1448, 2015.
7. W. Xia, C. Domokos, J. Dong, L.-F. Cheong, and S. Yan, "Semantic segmentation without annotating segments," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2176-2183.
8. J. Dai, K. He, and J. Sun, "BoxSup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation," IEEE ICCV, pp. 1635-1643, 2015.
9. B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik, "Simultaneous detection and segmentation," in Proc. 13th Eur. Conf. Comput. Vis., 2014, pp. 297-312.
10. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. Torr, "Conditional random fields as recurrent neural networks," IEEE ICCV, pp. 1529-1537, 2015.
11. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 3431-3440.
12. J. Dai, K. He, and J. Sun, "Convolutional feature masking for joint object and stuff segmentation," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 3992-4000.
13. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, "Semantic image segmentation with deep convolutional nets and fully connected CRFs," arXiv:1412.7062, 2014.
14. G. Papandreou, L.-C. Chen, K. Murphy, and A. L. Yuille, "Weakly- and semi-supervised learning of a DCNN for semantic image segmentation," IEEE ICCV, pp. 1742-1750, 2015.
15. P. O. Pinheiro and R. Collobert, "From image-level to pixel-level labeling with convolutional networks," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1713-1721.
16. D. Pathak, E. Shelhamer, J. Long, and T. Darrell, "Fully convolutional multi-class multiple instance learning," arXiv:1412.7144, 2014.
17. J. Xu, A. G. Schwing, and R. Urtasun, "Learning to segment under various forms of weak supervision," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 3781-3790.
18. D. Pathak, P. Krähenbühl, and T. Darrell, "Constrained convolutional neural networks for weakly supervised segmentation," IEEE ICCV, pp. 1796-1804, 2015.
19. A. Borji, M.-M. Cheng, H. Jiang, and J. Li, "Salient object detection: A benchmark," IEEE Trans. Image Process., vol. 24, no. 12, pp. 5706-5722, Dec. 2015.
20. M. Cheng, N. J. Mitra, X. Huang, P. H. Torr, and S. Hu, "Global contrast based salient region detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569-582, Mar. 2015.
21. J. Shi, Q. Yan, L. Xu, and J. Jia, "Hierarchical image saliency detection on extended CSSD," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 4, pp. 717-729, Apr. 2016.
22. H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, "Salient object detection: A discriminative regional feature integration approach," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2083-2090.
23. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
24. M. Everingham, S. A. Eslami, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, "The pascal visual object classes challenge: A retrospective," Int. J. Comput. Vis., vol. 111, no. 1, pp. 98-136, 2014.
25. T.-Y. Lin, et al., "Microsoft COCO: Common objects in context," in Proc. 13th Eur. Conf. Comput. Vis., 2014, pp. 740-755.
26. W. Shimoda and K. Yanai, "Distinct class-specific saliency maps for weakly supervised semantic segmentation," in Proc. 14th Eur. Conf. Comput. Vis., 2016, pp. 218-234.
27. F. Saleh, M. S. A. Akbarian, M. Salzmann, L. Petersson, S. Gould, and J. M. Alvarez, "Built-in foreground/background prior for weakly-supervised semantic segmentation," in Proc. 14th Eur. Conf. Comput. Vis., 2016, pp. 413-432.
28. A. Kolesnikov and C. H. Lampert, "Seed, expand and constrain: Three principles for weakly-supervised image segmentation," in Proc. 14th Eur. Conf. Comput. Vis., 2016, pp. 695-711.
29. O. Maron and T. Lozano-Pérez, "A framework for multiple-instance learning," in Proc. Conf. Advances Neural Inf. Process. Syst., 1998, pp. 570-576.
30. M. P. Kumar, B. Packer, and D. Koller, "Self-paced learning for latent variable models," in Proc. 23rd Int. Conf. Neural Inf. Processing Syst., 2010, pp. 1189-1197.
31. K. Tang, V. Ramanathan, L. Fei-Fei, and D. Koller, "Shifting weights: Adapting object detectors from image to video," in Proc. Advances Int. Conf. Neural Inf. Processing Syst., 2012, pp. 638-646.
32. L. Jiang, D. Meng, S.-I. Yu, Z. Lan, S. Shan, and A. Hauptmann, "Self-paced learning with diversity," in Proc. Advances Int. Conf. Neural Inf. Processing Syst., 2014, pp. 2078-2086.
33. X. Liang, S. Liu, Y. Wei, L. Liu, L. Lin, and S. Yan, "Towards computational baby learning: A weakly-supervised approach for object detection," in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 999-1007.
34. J. Xu, A. G. Schwing, and R. Urtasun, "Tell me what you see and I will show youwhere it is," in Proc. IEEE Int. Conf. Comput. Vis., 2014, pp. 3190-3197.
35. S. Liu, S. Yan, T. Zhang, C. Xu, J. Liu, and H. Lu, "Weakly supervised graph propagation towards collective image parsing," IEEE Trans. Multimedia, vol. 14, no. 2, pp. 361-373, Apr. 2012.
36. M. Rubinstein, C. Liu, and W. T. Freeman, "Annotation propagation in large image databases via dense image correspondence," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 85-99.
37. A. Vezhnevets, V. Ferrari, and J. M. Buhmann, "Weakly supervised structured output learning for semantic segmentation," in Proc. IEEE Int. Conf. Comput. Vis., 2012, pp. 845-852.
38. J. Shotton, J. Winn, C. Rother, and A. Criminisi, "Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context," Int. J. Comput. Vis., vol. 81, no. 1, pp. 2-23, 2009.
39. C. Liu, J. Yuen, and A. Torralba, "Nonparametric scene parsing: Label transfer via dense scene alignment," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 1972-1979.
40. T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu, "Sketch2photo: Internet image montage," ACM Trans. Graph., vol. 28, no. 5, 2009, Art. no. 124.
41. M.-M. Cheng, N. J. Mitra, X. Huang, and S.-M. Hu, "Salient Shape: Group saliency in image collections," Vis. Comput., vol. 30, no. 4, pp. 443-453, 2014.
42. B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji, and J. Malik, "Semantic contours from inverse detectors," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 991-998.
43. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Image Net: A large-scale hierarchical image database," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 248-255.
44. Y. Jia, et al., "Caffe: Convolutional architecture for fast feature embedding," in Proc. 22nd ACM Int. Conf. Multimedia, 2014, pp. 675-678.
45. M.-M. Cheng, Z. Zhang, W.-Y. Lin, and P. Torr, "Bing: Binarized normed gradients for objectness estimation at 300fps," in Proc. IEEE Int. Conf. Comput. Vis., 2014, pp. 3286-3293.
46. P. Arbelaez, J. Pont-Tuset, J. Barron, F. Marques, and J. Malik, "Multiscale combinatorial grouping," in Proc. IEEE Int. Conf. Comput. Vis., 2014, pp. 328-335.
47. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Over Feat: Integrated recognition, localization and detection using convolutional networks," in Proc. Int. Conf. Learn. Representations, 2014.