Fully Convolutional Networks for Semantic Segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 39, Issue 4, 2017, Pages 640-651

  Shelhamer, Evan ^a   Long, Jonathan ^a   Darrell, Trevor ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Convolutional Networks; Deep Learning; Semantic Segmentation; Transfer Learning

Indexed keywords

DEEP LEARNING; IMAGE SEGMENTATION; PIXELS; SEMANTIC WEB; SEMANTICS;

CLASSIFICATION NETWORKS; CONVOLUTIONAL NETWORKS; FINE TUNING; PREDICTION TASKS; SEMANTIC INFORMATION; SEMANTIC SEGMENTATION; TRANSFER LEARNING; VISUAL MODEL;

CONVOLUTION;

EID: 85015843890     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2016.2572683     Document Type: Article

References (62)

1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. Neural Inf. Process. Syst., 2012, pp. 1106-1114.
2. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Represent., 2015.
3. C. Szegedy, et al., "Going deeper with convolutions," in Proc. Comput. Vis. Pattern Recognit., 2015, pp. 1-9.
4. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "OverFeat: Integrated recognition, localization and detection using convolutional networks," in Proc. Int. Conf. Learn. Represent., 2014.
5. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Region-based convolutional networks for accurate object detection and segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 1, pp. 142-158, Jan. 2015.
6. K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 346-361.
7. N. Zhang, J. Donahue, R. Girshick, and T. Darrell, "Part-based R-CNNs for fine-grained category detection," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 834-849.
8. J. Long, N. Zhang, and T. Darrell, "Do convnets learn correspondence?" in Proc. Neural Inf. Process. Syst, 2014, pp. 1601-1609.
9. P. Fischer, A. Dosovitskiy, and T. Brox, "Descriptor matching with convolutional neural networks: A comparison to SIFT," arXiv preprint arXiv:1405.5769, 2014.
10. F. Ning, D. Delhomme, Y. LeCun, F. Piano, L. Bottou, and P. E. Barbano, "Toward automatic phenotyping of developing embryos from videos," IEEE Trans. Image Process., vol. 14, no. 9, pp. 1360-1371, Sep. 2005.
11. D. C. Ciresan, A. Giusti, L. M. Gambardella, and J. Schmidhuber, "Deep neural networks segment neuronal membranes in electron microscopy images," in Proc. Neural Inf. Process. Syst., 2012, pp. 2852-2860.
12. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling," Proc. IEEE Trans. Pattern Anal. Mach. Intel., vol. 35, no. 8, pp. 1915-1929, Aug. 2013.
13. P. H. Pinheiro and R. Collobert, "Recurrent convolutional neural networks for scene labeling," in Proc. 31st Int. Conf. Mach. Learn., 2014, pp. 82-90.
14. B. Hariharan, P. Arbelaez, R. Girshick, and J. Malik, "Simultaneous detection and segmentation," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 297-312.
15. S. Gupta, R. Girshick, P. Arbelaez, and J. Malik, "Learning rich features from RGB-D images for object detection and segmentation," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 345-360.
16. Y. Ganin and V. Lempitsky, "N4-fields: Neural network nearest neighbor fields for image transforms," in Proc. Asian Conf. Comput. Vis., 2014, pp. 536-551.
17. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proc. Comput. Vis. Pattern Recognit., 2015.
18. J. Donahue, et al., "DeCAF: A deep convolutional activation feature for generic visual recognition," in Proc. Int. Conf. Mach. Learn., 2014, pp. 647-655.
19. M.D.Zeiler andR. Fergus, "Visualizing andunderstanding convolutionalnetworks," inProc.Eur.Conf.Comput.Vis.,2014, pp. 818-833.
20. O. Matan, C. J. Burges, Y. LeCun, and J. S. Denker, "Multi-digit recognition using a space displacement neural network," in Proc. Neural Inf. Process. Syst., 1991, pp. 488-495.
21. Y. LeCun, et al., "Backpropagation applied to hand-written zip code recognition," in Proc. Neural Comput., 1989, pp. 541-551.
22. R. Wolf and J. C. Platt, "Postal address block location using a convolutional locator network," in Proc. Neural Inf. Process. Syst., 1994, pp. 745-745.
23. D. Eigen, D. Krishnan, and R. Fergus, "Restoring an image taken through a window covered with dirt or rain," in Proc. Int. Conf. Comput. Vis., 2013, pp. 633-640.
24. J. Tompson, A. Jain, Y. LeCun, and C. Bregler, "Joint training of a convolutional network and a graphical model for human pose estimation," in Proc. Neural Inf. Process. Syst., 2014, pp. 1799-1807.
25. D. Eigen, C. Puhrsch, and R. Fergus, "Depth map prediction from a single image using a multi-scale deep network," in Proc. Neural Inf. Process. Syst., 2014, pp. 2366-2374.
26. P. Burt and E. Adelson, "The Laplacian pyramid as a compact image code," IEEE Trans. Commun., vol. 31, no. 4, pp. 532-540, Apr. 1983.
27. J. J. Koenderink and A. J. van Doorn, "Representation of local geometry in the visual system," Biol. Cybern., vol. 55, no. 6, pp. 367-375, Mar. 1987.
28. P. Sermanet, K. Kavukcuoglu, S. Chintala, and Y. LeCun, "Pedestrian detection with unsupervised multi-stage feature learning," in Proc. Comput. Vis. Pattern Recognit., 2013, pp. 3626-3633.
29. B. Hariharan, P. Arbelaez, R. Girshick, and J. Malik, "Hypercolumns for object segmentation and fine-grained localization," in Proc. Comput. Vis. Pattern Recognit., 2015, pp. 447-456.
30. M. Mostajabi, P. Yadollahpour, and G. Shakhnarovich, "Feedforward semantic segmentation with zoom-out features," in Proc. Comput. Vis. Pattern Recognit., 2015, pp. 3376-3385.
31. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," in Proc. Neural Inf. Process. Syst., 2015, pp. 91-99.
32. S. Xie and Z. Tu, "Holistically-nested edge detection," in Proc. Int. Conf. Comput. Vis., 2015, pp. 1395-1403.
33. F. Liu, C. Shen, G. Lin, and I. Reid, "Learning depth from single monocular images using deep convolutional neural fields," IEEE Trans. Pattern Anal. Mach. Intell., 2015, Doi: 10.1109/TPAMI.2015.2505283.
34. P. Fischer, et al., "Learning optical flow with convolutional networks," in Proc. Int. Conf. Comput. Vis., 2015.
35. D. Pathak, P. Krähenbühl, and T. Darrell, "Constrained convolutional neural networks for weakly supervised segmentation," in Proc. Int. Conf. Comput. Vis., 2015, pp. 1796-1804.
36. G. Papandreou, L.-C. Chen, K. Murphy, and A. L. Yuille, "Weakly-and semi-supervised learning of a DCNN for semantic image segmentation," in Proc. Int. Conf. Comput. Vis., 2015, pp. 1742-1750.
37. J. Dai, K. He, and J. Sun, "Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation," in Proc. Int. Conf. Comput. Vis., 2015, pp. 1635-1643.
38. S. Hong, H. Noh, and B. Han, "Decoupled deep neural network for semi-supervised semantic segmentation," in Proc. Neural Inf. Process. Syst., 2015, pp. 1495-1503.
39. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, "Semantic image segmentation with deep convolutional nets and fully connected CRFs," in Proc. Int. Conf. Learn. Represent., 2015.
40. S. Zheng, et al., "Conditional random fields as recurrent neural networks," in Proc. Int. Conf. Comput. Vis., 2015, pp. 1529-1537.
41. W. Liu, A. Rabinovich, and A. C. Berg, "ParseNet: Looking wider to see better," arXiv preprint arXiv:1506.04579, 2015.
42. H. Noh, S. Hong, and B. Han, "Learning deconvolution network for semantic segmentation," in Proc. Int. Conf. Comput. Vis., 2015, pp. 1520-1528.
43. O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional networks for biomedical image segmentation," in Proc. Med. Image Comput. Comput.-Assist. Intervention, 2015, pp. 234-241.
44. F. Yu and V. Koltun, "Multi-scale context aggregation by dilated convolutions," in Proc. Int. Conf. Learn. Represent., 2016.
45. A. Giusti, D. C. Ciresan, J. Masci, L. M. Gambardella, and J. Schmidhuber, "Fast image scanning with deep max-pooling convolutional neural networks," in Proc. Int. Conf. Image Process., 2013, pp. 4034-4038.
46. M. Holschneider, R. Kronland-Martinet, J. Morlet, and P. Tchamitchian, "A real-time algorithm for signal analysis with the help of the wavelet transform," in Proc. Int. Conf. Time-Freq. Methods Phase Space, 1989, pp. 286-297.
47. S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. New York, NY, USA: Academic, 1999.
48. P. P. Vaidyanathan, "Multirate digital filters, filter banks, polyphase networks, and applications: A tutorial," Proc. IEEE, vol. 78, no. 1, pp. 56-93, Jan. 1990.
49. L. Wan, M. Zeiler, S. Zhang, Y. L. Cun, and R. Fergus, "Regularization of neural networks using DropConnect," in Proc. Int. Conf. Mach. Learn., 2013, pp. 1058-1066.
50. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, The PASCAL Visual Object Classes Challenge 2011 Results. [Online]. Available: http://www.pascal-network.org/challenges/VOC/voc2011/workshop/index.html.
51. C. M. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: Springer-Verlag, 2006, p. 229.
52. B. Hariharan, P. Arbelaez, L. Bourdev, S. Maji, and J. Malik, "Semantic contours from inverse detectors," in Proc. Int. Conf. Comput. Vis., 2011, pp. 991-998.
53. Y. A. LeCun, L. Bottou, G. B. Orr, and K.-R. Müller, "Efficient backprop," in Neural Networks: Tricks of the Trade. Berlin, Germany: Springer, 1998, pp. 9-48.
54. Y. Jia, et al., "Caffe: Convolutional architecture for fast feature embedding," arXiv preprint arXiv:1408.5093, 2014.
55. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor segmentation and support inference from RGBD images," in Proc. Eur. Conf. Comput. Vis., 2012, pp. 746-760.
56. S. Gupta, P. Arbelaez, and J. Malik, "Perceptual organization and recognition of indoor scenes from RGB-D images," in Proc. Comput. Vis. Pattern Recognit., 2013, pp. 564-571.
57. C. Liu, J. Yuen, and A. Torralba, "Sift flow: Dense correspondence across scenes and its applications," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 978-994, May 2011.
58. J. Tighe and S. Lazebnik, "Superparsing: scalable nonparametric image parsing with superpixels," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 352-365.
59. J. Tighe and S. Lazebnik, "Finding things: Image parsing with regions and per-exemplar detectors," in Proc. Comput. Vis. Pattern Recognit., 2013, pp. 3001-3008.
60. J. Dai, K. He, and J. Sun, "Convolutional feature masking for joint object and stuff segmentation," in Proc. Comput. Vis. Pattern Recognit., 2015, pp. 3992-4000.
61. J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu, "Semantic segmentation with second-order pooling," in Proc. Eur. Conf. Comput. Vis., 2012 pp. 430-443.
62. R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille, "The role of context for object detection and semantic segmentation in the wild," in IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 891-898.