The Cityscapes Dataset for Semantic Urban Scene Understanding

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 3213-3223

  Cordts, Marius ^a,b   Omran, Mohamed ^c   Ramos, Sebastian ^a,d   Rehfeld, Timo ^a,b   Enzweiler, Markus ^a   Benenson, Rodrigo ^c   Franke, Uwe ^a   Roth, Stefan ^b   Schiele, Bernt ^c  

^a DAIMLER AG   (Germany)

^b DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^c MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^d DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; COMPUTER VISION; PIXELS; SEMANTICS; STATISTICAL TESTS; STEREO IMAGE PROCESSING;

BENCHMARK SUITES; EMPIRICAL STUDIES; IN-DEPTH ANALYSIS; LARGE-SCALE DATASET; LARGE-SCALE DATASETS; SEMANTIC LABELING; STATE-OF-THE-ART APPROACH; STEREO VIDEO SEQUENCES;

PATTERN RECOGNITION;

EID: 84986255616     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.350     Document Type: Conference Paper

References (82)

1. P. Arbelaez, J. Pont-Tuset, J. Barron, F. Marqués, and J. Malik. Multiscale combinatorial grouping. In CVPR, 2014.
2. S. Ardeshir, K. M. Collins-Sibley, and M. Shah. Geosemantic segmentation. In CVPR, 2015.
3. A. Arnab, S. Jayasumana, S. Zheng, and P. H. S. Torr. Higher order conditional random fields in deep neural networks. arXiv:1511.08119v3 [cs.CV], 2015.
4. V. Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv:1511.00561v2 [cs.CV], 2015.
5. A. Bearman, O. Russakovsky, V. Ferrari, and L. Fei-Fei. What's the point: Semantic segmentation with point supervision. arXiv:1506.02106v4 [cs.CV], 2015.
6. R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at 100 frames per second. In CVPR, 2012.
7. G. J. Brostow, J. Fauqueur, and R. Cipolla. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters, 30(2):88-97, 2009.
8. W. Byeon, T. M. Breuel, F. Raue, and M. Liwicki. Scene Labeling with LSTM Recurrent Neural Networks. In CVPR, 2015.
9. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR, 2015.
10. Y.-T. Chen, X. Liu, and M.-H. Yang. Multi-instance object segmentation with occlusion handling. In CVPR, 2015.
11. J. Dai, K. He, and J. Sun. Convolutional feature masking for joint object and stuff segmentation. In CVPR, 2015.
12. P. Dollár, C. Wojek, B. Schiele, and P. Perona. Pedestrian detection: An evaluation of the state of the art. Trans. PAMI, 34(4):743-761, 2012.
13. M. Enzweiler and D. M. Gavrila. Monocular pedestrian detection: Survey and experiments. 31(12):2179-2195, 2009.
14. M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL visual object classes challenge: A retrospective. IJCV, 111(1):98-136, 2015.
15. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained partbased models. Trans. PAMI, 32(9):1627-1645, 2010.
16. U. Franke, D. Pfeiffer, C. Rabe, C. Knöppel, M. Enzweiler, F. Stein, and R. G. Herrtwich. Making Bertha see. In ICCV Workshops, 2013.
17. P. Furgale, U. Schwesinger, M. Rufli, W. Derendarz, H. Grimmett, P. Mühlfellner, S. Wonneberger, B. Li, et al. Toward automated driving in cities using close-to-market sensors: An overview of the V-Charge project. In IV Symposium, 2013.
18. A. Geiger, M. Lauer, C.Wojek, C. Stiller, and R. Urtasun. 3D traffic scene understanding from movable platforms. Trans. PAMI, 36(5):1012-1025, 2014.
19. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The KITTI dataset. IJRR, 32(11), 2013.
20. R. Girshick. Fast R-CNN. In ICCV, 2015.
21. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014.
22. F. Gueney and A. Geiger. Displets: Resolving stereo ambiguities using object knowledge. In CVPR, 2015.
23. B. Hariharan, P. Arbeláez, R. B. Girshick, and J. Malik. Simultaneous detection and segmentation. In ECCV, 2014.
24. B. Hariharan, P. A. Arbeláez, R. B. Girshick, and J. Malik. Hypercolumns for object segmentation and fine-grained localization. In CVPR, 2015.
25. H. Hattori, V. N. Boddeti, K. M. Kitani, and T. Kanade. Learning scene-specific pedestrian detectors without real data. In CVPR, 2015.
26. X. He and S. Gould. An exemplar-based CRF for multiinstance object segmentation. In CVPR, 2014.
27. D. Hoiem, J. Hays, J. Xiao, and A. Khosla. Guest editorial: Scene understanding. IJCV, 2015.
28. J. Hosang, R. Benenson, P. Dollár, and B. Schiele. What makes for effective detection proposals? Trans. PAMI, 38(4):814-830, 2015.
29. H. Hu and B. Upcroft. Nonparametric semantic segmentation for 3D street scenes. In IROS, 2013.
30. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, 2012.
31. L. Krüger, C. Wöhler, A. Würz-Wessel, and F. Stein. Infactory calibration of multiocular camera systems. In SPIE Photonics Europe (Optical Metrology in Production Engineering), 2004.
32. A. Kundu, Y. Li, F. Dellaert, F. Li, and J. Rehg. Joint semantic segmentation and 3D reconstruction from monocular video. In ECCV, 2014.
33. L. Ladicky, J. Shi, and M. Pollefeys. Pulling things out of perspective. In CVPR, 2014.
34. Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 2015.
35. B. Leibe, N. Cornelis, K. Cornelis, and L. Van Gool. Dynamic 3D scene analysis from a moving vehicle. In CVPR, 2007.
36. B. Leibe, A. Leonardis, and B. Schiele. Robust object detection with interleaved categorization and segmentation. IJCV, 77(1-3):259-289, 2008.
37. G. Lin, C. Shen, A. van den Hengel, and I. Reid. Efficient piecewise training of deep structured models for semantic segmentation. In CVPR, 2016, to appear.
38. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.
39. W. Liu, A. Rabinovich, and A. C. Berg. Parsenet: Looking wider to see better. arXiv:1506.04579v2 [cs.CV], 2015.
40. Z. Liu, X. Li, P. Luo, C. C. Loy, and X. Tang. Semantic image segmentation via deep parsing network. In ICCV, 2015.
41. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
42. M. Maire, S. X. Yu, and P. Perona. Object detection and segmentation from joint embedding of parts and pixels. In ICCV, 2011.
43. I. Misra, A. Shrivastava, and M. Hebert. Watch and learn: Semi-supervised learning for object detectors from video. In CVPR, 2015.
44. M. Mostajabi, P. Yadollahpour, and G. Shakhnarovich. Feedforward semantic segmentation with zoom-out features. In CVPR, 2015.
45. R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille. The role of context for object detection and semantic segmentation in the wild. In CVPR, 2014.
46. A. Oliva and A. Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV, 42(3):145-175, 2001.
47. M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Is object localization for free? Weakly-supervised learning with convolutional neural networks. In CVPR, 2015.
48. G. Papandreou, L.-C. Chen, K. Murphy, and A. L. Yuille. Weakly-and semi-supervised learning of a DCNN for semantic image segmentation. In ICCV, 2015.
49. D. Pathak, P. Kraehenbuehl, and T. Darrell. Constrained convolutional neural networks for weakly supervised segmentation. In ICCV, 2015.
50. D. Pathak, E. Shelhamer, J. Long, and T. Darrell. Fully convolutional multi-class multiple instance learning. In ICLR, 2015.
51. D. Pfeiffer, S. K. Gehrig, and N. Schneider. Exploiting the power of stereo confidences. In CVPR, 2013.
52. P. H. Pinheiro and R. Collobert. Recurrent convolutional neural networks for scene parsing. In ICML, 2014.
53. P. H. Pinheiro and R. Collobert. From image-level to pixellevel labeling with convolutional networks. In CVPR, 2015.
54. J. Pont-Tuset and L. Van Gool. Boosting object proposals: From Pascal to COCO. In ICCV, 2015.
55. S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015.
56. H. Riemenschneider, A. Bódis-Szomorú, J. Weissenberg, and L. Van Gool. Learning where to classify in multi-view semantic segmentation. In ECCV. 2014.
57. H. Riemenschneider, S. Sternig, M. Donoser, P. M. Roth, and H. Bischof. Hough regions for joining instance localization and segmentation. In ECCV, 2012.
58. G. Ros, S. Ramos, M. Granados, D. Vazquez, and A. M. Lopez. Vision-based offline-online perception paradigm for autonomous driving. In WACV, 2015.
59. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recognition challenge. IJCV, 115(3):211-252, 2015.
60. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. LabelMe: A database and web-based tool for image annotation. IJCV, 77(1-3):157-173, 2008.
61. T. Scharwächter, M. Enzweiler, U. Franke, and S. Roth. Efficient multi-cue scene segmentation. In GCPR, 2013.
62. T. Scharwächter, M. Enzweiler, U. Franke, and S. Roth. Stixmantics: A medium-level model for real-time semantic scene understanding. In ECCV, 2014.
63. A. Schwing and R. Urtasun. Fully connected deep structured networks. arXiv:1503.02351v1 [cs.CV], 2015.
64. S. Sengupta, E. Greveson, A. Shahrokni, and P. H. S. Torr. Semantic modelling of urban scenes. In ICRA, 2013.
65. S. Sengupta, P. Sturgess, L. Ladicky, and P. H. S. Torr. Automatic dense visual semantic mapping from street-level imagery. In IROS, 2012.
66. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. OverFeat: Integrated recognition, localization and detection using convolutional networks. In ICLR, 2014.
67. A. Sharma, O. Tuzel, and D. W. Jacobs. Deep hierarchical parsing for semantic segmentation. In CVPR, 2015.
68. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556v6 [cs.CV], 2014.
69. S. Song, S. P. Lichtenberg, and J. Xiao. Sun RGB-D: A RGB-D scene understanding benchmark suite. In CVPR, 2015.
70. J. Tighe and S. Lazebnik. Superparsing. IJCV, 101(2):329-349, 2013.
71. J. Tighe, M. Niethammer, and S. Lazebnik. Scene parsing with object instance inference using regions and perexemplar detectors. IJCV, 112(2):150-171, 2015.
72. J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A.W. M. Smeulders. Selective search for object recognition. International journal of computer vision, 104(2):154-171, 2013.
73. V. Vineet, O. Miksik, M. Lidegaard, M. Niessner, S. Golodetz, V. A. Prisacariu, O. Kahler, D. W. Murray, S. Izadi, P. Perez, and P. H. S. Torr. Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction. In ICRA, 2015.
74. Y. Wei, X. Liang, Y. Chen, X. Shen, M.-M. Cheng, Y. Zhao, and S. Yan. STC: A simple to complex framework for weakly-supervised semantic segmentation. arXiv:1509.03150v1 [cs.CV], 2015.
75. J. Xie, M. Kiefel, M.-T. Sun, and A. Geiger. Semantic instance annotation of street scenes by 3D to 2D label transfer. In CVPR, 2016, to appear.
76. J. Xu, A. G. Schwing, and R. Urtasun. Learning to segment under various forms of weak supervision. In CVPR, 2015.
77. P. Xu, F. Davoine, J.-B. Bordes, H. Zhao, and T. Denoeux. Information fusion on oversegmented images: An application for urban scene understanding. In MVA, 2013.
78. J. Yao, S. Fidler, and R. Urtasun. Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation. In CVPR, 2012.
79. F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016, to appear.
80. Z. Zhang, A. Schwing, S. Fidler, and R. Urtasun. Monocular object instance segmentation and depth ordering with CNNs. In ICCV, 2015.
81. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr. Conditional random fields as recurrent neural networks. In ICCV, 2015.
82. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In NIPS, 2014.