Video propagation networks

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 3154-3164

  Jampani, Varun ^a   Gadde, Raghudeep ^a,b   Gehler, Peter V ^a,b  

^a MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

^b Bernstein Center for Computational Neuroscience   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE FILTERING; ADAPTIVE FILTERS; COMPUTER VISION; IMAGE SEGMENTATION; SEMANTICS; VIDEO RECORDING;

INCREASED FLEXIBILITY; SEMANTIC LABELS; SPATIAL NETWORK; STRUCTURED INFORMATION; TWO-COMPONENT; VIDEO CONTENTS; VIDEO SEGMENTATION; VIDEO-OBJECT SEGMENTATION;

PATTERN RECOGNITION;

EID: 85041928016     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.336     Document Type: Conference Paper

References (81)

1. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk. SLIC superpixels compared to state-of-the-art superpixel methods. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(11): 2274-2282, 2012.
2. A. Adams, J. Baek, and M. A. Davis. Fast high-dimensional filtering using the permutohedral lattice. In Computer Graphics Forum, volume 29, pages 753-762. Wiley Online Library, 2010.
3. A. Adams, N. Gelfand, J. Dolson, and M. Levoy. Gaussian kd-trees for fast high-dimensional filtering. ACM Transactions on Graphics (ToG), 28(3): 21, 2009.
4. A. Agarwala, A. Hertzmann, D. H. Salesin, and S. M. Seitz. Keyframe-based tracking for rotoscoping and animation. ACM Transactions on Graphics (ToG), 23(3): 584-591, 2004.
5. V. Aurich and J.Weule. Non-linear Gaussian filters performing edge preserving diffusion. In DAGM, pages 538-545. Springer, 1995.
6. X. Bai, J. Wang, D. Simons, and G. Sapiro. Video snapcut: robust video object cutout using localized classifiers. ACM Transactions on Graphics (TOG), 28(3): 70, 2009.
7. J. T. Barron, A. Adams, Y. Shih, and C. Hernández. Fast bilateral-space stereo for synthetic defocus. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 4466-4474, 2015.
8. J. T. Barron and B. Poole. The fast bilateral solver. In European Conference on Computer Vision. Springer, 2016.
9. Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(11): 1222-1239, 2001.
10. Y. Y. Boykov and M.-P. Jolly. Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In Proceedings of the IEEE International Conference on Computer Vision, volume 1, pages 105-112. IEEE, 2001.
11. G. J. Brostow, J. Fauqueur, and R. Cipolla. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters, 30(2): 88-97, 2009.
12. G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla. Segmentation and recognition using structure from motion point clouds. In European Conference on Computer Vision, pages 44-57. Springer, 2008.
13. T. Brox, C. Bregler, and J. Malik. Large displacement optical flow. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 41-48. IEEE, 2009.
14. S. Caelles, K.-K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool. One-shot video object segmentation. arXiv preprint arXiv: 1611.05198, 2016.
15. J.-H. R. Chang and Y.-C. F. Wang. Propagated image filtering. In Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pages 10-18. IEEE, 2015.
16. A. Y. Chen and J. J. Corso. Temporally consistent multiclass video-object segmentation with the video graph-shifts algorithm. In IEEE Winter Conference on Applications of Computer Vision, pages 614-621. IEEE, 2011.
17. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv: 1412.7062, 2014.
18. Y.-Y. Chuang, A. Agarwala, B. Curless, D. H. Salesin, and R. Szeliski. Video matting of complex scenes. ACM Transactions on Graphics (ToG), 21(3): 243-248, 2002.
19. R. De Nijs, S. Ramos, G. Roig, X. Boix, L. Van Gool, and K. Kühnlenz. On-line semantic perception using uncertainty. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4185-4191. IEEE, 2012.
20. P. Dollár and C. L. Zitnick. Structured forests for fast edge detection. In Proceedings of the IEEE International Conference on Computer Vision, 2013.
21. P. Dollár and C. L. Zitnick. Fast edge detection using structured forests. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 37(8): 1558-1570, 2015.
22. J. Domke. Learning graphical model parameters with approximate marginal inference. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(10): 2454-2467, 2013.
23. R. Dondera, V. Morariu, Y. Wang, and L. Davis. Interactive video segmentation using occlusion boundaries and temporally coherent superpixels. In IEEE Winter Conference on Applications of Computer Vision, pages 784-791. IEEE, 2014.
24. A. Ess, T. Mueller, H. Grabner, and L. J. Van Gool. Segmentation-based urban traffic scene understanding. In British Machine Vision Conference, 2009.
25. A. Faktor and M. Irani. Video segmentation by non-local consensus voting. In BMVC, volume 2, page 6, 2014.
26. Q. Fan, F. Zhong, D. Lischinski, D. Cohen-Or, and B. Chen. Jumpcut: non-successive mask transfer and interpolation for video cutout. ACM Transactions on Graphics (ToG), 34(6): 195, 2015.
27. R. Gadde, V. Jampani, M. Kiefel, D. Kappler, and P. Gehler. Superpixel convolutional networks using bilateral inceptions. In European Conference on Computer Vision. Springer, 2016.
28. E. S. Gastal and M. M. Oliveira. Domain transform for edgeaware image and video processing. ACM Transactions on Graphics (ToG), 30(4): 69, 2011.
29. M. Grundmann, V. Kwatra, M. Han, and I. Essa. Efficient hierarchical graph-based video segmentation. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 2141-2148. IEEE, 2010.
30. K. He, J. Sun, and X. Tang. Guided image filtering. In European Conference on Computer Vision, pages 1-14. Springer, 2010.
31. J.-H. Heu, D.-Y. Hyun, C.-S. Kim, and S.-U. Lee. Image and video colorization based on prioritized source propagation. In 2009 16th IEEE International Conference on Image Processing (ICIP), pages 465-468. IEEE, 2009.
32. T.-W. Hui, C. C. Loy, and X. Tang. Depth map superresolution by deep multi-scale guidance. In European Conference on Computer Vision, pages 353-369. Springer, 2016.
33. S. D. Jain and K. Grauman. Supervoxel-consistent foreground propagation in video. In European Conference on Computer Vision, pages 656-671. Springer, 2014.
34. V. Jampani, M. Kiefel, and P. V. Gehler. Learning sparse high dimensional filters: Image filtering, dense CRFs and bilateral neural networks. In Computer Vision and Pattern Recognition, IEEE Conference on, June 2016.
35. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM International Conference on Multimedia, pages 675-678. ACM, 2014.
36. A. Khoreva, F. Perazzi, R. Benenson, B. Schiele, and A. Sorkine-Hornung. Learning video object segmentation from static images. arXiv preprint arXiv: 1612.02646, 2016.
37. M. Kiefel, V. Jampani, and P. V. Gehler. Permutohedral lattice CNNs. International Conference on Learning Representations Workshop, 2015.
38. D. Kingma and J. Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2015.
39. P. Kohli and P. H. Torr. Dynamic graph cuts for efficient inference in markov random fields. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29(12): 2079-2088, 2007.
40. J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele. Joint bilateral upsampling. ACM Transactions on Graphics (ToG), 26(3): 96, 2007.
41. P. Krähenbühl and V. Koltun. Efficient inference in fully connected CRFs with Gaussian edge potentials. In Advances in neural information processing systems, 2011.
42. T. Kroeger, R. Timofte, D. Dai, and L. Van Gool. Fast optical flow using dense inverse search. In European Conference on Computer Vision. Springer, 2016.
43. A. Kundu, V. Vineet, and V. Koltun. Feature space optimization for semantic video segmentation. In Computer Vision and Pattern Recognition, IEEE Conference on, 2016.
44. M. Lang, O.Wang, T. O. Aydin, A. Smolic, and M. H. Gross. Practical temporal consistency for image-based graphics applications. ACM Transactions on Graphics (ToG), 31(4): 34-1, 2012.
45. Y. J. Lee, J. Kim, and K. Grauman. Key-segments for video object segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pages 1995-2002. IEEE, 2011.
46. A. Levin, D. Lischinski, and Y. Weiss. Colorization using optimization. ACM Transactions on Graphics (ToG), 23(3): 689-694, 2004.
47. A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(2): 228-242, 2008.
48. F. Li, T. Kim, A. Humayun, D. Tsai, and J. M. Rehg. Video segmentation by tracking many figure-ground segments. In Proceedings of the IEEE International Conference on Computer Vision, pages 2192-2199, 2013.
49. Y. Li, J.-B. Huang, N. Ahuja, and M.-H. Yang. Deep joint image filtering. In European Conference on Computer Vision, pages 154-169. Springer, 2016.
50. Y. Li, J. Sun, and H.-Y. Shum. Video object cut and paste. ACM Transactions on Graphics (ToG), 24(3): 595-600, 2005.
51. Y. Li and R. Zemel. Mean-field networks. arXiv preprint arXiv: 1410.5884, 2014.
52. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 3431-3440, 2015.
53. N.Märki, F. Perazzi, O.Wang, and A. Sorkine-Hornung. Bilateral space video segmentation. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 743-751, 2016.
54. O. Miksik, D. Munoz, J. A. Bagnell, and M. Hebert. Efficient temporal consistency for streaming video scene analysis. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 133-139. IEEE, 2013.
55. A. Papazoglou and V. Ferrari. Fast object segmentation in unconstrained video. In Proceedings of the IEEE International Conference on Computer Vision, pages 1777-1784, 2013.
56. S. Paris. Edge-preserving smoothing and mean-shift segmentation of video streams. In European Conference on Computer Vision, pages 460-473. Springer, 2008.
57. S. Paris and F. Durand. A fast approximation of the bilateral filter using a signal processing approach. In European Conference on Computer Vision, pages 568-580. Springer, 2006.
58. S. Paris, P. Kornprobst, J. Tumblin, and F. Durand. Bilateral filtering: Theory and applications. Now Publishers Inc, 2009.
59. F. Perazzi, J. Pont-Tuset, B. McWilliams, L. V. Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Computer Vision and Pattern Recognition, IEEE Conference on, 2016.
60. F. Perazzi, O. Wang, M. Gross, and A. Sorkine-Hornung. Fully connected object proposals for video segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pages 3227-3234, 2015.
61. B. L. Price, B. S. Morse, and S. Cohen. Livecut: Learningbased interactive video segmentation by evaluation of multiple propagated cues. In Proceedings of the IEEE International Conference on Computer Vision, pages 779-786. IEEE, 2009.
62. S. A. Ramakanth and R. V. Babu. Seamseg: Video object segmentation using patch seams. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 376-383. IEEE, 2014.
63. C. Y. Ren, V. A. Prisacariu, and I. D. Reid. gslicr: Slic superpixels at over 250hz. ArXiv e-prints, (1509.04232), 2015.
64. M. Reso, B. Scheuermann, J. Jachalsky, B. Rosenhahn, and J. Ostermann. Interactive segmentation of high-resolution video content using temporally coherent superpixels and graph cut. In International Symposium on Visual Computing, pages 281-292. Springer, 2014.
65. S. R. Richter, V. Vineet, S. Roth, and V. Koltun. Playing for data: Ground truth from computer games. In European Conference on Computer Vision, pages 102-118. Springer, 2016.
66. A. G. Schwing and R. Urtasun. Fully connected deep structured networks. arXiv preprint arXiv: 1503.02351, 2015.
67. E. Shelhamer, K. Rakelly, J. Hoffman, and T. Darrell. Clockwork convnets for video semantic segmentation. arXiv preprint arXiv: 1608.03609, 2016.
68. B. Sheng, H. Sun, M. Magnor, and P. Li. Video colorization using parallel optimization in feature space. IEEE Transactions on Circuits and Systems for Video Technology, 24(3): 407-417, 2014.
69. J. Shi and J. Malik. Normalized cuts and image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(8): 888-905, 2000.
70. P. Sturgess, K. Alahari, L. Ladicky, and P. H. Torr. Combining appearance and structure from motion features for road scene understanding. In British Machine Vision Conference. BMVA, 2009.
71. N. Sundaram, T. Brox, and K. Keutzer. Dense point trajectories by gpu-accelerated large displacement optical flow. In European Conference on Computer Vision, pages 438-451. Springer, 2010.
72. B. Taylor, V. Karasev, and S. Soattoc. Causal video object segmentation from persistence of occlusions. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 4268-4276. IEEE, 2015.
73. C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proceedings of the IEEE International Conference on Computer Vision, pages 839-846. IEEE, 1998.
74. S. Tripathi, S. Belongie, Y. Hwang, and T. Nguyen. Semantic video segmentation: Exploring inference efficiency. In 2015 International SoC Design Conference (ISOCC), pages 157-158. IEEE, 2015.
75. Y.-H. Tsai, M.-H. Yang, and M. J. Black. Video segmentation via object flow. In Computer Vision and Pattern Recognition, IEEE Conference on, 2016.
76. J.Wang, P. Bhat, R. A. Colburn, M. Agrawala, and M. F. Cohen. Interactive video cutout. ACM Transactions on Graphics (ToG), 24(3): 585-594, 2005.
77. W. Wang, J. Shen, and F. Porikli. Saliency-aware geodesic video object segmentation. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 3395-3402, 2015.
78. L. Xu, J. S. Ren, Q. Yan, R. Liao, and J. Jia. Deep edge-aware filters. In Proceedings of the 32nd International Conference on Machine Learning, pages 1669-1678, 2015.
79. F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. International Conference on Learning Representations, 2016.
80. D. Zhang, O. Javed, and M. Shah. Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In Computer Vision and Pattern Recognition, IEEE Conference on, pages 628-635, 2013.
81. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. Torr. Conditional random fields as recurrent neural networks. In Proceedings of the IEEE International Conference on Computer Vision, 2015.