An Approach to Streaming Video Segmentation with Sub-Optimal Low-Rank Decomposition

IEEE Transactions on Image Processing

Volumn 25, Issue 5, 2016, Pages 1947-1960

  Li, Chenglong ^a   Lin, Liang ^b   Zuo, Wangmeng ^c   Wang, Wenzhong ^a   Tang, Jin ^a  

^a ANHUI UNIVERSITY   (China)

^b SUN YAT SEN UNIVERSITY   (China)

^c HARBIN INSTITUTE OF TECHNOLOGY   (China)

Author keywords

Lowrank representation; spectral clustering; Streaming segmentation; Video processing

Indexed keywords

ALGORITHMS; CLUSTERING ALGORITHMS; INFERENCE ENGINES; ITERATIVE METHODS; MATRIX ALGEBRA; VIDEO SIGNAL PROCESSING;

CLOSED FORM SOLUTIONS; CONSISTENT CONSTRAINT; EFFECTIVE APPROACHES; LOW-RANK DECOMPOSITION; LOW-RANK REPRESENTATIONS; SPATIO-TEMPORAL SEGMENTATION; SPECTRAL CLUSTERING; VIDEO PROCESSING;

IMAGE SEGMENTATION;

EID: 84963853529     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2016.2537211     Document Type: Article

References (50)

1. S. Paris, "Edge-preserving smoothing and mean-shift segmentation of video streams, " in Proc. Eur. Conf. Comput. Vis., 2008, pp. 460-473.
2. K. Fragkiadaki, G. Zhang, J. Shi, "Video segmentation by tracing discontinuities in a trajectory embedding, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 1846-1853.
3. F. Galasso, R. Cipolla, B. Schiele, "Video segmentation with superpixels, " in Proc. Asian Conf. Comput. Vis., 2012, pp. 760-774.
4. M. Grundmann, V. Kwatra, M. Han, I. Essa, "Efficient hierarchical graph-based video segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 2141-2148.
5. C. Xu, C. Xiong, J. J. Corso, "Streaming hierarchical video segmentation, " in Proc. Eur. Conf. Comput. Vis., 2012, pp. 626-639.
6. A. Vazquez-Reina, S. Avidan, H. Pfister, E. Miller, "Multiple hypothesis video segmentation from superpixel flows, " in Proc. Eur. Conf. Comput. Vis., 2010, pp. 268-281.
7. C. Xu and J. J. Corso, "Evaluation of super-voxel methods for early video processing, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 1202-1209.
8. F. Galasso, N. S. Nagaraja, T. J. Cardenas, T. Brox, B. Schiele, "A unified video segmentation benchmark: Annotation, metrics and analysis, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 3527-3534.
9. J. Wang, P. Bhat, R. A. Colburn, M. Agrawala, M. F. Cohen, "Interactive video cutout, " ACM Trans. Graph., vol. 24, no. 3, pp. 585-594, Jul. 2005.
10. X. Bai, J. Wang, D. Simons, G. Sapiro, "Video SnapCut: Robust video object cutout using localized classifiers, " ACM Trans. Graph., vol. 28, no. 3, Aug. 2009, Art. no. 70.
11. B. L. Price, B. S. Morse, S. Cohen, "LIVEcut: Learningbased interactive video segmentation by evaluation of multiple propagated cues, " in Proc. IEEE Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 779-786.
12. R. Dondera, V. Morariu, Y. Wang, L. Davis, "Interactive video segmentation using occlusion boundaries and temporally coherent superpixels, " in Proc. IEEE Workshop Appl. Comput. Vis., Mar. 2014, pp. 784-791.
13. L. Lin, W. Yang, C. Li, J. Tang, X. Cao, "Inference with collaborative model for interactive tumor segmentation in medical image sequences, " IEEE Trans. Cybern., doi: 10. 1109/TCYB. 2015. 2489719, 2016.
14. V. Badrinarayanan, F. Galasso, R. Cipolla, "Label propagation in video sequences, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3265-3272.
15. K. Tang, R. Sukthankar, J. Yagnik, L. Fei-Fei, "Discriminative segment annotation in weakly labeled video, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 2483-2490.
16. B. Cheng, G. Liu, J. Wang, Z. Huang, S. Yan, "Multi-task lowrank affinity pursuit for image segmentation, " in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 2439-2446.
17. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, Y. Ma, "Robust recovery of subspace structures by low-rank representation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 171-184, Jan. 2013.
18. X. Liu, Q. Xu, J. Ma, H. Jin, Y. Zhang, "MsLRR: A unified multiscale low-rank representation for image segmentation, " IEEE Trans. Image Process., vol. 23, no. 5, pp. 2159-2167, May 2014.
19. Z. Lin, A. Ganesh, J. Wright, M. Chen, L. Wu, Y. Ma, "Fast convex optimization algorithms for exact recovery of a corrupted lowrank matrix, " Univ. Illinois Urbana-Champaign, Champaign, IL, USA, Tech. Rep. UILU-ENG-09-2214, 2009.
20. G. Liu, Z. Lin, Y. Yu, "Robust subspace segmentation by low-rank representation, " in Proc. Int. Conf. Mach. Learn., 2010, pp. 1-8.
21. E. Elhamifar and R. Vidal, "Sparse subspace clustering, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 2790-2797.
22. A. Faktor and M. Irani, "Video segmentation by non-local consensus voting, " in Proc. Brit. Mach. Vis. Conf., Sep. 2014, pp. 1-12.
23. X. Liu, L. Lin, A. L. Yuille, "Robust region grouping via internal patch statistics, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1931-1938.
24. S. X. Yu and J. Shi, "Segmentation given partial grouping constraints, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 2, pp. 173-183, Feb. 2004.
25. N. Komodakis, G. Tziritas, N. Paragios, "Performance vs computational efficiency for optimizing single and dynamic MRFs: Setting the state of the art with primal-dual strategies, " Comput. Vis. Image Understand., vol. 112, no. 1, pp. 14-29, Oct. 2008.
26. J. J. Corso, E. Sharon, S. Dube, S. El-Saden, U. Sinha, A. Yuille, "Efficient multilevel brain tumor segmentation with integrated Bayesian model classification, " IEEE Trans. Med. Imag., vol. 27, no. 5, pp. 629-640, May 2008.
27. L. Lin, X. Wang, W. Yang, J.-H. Lai, "Discriminatively trained And-Or graph models for object shape detection, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 5, pp. 959-972, May 2015.
28. F. Galasso, M. Keuper, T. Brox, B. Schiele, "Spectral graph reduction for efficient image and streaming video segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 49-56.
29. K. Wang, L. Lin, J. Lu, C. Li, K. Shi, "PISA: Pixelwise image saliency by aggregating complementary appearance contrast measures with edge-preserving coherence, " IEEE Trans. Image Process., vol. 24, no. 10, pp. 3019-3033, Oct. 2015.
30. L. Lin, K. Wang, W. Zuo, M. Wang, J. Luo, L. Zhang, "A deep structured model with radius-margin bound for 3D human activity recognition, " Int. J. Comput. Vis., doi: 10. 1109/TCYB. 2015. 2489719, 2016.
31. T. Tarabalka, G. Charpiat, L. Brucker, B. H. Menze, "Spatiotemporal video segmentation with shape growth or shrinkage constraint, " IEEE Trans. Image Process., vol. 23, no. 9, pp. 3829-3840, Sep. 2014.
32. J. Feng, Z. Lin, H. Xu, S. Yan, "Robust subspace segmentation with block-diagonal prior, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 3818-3825.
33. J. Yang and X. Yuan, "Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization, " Math. Comput., vol. 82, no. 281, pp. 301-329, 2012.
34. Y. Hu, D. Zhang, J. Ye, X. Li, X. He, "Fast and accurate matrix completion via truncated nuclear norm regularization, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 9, pp. 2117-2130, Sep. 2013.
35. C. Tomasi and T. Kanade, "Shape and motion from image streams under orthography: A factorization method, " Int. J. Comput. Vis., vol. 9, no. 2, pp. 137-154, Nov. 1992.
36. T. Okatani and K. Deguchi, "On the Wiberg algorithm for matrix factorization in the presence of missing components, " Int. J. Comput. Vis., vol. 73, no. 3, pp. 329-337, May 2007.
37. C. Bregler, A. Hertzmann, H. Biermann, "Recovering non-rigid 3D shape from image streams, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2000, pp. 690-696.
38. P. Jain, P. Netrapalli, S. Sanghavi, "Low-rank matrix completion using alternating minimization, " in Proc. Annu. Symp. Theory Comput., 2013, pp. 665-674.
39. X. Cai, C. Ding, F. Nie, H. Huang, "On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions, " in Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2013, pp. 1124-1132.
40. R. Liu, Z. Lin, F. De la Torre, Z. Su, "Fixed-rank representation for unsupervised visual learning, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 598-605.
41. B. P. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA, USA: Athena Scientific, 1999.
42. D. Tsai, M. Flagg, J. M. Rehg, "Motion coherent tracking with multi-label MRF optimization, " in Proc. Brit. Mach. Vis. Conf., 2010.
43. M. Turk and A. Pentland, "Eigenfaces for recognition, " J. Cognit. Neurosci., vol. 3, no. 1, pp. 71-86, 1991.
44. S. X. Yu and J. Shi, "Multiclass spectral clustering, " in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2003, pp. 313-319.
45. H. Hu, J. Feng, C. Yu, J. Zhou, "Multi-class constrained normalized cut with hard, soft, unary and pairwise priors and its applications to object segmentation, " IEEE Trans. Image Process., vol. 22, no. 11, pp. 4328-4340, Nov. 2013.
46. C. Li, L. Lin, W. Zuo, S. Yan, J. Tang, "SOLD: Sub-optimal lowrank decomposition for efficient video segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 5519-5527.
47. P. Arbelaéz, M. Maire, C. Fowlkes, J. Malik, "Contour detection and hierarchical image segmentation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011.
48. X. Zach, T. Pock, H. Bischof, "A duality based approach for realtime TV-L1 optical flow, " in Proc. Joint DAGM Symp., 2008, pp. 214-223.
49. S. Vijayanarasimhan and K. Grauman, "Active frame selection for label propagation in videos, " in Proc. Eur. Conf. Comput. Vis., 2012, pp. 496-509.
50. R. Zhang, L. Lin, R. Zhang, W. Zuo, L. Zhang, "Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification, " IEEE Trans. Image Process., vol. 24, no. 12, pp. 4766-4779, Dec. 2015.