Real-time segmentation of stereo videos on a portable system with a mobile GPU

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 22, Issue 9, 2012, Pages 1292-1305

  Abramov, Alexey ^a   Pauwels, Karl ^b   Papon, Jeremie ^a   Wörgötter, Florentin ^a   Dellen, Babette ^c  

^a UNIVERSITY OF GÖTTINGEN   (Germany)

^b UNIVERSITY OF GRANADA   (Spain)

^c INSTITUT DE ROBÒTICA I INFORMÀTICA INDUSTRIAL CSIC UPC   (Spain)

Author keywords

Mobile systems; stereo segmentation; visual front end

Indexed keywords

CONDENSED REPRESENTATIONS; FRONT END; MOBILE GRAPHICS; MOBILE ROBOTIC; MOBILE SYSTEMS; MODEL FREE; MOVING CAMERAS; OBJECT MANIPULATION; PORTABLE SYSTEM; REAL-TIME SEGMENTATION; REAL-WORLD SEQUENCES; RESOURCE LIMITATIONS; SPATIO-TEMPORAL SEGMENTATION; STEREO VIDEO; VISUAL INFORMATION;

COMPUTER GRAPHICS; IMAGE SEGMENTATION; PORTABLE EQUIPMENT; PROGRAM PROCESSORS; VISUAL SERVOING;

COMPUTER GRAPHICS EQUIPMENT;

EID: 84866024188     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2012.2199389     Document Type: Article

References (48)

1. G. Bradski, "The OpenCV library," Doctor Dobb's J. Softw. Tools, vol. 25, no. 11, pp. 122-126, Nov. 2000.
2. Nvidia CUDA C Programming Guide, NVIDIA Corporation, Santa Clara, CA, 2011.
3. A. Abramov, T. Kulvicius, F. Wörgötter, and B. Dellen, "Real-time image segmentation on a GPU," in Proc. Facing Multicore-Challenge, vol. 6310. 2010, pp. 131-142.
4. K. Nummiaro, E. Koller-Meier, and L. J. Van Gool, "Object tracking with an adaptive color-based particle filter," in Proc. DAGM Symp., 2002, pp. 353-360.
5. Y. Huang, Q. Liu, and D. N. Metaxas, "Video object segmentation by hypergraph cut," in Proc. CVPR, 2009, pp. 1738-1745.
6. M. D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. J. Van Gool, "Robust tracking-by-detection using a detector confidence particle filter," in Proc. ICCV, 2009, pp. 1515-1522.
7. M. Unger, T. Mauthner, T. Pock, and H. Bischof, "Tracking as segmentation of spatial-temporal volumes by anisotropic weighted TV," in Proc. EMMCVPR, 2009, pp. 193-206.
8. L. Ladický, P. Sturgess, C. Russell, S. Sengupta, Y. Bastanlar, W. Clocksin, and P. Torr, "Joint optimization for object class segmentation and dense stereo reconstruction," in Proc. BMVC, 2010, pp. 1-11.
9. C. D. Mutto, P. Zanuttigh, G. M. Cortelazzo, and S. Mattoccia, "Scene segmentation assisted by stereo vision," in Proc. 3DIMPVT, 2011, pp. 57-64.
10. C. Wang, M. de la Gorce, and N. Paragios, "Segmentation, ordering and multi-object tracking using graphical models," in Proc. ICCV, 2009, pp. 747-754.
11. M. Grundmann, V. Kwatra, M. Han, and I. A. Essa, "Efficient hierarchical graph-based video segmentation," in Proc. CVPR, 2010, pp. 2141-2148.
12. A. V. Reina, S. Avidan, H. Pfister, and E. L. Miller, "Multiple hypothesis video segmentation from superpixel flows," in Proc. ECCV, 2010, pp. 268-281.
13. S. Paris, "Edge-preserving smoothing and mean-shift segmentation of video streams," in Proc. ECCV, 2008, pp. 460-473.
14. S. Liu, G. Dong, C. H. Yan, and S. H. Ong, "Video segmentation: Propagation, validation and aggregation of a preceding graph," in Proc. CVPR, 2008, pp. 1-7.
15. W. Brendel and S. Todorovic, "Video object segmentation by tracking regions," in Proc. ICCV, 2009, pp. 833-840.
16. C. Liu, W. T. Freeman, E. H. Adelson, and Y. Weiss, "Human-assisted motion annotation," in Proc. CVPR, 2008, pp. 1-8.
17. P. Salamon, P. Sibani, and R. Frost, Facts, Conjectures, and Improvements for Simulated Annealing. Philadelphia, PA: Soc. Indust. Appl. Math., 2002.
18. V. Hedau, H. Arora, and N. Ahuja, "Matching images under unstable segmentations," in Proc. CVPR, 2008, pp. 1-8.
19. D. C. Tseng and C. H. Chang, "Color segmentation using perceptual attributes," in Proc. Int. Conf. Patt. Recog., 1992, pp. 228-231.
20. Industrial Colour-Difference Evaluation, CIE Pub. 116-1995, CIE Central Bureau, Vienna, Austria, 1995.
21. Y. Boykov and G. Funka-Lea, "Graph cuts and efficient N-D image segmentation," Int. J. Comput. Vis., vol. 70, no. 2, pp. 109-131, 2006.
22. H. D. Cheng, X. H. Jiang, Y. Sun, and J. L. Wang, "Color image segmentation: Advances and prospects," Pattern Recognit., vol. 34, no. 12, pp. 2259-2281, 2001.
23. H. Kjellström, J. Romero, and D. Kragic, "Visual object-action recognition: Inferring object affordances from human demonstration," Comput. Vision Image Understand., vol. 115, no. 1, pp. 81-90, 2011.
24. P. Arbelaez, M. Maire, C. C. Fowlkes, and J. Malik, "From contours to regions: An empirical evaluation," in Proc. CVPR, 2009, pp. 2294-2301.
25. J. Y. A. Wang and E. H. Adelson, "Representing moving images with layers," IEEE Trans. Image Process., vol. 3, no. 5, pp. 625-638, May 1994.
26. A. Abramov, E. E. Aksoy, J. Dörr, K. Pauwels, F. Wörgötter, and B. Dellen, "3D semantic representation of actions from efficient stereo-image-sequence segmentation on GPUs," in Proc. 5th Int. Symp. 3DPVT, 2010, pp. 1-8.
27. P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image segmentation," Int. J. Comput. Vision, vol. 59, no. 2, pp. 167-181, 2004.
28. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603-619, May 2002.
29. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
30. E. E. Aksoy, A. Abramov, J. Dörr, K. Ning, B. Dellen, and F. Wörgötter, "Learning the semantics of object-action relations by observation," Int. J. Robot. Res., vol. 30, no. 10, pp. 1229-1249, 2011.
31. M. Blatt, S. Wiseman, and E. Domany, "Superparamagnetic clustering of data," Phys. Rev. Lett., vol. 76, no. 18, pp. 3251-3254, 1996.
32. C. von Ferber and F. Wörgötter, "Cluster update algorithm and recognition," Phys. Rev. E, vol. 62, no. 2, pp. 1461-1464, 2000.
33. R. Opara and F. Wörgötter, "A fast and robust cluster update algorithm for image segmentation in split-lattice models without annealing: Visual latencies revisited," Neural Computat., vol. 10, no. 6, pp. 1547-1566, 1998.
34. K. Pauwels, N. Krüger, M. Lappe, F. Wörgötter, and M. M. Van Hulle, "A cortical architecture on parallel hardware for motion processing in real time," J. Vision, vol. 10, no. 10, pp. 1-18, 2010.
35. K. Pauwels, M. Tomasi, J. Diaz, E. Ros, and M. M. Van Hulle, "A comparison of FPGA and GPU for real-time phase-based optical flow, stereo, and local image features," IEEE Trans. Comput., vol. 61, no. 7, pp. 999-1012, Jul. 2012.
36. A. Wedel, T. Pock, C. Zach, D. Cremers, and H. Bischof, "An improved algorithm for TV-L1 optical flow," in Proc. Dagstuhl Motion Workshop, 2008.
37. T. Gautama and M. M. Van Hulle, "A phase-based approach to the estimation of the optical flow field using spatial filtering," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1127-1136, Sep. 2002.
38. R. Swendsen and S. Wang, "Nonuniversal critical dynamics in Monte Carlo simulations," Phys. Rev. Lett., vol. 76, no. 18, pp. 86-88, 1987.
39. U. Wolff, "Collective Monte Carlo updating for spin systems," Phys. Rev. Lett., vol. 62, no. 4, pp. 361-364, 1989.
40. P. König and N. Krüger, "Perspectives: Symbols as self-emergent entities in an optimization process of feature extraction and predictions," Biol. Cybern., vol. 94, no. 4, pp. 325-334, 2006.
41. P. Salembier and F. Marqués, "Region-based representations of image and video: Segmentation tools for multimedia services," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 8, pp. 1147-1169, Dec. 1999.
42. D. Geman and S. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 6, pp. 721-741, Nov. 1984.
43. B. Dellen and F. Wörgötter, "Disparity from stereo-segment silhouettes of weakly-textured images," in Proc. BMVC, 2009, pp. 1-11.
44. R. B. Potts, "Some generalized order-disorder transformations," Proc. Cambridge Philos. Soc., vol. 48, no. 1, pp. 106-109, 1952.
45. G. T. Barkema and T. MacFarland, "Parallel simulation of the Ising model," Phys. Rev. E, vol. 50, no. 2, pp. 1623-1628, 1994.
46. E. Lindholm, J. Nickolls, S. Oberman, and J. Montrym, "NVIDIA Tesla: A unified graphics and computing architecture," IEEE Micro, vol. 28, no. 2, pp. 39-55, Mar.-Apr. 2008.
47. C. Eckes and J. C. Vorbrüggen, "Combining data-driven and modelbased cues for segmentation of video sequences," in Proc. World Congr. Neural Netw., 1996, pp. 868-875.
48. T. Brox and J. Malik, "Large displacement optical flow: Descriptor matching in variational motion estimation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 3, pp. 500-513, Mar. 2011.