Stream-centric stereo matching and view synthesis: A high-speed approach on GPUs

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 19, Issue 11, 2009, Pages 1598-1611

  Lu, Jiangbo ^a,b,c   Rogmans, Sammy ^b,d   Lafruit, Gauthier ^b   Catthoor, Francky ^a,b  

^a Katholieke Universiteit Leuven   (Belgium)

^b IMEC   (Belgium)

^c Singapore Pte Ltd   (Singapore)

^d HASSELT UNIVERSITY   (Belgium)

Author keywords

Disparity estimation; General purpose computation on GPUs (GPGPU); Image based rendering; Stereo matching; View interpolation; View synthesis

Indexed keywords

DISPARITY ESTIMATION; GENERAL-PURPOSE COMPUTATION ON GPUS (GPGPU); IMAGE-BASED RENDERING; STEREO MATCHING; VIEW INTERPOLATION; VIEW SYNTHESIS;

BENCHMARKING; COMPUTATIONAL EFFICIENCY; COMPUTER GRAPHICS EQUIPMENT; ESTIMATION; FACE RECOGNITION; IMAGE CODING; IMAGE MATCHING; IMAGE RECONSTRUCTION; INTERPOLATION; PROGRAM PROCESSORS; VIDEO AMPLIFIERS;

SYNTHESIS (CHEMICAL);

EID: 70449372277     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2009.2026948     Document Type: Article

References (35)

1. H.-Y. Shum, S.-C. Chan, and S. B. Kang, Image-Based Rendering. New York: Springer, 2006.
2. C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video interpolation using a layered representation," in Proc. ACM Conf. Comput. Graphics (SIGGRAPH), Aug. 2004, pp. 600-608.
3. C. L. Zitnick and S. B. Kang, "Stereo for image-based rendering using image over-segmentation," Int. J. Comput. Vision, vol.75, no.1, pp. 49-65, Oct. 2007.
4. R. Yang, G. Welch, and G. Bishop, "Real-time consensus-based stereo reconstruction using commodity graphics hardware," in Proc. Pacific Graphics, Oct. 2002, pp. 225-234.
5. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comput. Vision, vol.47, pp. 7-42, May 2002.
6. J. D. Owens, D. Luebke, N. Govindaraju, M. Harris, J. Krüger, A. E. Lefohn, and T. J. Purcell, "A survey of general-purpose computation on graphics hardware," Comput. Graphics Forum, vol.26, no.1, pp. 80-113, Mar. 2007.
7. J. Fung, S. Mann, and C. Aimone, "OpenVIDIA: Parallel GPU computer vision," in Proc. ACM Int. Conf. Multimedia, Nov. 2005, pp. 849-852.
8. I. Geys, T. P. Koninckx, and L. Van Gool, "Fast interpolated cameras by combining a GPU based plane sweep with a max-flow regularisation algorithm," in Proc. Int. Conf. 3-D Data Process., Visual. Transmission, Sep. 2004, pp. 534-541.
9. J. Lu, S. Rogmans, G. Lafruit, and F. Catthoor, "Real-time stereo correspondence using a truncated separable Laplacian kernel approximation on graphics hardware," in Proc. IEEE Int. Conf. Multimedia Expo, Beijing, China, Jul. 2007, pp. 1946-1949.
10. J. Lu, G. Lafruit, and F. Catthoor, "Fast variable center-biased windowing for high-speed stereo on programmable graphics hardware," in Proc. IEEE Int. Conf. Image Process., Sep. 2007, pp. 568-571.
11. J. Lu, S. Rogmans, G. Lafruit, and F. Catthoor, "High-speed stream-centric dense stereo and view synthesis on graphics hardware," in Proc. IEEE Int. Workshop Multimedia Signal Process., Crete, Greece, May 2007, pp. 243-246.
12. S. Rogmans, J. Lu, and G. Lafruit, "A scalable end-to-end optimized real-time image-based rendering framework on graphics hardware," in Proc. IEEE Comput. Soc. 3DTV-CON, True Vision Capture, Transmission Display 3-D Video, Istanbul, Turkey, May 2008, pp. 129-132.
13. A. Fusiello, V. Roberto, and E. Trucco, "Efficient stereo with multiple windowing," in Proc. IEEE Int. Conf. Computer Vision Pattern Recog-nit., San Juan, Puerto Rico, Jun. 1997, pp. 858-863.
14. O. Veksler, "Fast variable window for stereo correspondence using integral images," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2003, pp. 556-561.
15. K. J. Yoon and S. Kweon, "Adaptive support-weight approach for correspondence search," IEEE Trans. Pattern Anal. Mach. Intell., vol.28, no.4, pp. 650-656, Apr. 2006.
16. S. Forstmann, Y. Kanou, J. Ohya, S. Thuering, and A. Schmitt, "Realtime stereo by using dynamic-programming," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit. Workshop, vol.3. Jun. 2004.
17. R. Yang and M. Pollefeys, "Multiresolution real-time stereo on commodity graphics hardware," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., vol.1. Jun. 2003, pp. 211-218.
18. R. Yang, M. Pollefeys, and S. Li, "Improved real-time stereo on commodity graphics hardware," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit. Workshop, vol.3. Jun. 2004.
19. M. Gong and R. Yang, "Image-gradient-guided real-time stereo on graphics hardware," in Proc. Int. Conf. 3-D Digital Imaging Modeling, Jun. 2005, pp. 548-555.
20. L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, "High-quality realtime stereo using adaptive cost aggregation and dynamic-programming," in Proc. Int. Conf. 3-D Data Process., Visual. Transmission, Jun. 2006, pp. 798-805.
21. J. Sun, Y. Li, S. B. Kang, and H.-Y. Shum, "Symmetric stereo matching for occlusion handling," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., vol.2. Jun. 2005, pp. 399-406.
22. A. Fitzgibbon, Y. Wexler, and A. Zisserman, "Image-based rendering using image-based priors," in Proc. IEEE Int. Conf. Comput. Vision, vol.2. Oct. 2003, pp. 1176-1183.
23. M. D. McCool, "Signal processing and general-purpose computing on GPUs," IEEE Signal Process. Mag., vol.24, no.3, pp. 109-114, May 2007
24. NVIDIA Compute Unified Device Architecture (CUDA) [Online]. Available: http://www.nvidia.com/object/cuda home.html.
25. S. Ryoo, C. Rodrigues, S. Baghsorkhi, S. Stone, D. Kirk, and W.-M. Hwu, "Optimization principles and application performance evaluation of a multithreaded GPU using CUDA," in Proc. ACM SIGPLAN Symp. Principles Practice Parallel Program., 2008, pp. 73-82.
26. M. Gong, R. Yang, L. Wang, and M. Gong, "A performance study on different cost aggregation approaches used in real-time stereo matching," Int. J. Comput. Vision, vol.75, no.2, pp. 283-296, Nov. 2007.
27. X. Huang and E. Dubois, "Disparity estimation for the intermediate view interpolation of stereoscopic images," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol.2. Mar. 2005, pp. 881-884.
28. A. Mancini and J. Konrad, "Robust quadtree-based disparity estimation for the reconstruction of intermediate stereoscopic images," in Proc. SPIE Stereoscopic Displays Virtual Reality Syst., vol.3295. Apr. 1998, pp. 53-64. (Pubitemid 35003995)
29. H. Hirschmuller, P. Innocent, and J. Garibaldi, "Real-time correlation-based stereo vision with reduced border errors," Int. J. Comput. Vision, vol.47, pp. 229-246, Apr. 2002
30. Middlebury Stereo Vision Page [Online]. Available: http://vision. middlebury.edu/stereo.
31. D. Scharstein, "Stereo vision for view synthesis," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 1996, pp. 852-858.
32. R. Szeliski, "Prediction error as a quality metric for motion and stereo," in Proc. IEEE Int. Conf. Comput. Vision, Sep. 1999, pp. 781-788.
33. S. H. Lee, Y. Kanatsugu, and J. Park, "MAP-based stochastic diffusion for stereo matching and line fields estimation," Int. J. Comput. Vision, vol.47, pp. 195-218, Apr. 2002. (Pubitemid 34428431)
34. L. Zhang, "Fast stereo matching algorithm for intermediate view reconstruction of stereoscopic television images," IEEE Trans. Circuits Syst. Video Technol., vol.16, no.10, pp. 1259-1270, Oct. 2006
35. Image Convolution with CUDA, document CUDA SDK, NVIDIA Corporation, Jun. 2007.