On dependent bit allocation for multiview image coding with depth-image-based rendering

IEEE Transactions on Image Processing

Volumn 20, Issue 11, 2011, Pages 3179-3194

  Cheung, Gene ^a   Velisavljević, Vladan ^b   Ortega, Antonio ^c  

^a NATIONAL INSTITUTE OF INFORMATICS   (Japan)

^b DEUTSCHE TELEKOM LABORATORIES   (Germany)

^c University of Southern California ^*  (United States)

Author keywords

3 D image coding; Bit allocation; depth image based rendering

Indexed keywords

3-D IMAGE CODING; BIT ALLOCATION; CODING TOOLS; DEPTH IMAGE BASED RENDERING; DEPTH MAP; DISTORTION MODEL; INTERMEDIATE VIEW; LOW COMPLEXITY; MONOTONICITY; MULTI-VIEW IMAGE; OPTIMAL SELECTION; QUANTIZATION LEVELS; SHORTEST PATH; SUBOPTIMAL SOLUTION; VIDEO STANDARD; VISUAL DISTORTION; VISUAL QUALITIES;

COMMUNICATION SYSTEMS; ENCODING (SYMBOLS); IMAGE COMMUNICATION SYSTEMS; IMAGE RECONSTRUCTION; IMAGE UNDERSTANDING; TEXTURES; THREE DIMENSIONAL; VISUAL COMMUNICATION;

IMAGE CODING;

EID: 80054801018     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2158230     Document Type: Article

References (45)

1. B. Wilburn, M. Smulski, K. Lee, and M. A. Horowitz, "The light field video camera," in Proc. Media Processors SPIE Electron. Imag., San Jose, CA, Jan. 2002.
2. T. Fujii, K. Mori, K. Takeda, K. Mase, M. Tanimoto, and Y. Suenaga, "Multipoint measuring system for video and sound-100 camera and microphone system," in Proc. IEEE Int. Conf. Multimedia Expo, Toronto, ON, Canada, Jul. 2006, pp. 437-440.
3. MPEG-Technologies-Introduction to multiview video coding Jan. 2008, iSO/IEC JTC 1/SC 29/WG 11 N9580.
4. T. Chen, "Adaptive temporal interpolation using bidirectional motion estimation and compensation," in Proc. IEEE Int. Conf. Image Process., Rochester, NY, Sep. 2002, pp. II-313-II-316.
5. J. Zhai, K. Yu, J. Li, and S. Li, "A low complexity motion compensated frame interpolation method," in Proc. IEEE Int. Symp. Circuits Syst., Kobe, Japan, May 2005, pp. 4927-4930.
6. P. Merkle, A. Smolic, K. Mueller, and T. Wiegand, "Multi-view video plus depth representation and coding," in Proc. IEEE Int. Conf. Image Process., San Antonio, TX, Oct. 2007, pp. I-201-I-204.
7. P. Kauff, N. Atzpadin, C. Fehn, M. Mller, O. Schreer, A. Smolic, and R. Tanger, "Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability," Signal Process. Image Commun., vol. 22, no. 2, pp. 217-234, Feb. 2007, Special Issue on three-dimensional video and television.
8. S. Gokturk, H. Yalcin, and C. Bamji, "A time-of-flight depth sensor-System description, issues and solutions," in Proc. Conf. CVPRW, Washington, DC, Jun. 2004, p. 35.
9. H.-Y. Shum, S.-C. Chan, and S. B. Kang, Image-Based Rendering. New York: Springer-Verlag, 2007.
10. Y. Morvan, D. Farin, and P. H. N. de With, "Multiview depth-image compression using an extended H. 264 encoder," in Proc. Adv. Concepts Intell. Vis. Syst., 2007, vol. 4678, Lecture Notes in Computer Sciences, pp. 675-686.
11. G. Cheung and V. Velisavljević, "Efficient bit allocation for multiview image coding & view synthesis," in Proc. IEEE Int. Workshop Multimedia Signal Process., St. Malo, France, Oct. 2010.
12. J. Shade, S. Gortler, L. He, and R. Szeliski, "Layered depth images," in Proc. SIGGRAPH, New York, Sep. 1998, pp. 231-242.
13. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. SIGGRAPH, New Orleans, LA, Aug. 1996, pp. 31-42.
14. S. Gortler,R. Grzeszczuk, R. Szeliski, and M. Cohen, "The lumigraph," in Proc. SIGGRAPH, New Orleans, LA, Aug. 1996, pp. 43-54.
15. P. Debevec, C. Taylor, and J. Malik, "Modeling and rendering architecture from photographs," in Proc. SIGGRAPH, New Orleans, LA, Aug. 1996, pp. 11-20.
16. H.-Y. Shum, S. B. Kang, and S.-C. Chan, "Survey of image-based representations and compression techniques," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1020-1037, Nov. 2003.
17. M. Magnor, P. Ramanathan, and B. Girod, "Multi-view coding for image-based rendering using 3-D scene geometry," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1092-1106, Nov. 2003.
18. S. Seitz and C. Dyer, "View morphing," in Proc. SIGGRAPH, New Orleans, LA, Aug. 1996, pp. 21-30.
19. W. Mark, L. McMillan, and G. Bishop, "Post-rendering 3D warping," in Proc. Symp. Interactive 3D Graph., New York, Apr. 1997, p. 17.
20. M. Tanimoto, T. Fujii, and K. Suzuki, "Multi-view depth map of Rena and Akko & Kayo," ISO/IEC JTC1/SC29/WG11 MPEG Document M14888 Oct. 2007.
21. M. Flierl, A. Mavlankar, and B. Girod, "Motion and disparity compensated coding for multiview video," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 11, pp. 1474-1484, Nov. 2007.
22. P. Merkle, A. Smolic, K. Muller, and T. Wiegand, "Efficient prediction structures for multiview video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 11, pp. 1461-1473, Nov. 2007.
23. T.-Y. Chung, I.-L. Jung, K. Song, and C.-S. Kim, "Multi-view video coding with view interpolation prediction for 2D camera arrays," J. Vis. Commun. Image Represent., vol. 21, no. 5/6, pp. 474-486, Jul. 2010.
24. Y. Morvan, D. Farin, and P. H. de With, "Depth-image compression based on an R-D optimized quadtree decomposition for the transmission of multiview images," in Proc. IEEE Int. Conf. Image Process., San Antonio, TX, Sep. 2007, pp. V-105-V-108.
25. M. Maitre, Y. Shinagawa, and M. Do, "Wavelet-based joint estimation and encoding of depth-image-based representations for free-viewpoint rendering," IEEE Trans. Image Process., vol. 17, no. 6, pp. 946-957, Jun. 2008.
26. A. Smolic, K. Muller,K. Dix, P. Merkle, P. Kauff, and T. Wiegand, "Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems," in Proc. IEEE Int. Conf. Image Process., San Diego, CA, Oct. 2008, pp. 2448-2451.
27. J.-R. Ohm, E. Izquierdo, and K. Muller, "Systems for disparity-based multiple-view interpolation," in Proc. IEEE Int. Symp. Circuits Syst., Monterey, CA, 1998, vol. 5, pp. 502-505.
28. Y. Liu, Q. Huang, S. Ma, D. Zhao, and W. Gao, "Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model," Signal Process. Image Commun., vol. 24, no. 8, pp. 666-681, Sep. 2009.
29. W.-S. Kim, A. Ortega, P. Lai, D. Tian, and C. Gomila, "Depth map distortion analysis for view rendering and depth coding," in Proc. IEEE Int. Conf. Image Process., Cairo, Egypt, Nov. 2009, pp. 721-724.
30. W.-S. Kim, A. Ortega, P. Lai, D. Tian, and C. Gomila, "Depth map coding with distortion estimation of rendered view," in Proc. SPIE Visual Inf. Process. Commun., San Jose, CA, Jan. 2010, pp. 75 430B- 1-75 430B-10.
31. Y. Shoham and A. Gersho, "Efficient bit allocation for an arbitrary set of quantizers," IEEE Trans. Acoust., Speech, Signal Process., vol. 36, no. 9, pp. 1445-1453, Sep. 1988.
32. K. Ramchandran, A. Ortega, and M. Vetterli, "Bit allocation for dependent quantization with applications to multiresolution and MPEG video coders," IEEE Trans. Image Process., vol. 3, no. 5, pp. 533-545, Sep. 1994.
33. J.-H. Kim, J. Garcia, and A. Ortega, "Dependent bit allocation in multiview video coding," in Proc. IEEE Int. Conf. Image Process., Genoa, Italy, Sep. 2005, pp. II-293-II-296.
34. H. Song and C.-C. J. Kuo, "Rate control for low-bit-rate video via variable- encoding frame rates," IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 4, pp. 512-521, Apr. 2001.
35. G. Liebl, M. Kalman, and B. Girod, "Variable frame skipping scheme based on estimated quality of non-coded frames at decoder for real-time block-based video coding," in Proc. IEEE Int. Conf. Multimedia Expo., Taipei, Taiwan, Jul. 2004, pp. 1127-1130.
36. S. Liu and C.-C. J. Kuo, "Joint temporal-spatial bit allocation for video coding with dependency," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 1, pp. 15-26, Jan. 2005.
37. G. Cheung and V. Velisavljević, "Efficient bit allocation for multiview image coding & view synthesis," in Proc. IEEE Int. Conf. Image Process., Hong Kong, Sep. 2010, pp. 2613-2616.
38. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. . Cambridge, U. K. : Cambridge Univ. Press, 2003.
39. P. Merkle, Y. Morvan, A. Smolic, D. Farin, K. Muller, P. de With, and T. Wiegand, "The effects of multiview depth video compression on multiview rendering," Signal Process. Image Commun., vol. 24, no. 1/2, pp. 73-88, Jan. 2009.
40. C. Bishop, Pattern Recognition and Machine Learning. . New York: Springer-Verlag, 2006.
41. 2006 Stereo Datasets [Online]. Available: http://vision. middlebury. edu/stereo/data/scenes2006/
42. G. Cheung, W.-T. Tan, and C. Chan, "Reference frame optimization for multiple-path video streaming with complexity scaling," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 6, pp. 649-662, Jun. 2007.
43. I. Daribo, C. Tillier, and B. Pesquet-Popescu, "Motion vector sharing and bit-rate allocation for 3D video-plus-depth coding," EURASIP: Special Issue on 3DTV in Journal on Advances in Signal Processing, vol. 2009, pp. 258 920-1-258 920-13, Jan. 2009.
44. W. Li and B. Li, "Virtual view synthesis with heuristic spatial motion," in Proc. IEEE Int. Conf. Image Process., San Diego, CA, Oct. 2008, pp. 1508-1511.
45. T. Wiegand, G. Sullivan, G. Bjontegaard, and A. Luthra, "Overview of the H. 264/AVC video coding standard," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560-576, Jul. 2003.