Interactive 3-D video representation and coding technologies

Proceedings of the IEEE

Volumn 93, Issue 1, 2005, Pages 98-110

  Smolić, Aljoscha ^a   Kauff, Peter ^a  

^a FRAUNHOFER HEINRICH HERTZ INSTITUTE   (Germany)

Author keywords

Interactive media; MPEG standards; Three dimensional (3 D) video; Video coding

Indexed keywords

CAMERAS; COMPUTER GRAPHICS; INTERACTIVE COMPUTER SYSTEMS; MOTION PICTURES; STANDARDS; THREE DIMENSIONAL; VIRTUAL REALITY;

AUDIO-VISUAL MEDIA; INTERACTIVE MEDIA; MPEG STANDARDS; THREE-DIMENSIONAL (3-D) VIDEO; VIRTUAL REALITY MODELING LANGUAGE (VRML);

IMAGE CODING;

EID: 20744449299     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2004.839608     Document Type: Article

References (61)

1. E. H. Adelson and J. Bergen, "The plenoptic function and the elements of early vision," in Computational Models of Visual Processing. Cambridge, MA: MIT Press, 1991, pp. 3-20.
2. Y. Bayakovski, L. Levkovich-Maslyuk, A. Ignatenko, A. Konushin, D. Timasov, A. Zhirkov, M. Han, and I. K. Park, "Depth image-based representations for static and animated 3D objects," presented at the IEEE Int. Conf. Image Processing, Rochester, NY, 2002.
3. C. Buehler, M. Bosse, L. McMillan, S. Gortier, and M. Cohen, "Unstructured lumigraph rendering," in Proc. SIGGRAPH 2001, pp. 425-432.
4. J.-X. Chai, S.-C. Chan, H.-Y. Shum, and X. Tong, "Plenoptic sampling," in Proc. SIGGRAPH 2000, pp. 307-318.
5. S. E. Chen, "QuickTime VR - An image-based approach to virtual environment navigation," in Proc. ACM SIGGRAPH, 1995, pp. 29-38.
6. W.-C. Chen, J.-Y. Bouguet, M. H. Chu, and R. Grzeszczuk, "Light field mapping: Efficient representation and hardware rendering of surface light fields," ACM Trans. Graphics, vol. 21, no. 3, pp. 447-456, 2002.
7. P. Debevec, C. Taylor, and J. Malik, "Modeling and rendering architecture from photographs: A hybrid geometry- and image based approach," in Proc. SIGGRAPH 1996, pp. 11-20.
8. P. Eisert, E. Steinbach, and B. Girod, "Automatic reconstruction of stationary 3-D objects from multiple uncalibrated camera views," IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 261-277, Mar. 2000.
9. EyeVision [Online]. Available: http://www.pvi-inc.com/eyevision/
10. C. Fehn, P. Kauff, M. Op de Beeck, F. Ernst, W. Ijsselsteijn, M. Pollefeys, L. Vangool, E. Ofek, and I. Sexton, "An evolutionary and optimized approach on 3D-TV," presented at the Int. Broadcast Conv. (IBC) 2002, Amsterdam, The Netherlands.
11. C. Fehn, E. Cooke, O. Schreer, and P. Kauff, "3D analysis and image-based rendering for immersive TV applications," Signal Process. Image Commun. J., Oct. 2002.
12. T. Fujii, "A basic study on the integrated 3-D visual communication," Ph.D. Dissertation, Univ. Tokyo, Tokyo, Japan, 1994. In Japanese.
13. T. Fujii, T. Kimoto, and M. Tanimoto, "Ray-space coding for 3D visual communication," in Proc. Picture Coding Symp. '96, pp. 447-451.
14. T. Fujii, M. Kaneko, and H. Harashima, "Representation of 3-D spatial information by rays and its application," J. Inst. Television Eng. Jpn., vol. 50, no. 9, pp. 1312-1318, Sept. 1996. In Japanese.
15. T. Fujii, T. Kimoto, and M. Tanimoto, "A new flexible acquisition system of ray-space data for arbitrary objects," IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 218-224, Mar. 2000.
16. T. Fujii and M. Tanimoto, "Free-viewpoint TV system based on rayspace representation," in Proc. SPIE ITCom, vol. 4864-22, 2002, pp. 175-189.
17. B. Goldlücke and M, Magnor, "Real-time microfacet billboarding for free-viewpoint video rendering," presented at the IEEE Int. Conf. Image Processing (ICIP) 2003, Barcelona, Spain.
18. S. J. Gortler, R. Grzesczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proc. ACM SIGGRAPH'96, pp. 43-54.
19. M. Gross, S. Würmlin, M. Naef, E. Lamboray, C. Spagno, A. Kunz, E. Koller-Meier, T. Svoboda, L. Van Cool, S. Lang, K. Strehlke, A. Vande Moere, and O. Staadt, "Blue-c: A spatially immersive display and 3D video portal for telepresence," in Proc. ACM SIGGRAPH, 2003, pp. 819-827.
20. C. Grünheit, A. Smolić, and T. Wiegand, "Efficient representation and interactive streaming of high-resolution panoramic views," presented at the IEEE Int. Conf. Image Processing (ICIP) 2002, Rochester, NY.
21. T. Hamaguchi, T. Fujii, Y. Kajiki, and T. Honda, "Real-time view interpolation system for multi-view 3D display," Proc. SPIE, Electron. Imag., vol. 4297A, pp. 212-221, Jan. 2001.
22. Immersive Media Company [Online], Available: http://www. immersivemedia.com/
23. "Applications and requirements for 3DAV," Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC), ISO/IEC JTC1/SC29/WG11, Doc. N5877, July 2003.
24. "Report on 3DAV exploration," Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC), ISO/IEC JTC1/SC29/WG11, Doc. N5878, July 2003.
25. "Text of ISO/IEC 14496-16:2003/FDAM4,"Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC), ISO/IEC JTC1/SC29/WG11, Doc. N5397, Dec. 2002.
26. "Advanced video coding for generic audio-visual services," Int. Telecommun. Union-Telecommun. (ITU-T) and Int. Standards OrgVInt. Electrotech. Comm. (ISO/IEC), Recommendation H.264 and ISO/IEC 14496-10 AVC, 2003.
27. S. B. Kang, R. Szeliski, and P. Anandan, "The geometry-image representation tradeoff for rendering," in IEEE Int. Conf. Image Processing (ICIP 2000), Vancouver, BC, Canada.
28. T. Kobayashi, T. Fujii, T. Kimoto, and M. Tanimoto, "Interpolation of ray-space data by adaptive filtering," Proc. SPIE, Electron. Imag., vol. 3958, pp. 252-259, Jan. 2000.
29. K. N. Kutulakos and S. M. Seitz, "A theory of shape by space carving," Univ. Rochester, Rochester, NY, Tech. Rep. 692, May 1998.
30. A. Laurentini, "The visual hull concept for silhouette based image understanding," IEEE Trans. Pattern Anal. Mach. Intell., vol. 16, pp. 150-162, June 1994.
31. W. H. Leung and T. Chen, "Compression with mosaic prediction for image-based rendering applications," presented at the IEEE Int. Conf. Multimedia Expo (ICME 2000), New York.
32. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. ACM SIGGRAPH, 1996, pp. 31-42.
33. J. Li, H. Y. Shum, and Y. Q. Zhang, "On the compression of image based rendering scene," presented at the IEEE Int. Conf. Image Processing (ICIP 2000), Vancouver, BC, Canada, 2000.
34. M. Li, M. Magnor, and H.-P. Seidel, "Hardware-accelerated visual hull reconstruction and rendering," in Proc. Graphics Interface 2003, pp. 65-71.
35. W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan, "Image-based visual hulls," in Proc. SIGGRAPH2000, pp. 369-374.
36. W. Matusik, C. Buehler, and L. McMillan, "Polyhedral visual hulls for real-time rendering," in Proc. Eurographics Workshop Rendering, 2001, pp. 115-126.
37. L. McMillan and G. Bishop, "Plenoptic modeling: An image-based rendering system," presented at the SIGGRAPH'95 Conf., Los Angeles, CA.
38. D. McCutchen, "A dodecahedral approach to immersive imaging and display," in Proc. ACM SIGGRAPH Computer Graphics, vol. 31, 1997, pp. 35-37.
39. M. Magnor and B. Girod, "Data compression for light-field rendering," IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 338-343, Apr. 2000.
40. M. Magnor, P. Ramanathan, and B. Girod, "Multi-view coding for image-based rendering using 3-D scene geometry," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1092-1106, Nov. 2003.
41. T. Matsuyama and T. Takai, "Generation, visualization, and editing of 3D video," in Proc. Symp. 3D Data Processing Visualization and Transmission, 2002, pp. 234-245.
42. T. Matsuyama, X. Wu, T. Takai, and T. Wada, "Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video," IEEE Trans. Circuits Syst. Video Technol., vol. 14, pp. 357-369, Mar. 2004.
43. K. Müller, A. Smolić, M. Droese, P. Voigt, and T. Wiegand, "Multitexture modeling of 3D traffic scenes," presented at the IEEE Int. Conf. Multimedia Expo (ICME) 2003, Baltimore, MD.
44. S. Pastoor and M. Wöpking, "3-D displays: A review of current technologies," Displays, vol. 17, no. 2, pp. 100-110, 1997.
45. S. Pastoor and J. Liu, "3-D display and interaction technologies for desktop computing," in Three-Dimensional Television, Video and Display Technologies, B. Javidi and F. Okano, Eds. New York: Springer, 2002.
46. T. Pintaric, U. Neumann, and A. Rizzo, "Immersive panoramic video," in Proc. 8th ACM Int. Conf. Multimedia, 2000, pp. 493-494.
47. P. W. Rander, P. J. Narayanan, and T. Kanade, "Virtualized reality: Constructing time varying worlds from real world events," in Proc. IEEE Visualization Conf., 1997, pp. 277-552.
48. E. Scheirer, R. Väänänen, and J. Huopaniemi, "AudioBIFS: Describing audio scenes with the MPEG-4 multimedia standard," IEEE Trans. Multimedia, vol. 1, pp. 237-250, Sept. 1999.
49. S. M. Seitz and C. R. Dyer, "Photorealistic scene reconstruction by voxel coloring," Int. J. Comput. Vis., vol. 35, no. 2, pp. 151-173, 1999.
50. J. Shade, S. Gortler, L. W. He, and R. Szeliski, "Layered depth images," presented at the SIGGRAPH'98 Conf., Orlando, FL.
51. H. Y. Shum and L. W. He, "Rendering with concentric mosaics," in Proc. ACM SIGGRAPH, 1999, pp. 299-306.
52. H. Y. Shum, K. T. Ng, and S. C. Chan, "Virtual reality using the concentric mosaic: Construction, rendering and data compression," presented at the IEEE Int. Conf. Image Processing (ICIP) 2000, Vancouver, BC, Canada.
53. A. Smolić and D. McCutchen, "3DAV exploration of video-based rendering technology in MPEG," IEEE Trans. Circuits Syst. Video Technol., vol. 14, pp. 348-356, Mar. 2004.
54. R. Szeliski, "Rapid octree construction from image sequences," CVGIP: Image Understanding 58, vol. 1, pp. 23-32, 1993.
55. R. Szeliski and H. Y. Shum, "Creating full view panoramic image mosaics and texture-mapped models," in Proc. ACM SIGGRAPH, 1997, pp. 251-258.
56. R. Y. Tsai, "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV camera and lenses," IEEE J. Robot. Automat., vol. RA-3, no. 4, pp. 323-344, Aug. 1987.
57. D. Wood, D. Azuma, W. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle, "Surface light fields for 3D photography," in Proc. SIGGRAPH, 2000, pp. 287-296.
58. S. Würmlin, E. Lamboray, O. Staadt, and M. Gross, "3D video recorder: A system for recording, processing and playing three-dimensional video," in Computer Graphics Forum 22 (2), Oxford, U.K.: Blackwell, 2003, pp. 181-193.
59. S. Würmlin, E. Lamboray, and M. Gross, "3D video fragments: Dynamic point samples for real-time free-viewpoint video," Comput. Graph. (Special Issue on Coding, Compression, and Streaming Techniques for 3D and Multimedia Data), vol. 28, no. 1, pp. 3-14, 2004.
60. S. Würmlin, E. Lamboray, M. Waschbüsch, M. Gross, and, "Dynamic point samples for free-viewpoint video," presented at the Picture Coding Symp. 2004 (PCS04), San Francisco, CA.
61. C. Zhang and J. Li, "Compression of lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering," presented at the IEEE Data Compression Conf., Snowbird, UT, 2000.