Development of MPEG standards for 3D and free viewpoint video

Proceedings of SPIE - The International Society for Optical Engineering

Volumn 6016, Issue , 2005, Pages

  Smolic, Aljoscha ^a   Kimata, Hideaki ^b   Vetro, Anthony ^c  

^a FRAUNHOFER HEINRICH HERTZ INSTITUTE   (Germany)

^b NTT CORPORATION   (Japan)

^c MERL ^*  (United States)

Author keywords

3D video; 3D video objects; 3DAV; 3DTV; Free viewpoint video; MPEG; Multi view video coding

Indexed keywords

FOCUSING; IMAGE CODING; IMAGE PROCESSING; NAVIGATION; STANDARDIZATION; VIDEODISKS; VIRTUAL REALITY;

3D VIDEO; 3D VIDEO OBJECTS; 3DTV; FREE VIEWPOINT VIDEO; MPEG; MULTI-VIEW VIDEO CODING;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 33644755345     PISSN: 0277786X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1117/12.631192     Document Type: Conference Paper

References (62)

1. A. Smolic, and P. Kauff, "Interactive 3D Video Representation and Coding Technologies", Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, vol. 93, no. 1, Jan. 2005
2. ISO/IEC JTC1/SC29/WG11, "Call for Comments on 3DAV", Doc. N6051, Gold Coast, Australia, October 2003.
3. ISO/IEC JTC1/SC29/WG11, "Requirements on Multi-view Video Coding v.2", Doc. N7282, Poznan, Poland, July 2005.
4. ISO/IEC JTC1/SC29/WG11, "Survey of Algorithms used for Multi-view Video Coding (MVC)", Doc. N6909, Hong Kong, China, January 2005.
5. ISO/IEC JTC1/SC29/WG11, "Call for Evidence on Multi-View Video Coding", Doc. N6720, Palma de Mallorca, Spain, October 2004.
6. ISO/IEC JTC1/SC29/WG11, "Report of the subjective quality evaluation for MVC Call for Evidence", Doc. N6999, Hong Kong, China, January 2005.
7. W. Matusik, H. Pfister, "3D TV: A Scalable System for Real-Time Acquistion, Transmission and Autostereoscopic Display of Dynamic Scenes", ACM Transactions on Graphics (TOG) SIGGRAPH, ISSN: 0730-0301, Vol. 23, Issue 3, pp. 814-824, August 2004.
8. ISO/IEC JTC1/SC29/WG11, "Call for Proposals on Multi-view Video Coding", Doc. N7327, Poznan, Poland, July 2005.
9. Masayuki Tanimoto, "Free Viewpoint Television-FTV", Proc. PCS 2004, Picture Coding Symposium, San Francisco, CA, USA, December 15.-17. 2004.
10. T. Fujii, M. Tanimoto, "Free-Viewpoint TV System Based on Ray-Space Representation", SPIE ITCom Vol. 4864-22, pp. 175-189 (2002).
11. K. Mueller, A. Smolic, P. Merkle, M. Kautzner, and T. Wiegand, "Coding of 3D Meshes and Video Textures for 3D Video Objects", Proc. PCS 2004, Picture Coding Symposium, San Francisco, CA, USA, December 15.-17. 2004.
12. C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-Quality Video View Interpolation Using a Layered Representation" , SIGGRAPH04, Los Angeles, CA, USA, August 2004.
13. S.B. Kang, R. Szeliski, and P. Anandan, "The Geometry-Image Representation Tradeoff for Rendering", Proc. ICIP2000, IEEE International Conference on Image Processing, Vancouver, Canada, September 2000.
14. A. Smolic, and D. McCutchen, "3DAV Exploration of Video-Based Rendering Technology in MPEG", IEEE Trans. on Circuits and Systems for Video Technology, Vol. 14, No. 3, pp. 348-356, March 2004.
15. R.Y. Tsai, "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV camera and lenses", IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, August 1987.
16. ISO/IEC JTC1/SC29/WG11, "Applications and Requirements for 3DAV", Doc. N5877, Trondheim, Norway, July 2003.
17. ISO/IEC JTC1/SC29/WG11, "Report on 3DAV Exploration", Doc. N5878, Trondheim, Norway, July 2003.
18. ISO/IEC JTC1/SC29/WG11, "ISO/IEC 14496-16/PDAMI", Doc. N6544, Redmont, WA, USA, July 2004.
19. M. Magnor, and B. Girod, "Data Compression for Light-Field Rendering", IEEE Trans. on Circuits and Systems for Video Technology, vol. 10, no. 3, pp. 338-343, Apr. 2000.
20. M. Magnor, P. Ramanathan, and B. Girod, "Multi-View Coding for Image-based Rendering using 3-D Scene Geometry", IEEE Trans. on Circuits and Systems for Video Technology, Vol. 13, No. 11, pp. 1092-1106, November 2003.
21. W. E. Lorensen, and H. E. Cline, "Marching Cubes: A high resolution 3D surface reconstruction algorithm," Proceedings of SIGGRAPH, vol. 21, no. 4, pp 163-169, 1987.
22. G. Taubin, "Curve and Surface Smoothing Without Shrinkage", International Conference on Computer Vision (ICCV '95), pp. 852-857, 1995.
23. K. Mueller, A. Smolić, M. Droese, P. Voigt, and T. Wiegand, "Reconstruction of a Dynamic Environment with Fully Calibrated Background for Traffic Scenes", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No. 4, pp. 538-549, April 2005.
24. A. Laurentini, "The Visual Hull Concept for Silhouette Based Image Understanding", IEEE Trans. on PAMI, Vol. 16, No. 2, pp. 150-162, 1994.
25. W.H. Leung, and T. Chen, "Compression with Mosaic Prediction for Image-Based rendering Applications", Proc. ICME2000, IEEE International Conference on Multimedia and Expo, New York, NY, USA, July 2000.
26. M. Levoy, and P. Hanrahan, "Light Field Rendering", Proc. ACM SIGGRAPH, pp. 31-42, August 1996.
27. J. Li, H.Y. Shum, and Y.Q. Zhang, "On the Compression of Image Based Rendering Scene", Proc. ICIP2000, IEEE International Conference on Image Processing, Vancouver, Canada, September 2000.
28. M. Li, M. Magnor, and H.-P. Seidel, "Hardware-Accelerated Visual Hull Reconstruction and Rendering", pp. 65-71, Graphics Interface'2003, June 2003.
29. W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan, "Image-Based Visual Hulls", Proc. SIGGRAPH 2000, pages 369-374, 2000.
30. Matusik, W., Buehler, C., and McMillan, L., "Polyhedral Visual Hulls for Real-Time Rendering", Proc. Eurographics Workshop on Rendering 2001.
31. P. Debevec, C. Taylor, and J. Malik, "Modeling and rendering architecture from photographs: A hybrid geometry- and image based approach", Proceedings of SIGGRAPH 1996, pp. 11-20, 1996.
32. H.Y. Shum, and L.W. He, "Rendering with Concentric Mosaics", Proc. ACM SIGGRAPH, pp. 299-306, August 1999.
33. H.Y. Shum, K.T. Ng, and S.C. Chan, "Virtual Reality using the Concentric Mosaic: Construction, Rendering and Data Compression", Proc. ICIP2000, IEEE International Conference on Image Processing, Vancouver, Canada, September 2000.
34. R. Szeliski, and H.Y. Shum, "Creating Full View Panoramic Image Mosaics and Texture-mapped Models", Proc. ACM SIGGRAPH, pp. 251-258, August 1997.
35. S.E. Chen, "QuickTime VR - An Image-Based Approach to Virtual Environment Navigation", Proc. ACM SIGGRAPH, pp. 29-38, August 1995.
36. C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen, "Unstructured Lumigraph Rendering", Proceedings of SIGGRAPH 2001, pp. 425-432, 2001.
37. S.J. Gortler, R. Grzesczuk, R. Szeliski, and M.F. Cohen, "The Lumigraph", ACM SIGGRAPH '96, pp.43-54, Aug. 1996.
38. J. Shade, S. Gortler, L.W. He, and R. Szeliski, "Layered Depth Images", Proc. SIGGRAPH '98, Orlando, FL, USA, July 1998.
39. D. Wood, D. Azuma, W. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle, "Surface Light Fields for 3D Photography", Proceedings of SIGGRAPH 2000.
40. W.-C. Chen, J.-Y. Bouguet, M.H. Chu, and R. Grzeszczuk, "Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields", ACM Transactions on Graphics. 21 (3), pp. 447-456, 2002.
41. P. Eisert, E. Steinbach, and B. Girod, "Automatic Reconstruction of Stationary 3-D Objects from Multiple Uncalibrated Camera Views", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 10, No. 2, pp. 261-277, March 2000.
42. K.N. Kutulakos, and S.M. Seitz, "A Theory of Shape by Space Carving", University of Rochester, Tech. Report 692, May 1998.
43. T. Matsuyama, and T. Takai, "Generation, Visualization, and Editing of 3D Video", Proc. Symposium on 3D Data Processing Visualization and Transmission, pp.234-245, Padova, Italy, June 2002.
44. T. Matsuyama, X. Wu, T. Takai, and T. Wada, "Real-Time Dynamic 3-D Object Shape Reconstruction and High-Fidelity Texture Mapping for 3-D Video", IEEE Trans. on Circuits and Systems for Video Technology, Vol. 14, No. 3, pp. 357-369, March 2004.
45. S. M. Seitz, and C. R. Dyer, "Photorealistic Scene Reconstruction by Voxel Coloring", International Journal of Computer Vision, 35(2), 1999, pp. 151-173.
46. C. Grünheit, A. Smolic, and T. Wiegand, "Efficient Representation and Interactive Streaming of High-Resolution Panoramic Views", Proc. ICIP2002, IEEE International Conference on Image Processing, Rochester, NY, USA, September 22.-25. 2002.
47. T. Hamaguchi, T. Fujii, Y. Kajiki, and T. Honda, "Real-time View Interpolation System for Multi-View 3D Display", SPIE Electronic Imaging, Vol. 4297A, pp. 212-221, Jan. 2001.
48. T. Kobayashi, T. Fujii, T. Kimoto, and M. Tanimoto, "Interpolation of Ray-Space Data by Adaptive Filtering", SPIE Electronic Imaging 2000, Vol. 3958, pp. 252-259, Jan. 2000.
49. W.H. Leung, and T. Chen, "Compression with Mosaic Prediction for Image-Based rendering Applications", Proc. ICME2000, IEEE International Conference on Multimedia and Expo, New York, NY, USA, July 2000.
50. J. Li, H.Y. Shum, and Y.Q. Zhang, "On the Compression of Image Based Rendering Scene", Proc. ICIP2000, IEEE International Conference on Image Processing, Vancouver, Canada, September 2000.
51. T. Pintaric, U. Neumann, and A. Rizzo, "Immersive Panoramic Video", Proceedings of the 8th ACM International Conference on Multimedia, pp. 493.494, October 2000.
52. C. Zhang, and J. Li, "Compression of Lumigraph with Multiple Reference Frame (MRF) Prediction and Just-in-time Rendering", Proc. DCC2000, IEEE Data Compression Conference, Snowbird, Utah, USA, March 2000.
53. S. Wümlin, E. Lamboray, O. Staadt, and M. Gross, "3D Video Recorder: A System for Recording, Processing and Playing Three-Dimensional Video", Computer Graphics Forum 22 (2), Blackwell Publishing Ltd, Oxford, U.K., pp. 181-193, 2003.
54. S. Wümlin, E. Lamboray, and M. Gross, "3D video fragments: dynamic point samples for real-time free-viewpoint video", Computers and Graphics 28 (1), Special Issue on Coding, Compression and Streaming Techniques for 3D and Multimedia Data, pp. 3-14, Elsevier Ltd, 2004.
55. E. Lamboray, S. Wülin, M. Waschbüch, M. Gross, and H. Pfister, "Unconstrained Free-Viewpoint Video Coding", Proceedings of the IEEE International Conference on Image Processing (ICIP) 2004, Singapore, October 24-27, 2004.
56. C. Fehn, P. Kauff, M. Op de Beeck, F. Ernst, W. Ijsselsteijn, M. Pollefeys, L. Vangool, E. Ofek, and I. Sexton, "An Evolutionary and Optimised Approach on 3D-TV", Proc. of IBC 2002, Int. Broadcast Convention, Amsterdam, Netherlands, Sept. 2002.
57. A. Smolic, K. Mueller, P. Merkle, T. Rein, P. Eisert, and T. Wiegand, "Representation, Coding, and Rendering of 3D Video Objects with MPEG-4", MMSP 2004, IEEE International Workshop on Multimedia Signal Processing, Siena, Italy, September 29.-October 1. 2004.
58. K. Mueller, A. Smolić, M. Kautzner, P. Eisert, and T. Wiegand, "Predictive Compression of Dynamic 3D Meshes", Proceedings of the IEEE International Conference on Image Processing (ICIP) 2005, Genova, Italy, September 11-14,2005.
59. J. Carranza, C. Theobalt, M. Magnor, and H.-P. Seidel, "Free-Viewpoint Video of Human Actors", ACM Trans. on Graphics (special issue SIGGRAPH'03), vol. 22, no. 3, pp. 569-577, July 2003.
60. S. Pastoor, "3D Displays", in O. Schreer, P. Kauff, and T. Sikora (Editors), "3D Video Communication", Wiley, 2005.
61. C. Fehn, "3D TV Broadcasting", in O. Schreer, P. Kauff, and T. Sikora (Editors), "3D Video Communication", Wiley, 2005.
62. ITU-T Recommendation H.264 & ISO/IEC 14496-10 AVC, "Advanced Video Coding for Generic Audio-Visual Services", 2003.