Multi-view coding for image-based rendering using 3-D scene geometry

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 13, Issue 11, 2003, Pages 1092-1106

  Magnor, Marcus ^a   Ramanathan, Prashant ^b   Girod, Bernd ^b  

^a MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Geometry coding; Image based rendering (IBR); Light field compression; Model based coding; Multi view coding analysis; Multi view compression

Indexed keywords

ALGORITHMS; CAMERAS; COMPUTATIONAL GEOMETRY; COMPUTER SIMULATION; DATA ACQUISITION; IMAGE CODING; IMAGE QUALITY; IMAGE RECONSTRUCTION; OPTICAL FLOWS; OPTIMIZATION; THREE DIMENSIONAL COMPUTER GRAPHICS; WAVELET TRANSFORMS;

GEOMETRY CODING; IMAGE BASED RENDERING; LIGHT FIELD COMPRESSION; MODEL BASED CODING; MULTI-VIEW CODING; MULTI-VIEW COMPRESSION; TEXTURE BASED CODING;

IMAGE COMPRESSION;

EID: 0242551634     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2003.817630     Document Type: Article

References (52)

1. S. E. Chen, "Quicktime VR - An image-based approach to virtual environment navigation," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'95), Los Angeles, CA, Aug. 1995, pp. 29-38.
2. L. McMillan and G. Bishop, "Plenoptic modeling: An image-based rendering system," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'95), Los Angeles, CA, Aug. 1995, pp. 39-46.
3. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'96), New Orleans, LA, Aug. 1996, pp. 31-42.
4. S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, "The lumigraph," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'96), New Orleans, LA, Aug. 1996, pp. 43-54.
5. P. Debevec, C. Taylor, and J. Malik, "Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'96), New Orleans, LA, Aug. 1996, pp. 11-20.
6. R. Szeliski and H.-Y. Shum, "Creating full view panoramic mosaics and environment maps," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'97), Los Angeles, CA, Aug. 1997, pp. 251-258.
7. H.-Y. Shum and L.-W. He, "Rendering with concentric mosaics," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'99), Los Angeles, CA, Aug. 1999, pp. 299-306.
8. D. Wood, D. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle, "Surface light fields for 3D photography," in Proc. ACM Conf. Computer Graphics (SIGGRAPH-2000), New Orleans, LA, July 2000, pp. 287-296.
9. C. Buehler, M. Bosse, L. McMillan, S. Cohen, "Unstructured lumigraph rendering," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'01), Los Angeles, CA, Aug. 2001, pp. 425-432.
10. J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, "Plenoptic sampling," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'00), New Orleans, LA, Aug. 2000, pp. 307-318.
11. G. Miller, S. Rubin, and D. Ponceleon, "Lazy decompression of surface light fields for precomputed global illumination," in Proc. Eurographics Rendering Workshop'98, Vienna, Austria, Oct. 1998, pp. 281-292.
12. I. Ihm, S. Park, and R. K. Lee, "Rendering of spherical light fields," in Proc. 5th Pacific Conf. Computer Graphics and Applications, Seoul, Korea, Oct. 1997, pp. 59-68.
13. P. Lalonde and A. Fournier, "Interactive rendering of wavelet projected light fields," in Proc. Graphics Interface'99, Kingston, ON, Canada, June 1999, pp. 107-114.
14. Y. Wu, L. Luo, J. Li, and Y.-Q. Zhang, "Rendering of 3D wavelet compressed concentric mosaic scenery with progressive inverse wavelet synthesis (PIWS)," in Proc. SPIE Visual Communications and Image Processing (VCIP-2000), vol. 1, Perth, Australia, June 2000, pp. 31-42.
15. J. Li, H. Y. Shum, and Y. Q. Zhang, "On the compression of image based rendering scene," in Proc. IEEE Int. Conf. Image Processing (ICIP-00), vol. 2, Sept. 2000, pp. 21-24.
16. I. Peter and W. Strasser, "The wavelet stream: Interactive multi resolution light field rendering," in Proc. 12th Eurographics Workshop Rendering, London, U.K., June 2001, pp. 262-273.
17. H. Aydinoglu and M. Hayes, "Compression of multi-view images," in Proc. IEEE Int. Conf. Image Processing (ICIP'94), vol. 2, Nov. 1994. pp. 385-389.
18. C. Zhang and J. Li, "Compression and rendering of concentric mosaics with reference block codec (RBC)," in Proc. SPIE Visual Communications and Image Processing (VCIP-2000), vol. 1, Perth, Australia, June 2000, pp. 43-54.
19. X. Tong and R. M. Gray, "Interactive view synthesis from compressed light fields," in Proc. IEEE International Conference on Image Processing (ICIP-2001), Oct. 2001, pp. 85-88.
20. M. Magnor and B. Girod, "Data compression for light field rendering," IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 338-343, Apr. 2000.
21. H. Schirmacher, W. Heidrich, and H-P. Seidel, "High-quality interactive lumigraph rendering through warping," in Proc. Graphics Interface 2000, Montreal, QC, Canada, May 2000, pp. 87-94.
22. B. Girod and M. Magnor, "Two approaches to incorporate approximate geometry into multi-view image coding," in Proc. IEEE Int. Conf. Image Processing (ICIP-2000), vol. 2, Sept. 2000, pp. 5-8.
23. P. Debevec, Y. Yu, and G. Boshokov, "Efficient view-dependent image-based rendering with projective texture-mapping," Univ. California, Berkeley, CA, Rep. CSD-98-1003, vol. 20, 1998.
24. D. Cohen-Or, Y. Mann, and S. Fleishman, "Deep compression for streaming texture intensive animations," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'99), Los Angeles, CA, Aug. 1999, pp. 261-268.
25. M. Magnor and B. Girod, "Model-based coding of multi-viewpoint imagery," in Proc. SPIE Visual Communication and Image Processing (VCIP-2000), vol. 1, Perth, Australia, June 2000, pp. 14-22.
26. S. M. Seitz and C. M. Dyer, "View morphing," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'96), New Orleans, LA, Aug. 1996, pp. 21-30.
27. M. Magnor, P. Eisert, and B. Girod, "Model-aided coding of multi-viewpoint image data," in Proc. IEEE Int. Conf. Image Processing (ICIP-2000), vol. 2, Sept. 2000, pp. 919-922.
28. P. Eisert, "Model-based camera calibration using analysis by synthesis techniques," in Proc. Vision, Modeling, and Visualization (VMV'02), Erlangen, Germany, Nov. 2002, pp. 307-314.
29. S. Seitz and C. Dyer, "Photorealistic scene reconstruction by voxel coloring," in Proc. IEEE Computer Vision and Pattern Recognition (CVPR'97), June 1997, pp. 1067-1073.
30. K. Kutulakos, "Shape from the light field boundary," in Proc. IEEE Computer Vision and Pattern Recognition (CVPR'97), June 1997, pp.53-59.
31. K. Kutulakos and S. Seitz, "A theory of shape by space carving," in Proc. IEEE Int. Conf. Computer Vision (ICCV'99), vol. 1, Sept. 1999, pp. 307-314.
32. P. Eisert, E. Steinbach, and B. Girod, "Automatic reconstruction of stationary 3-D objects from multiple uncalibrated camera views," IEEE Trans. Circuits Syst. Video Technol. (Special Issue on 3D Video Technology), vol. 10, pp. 261-277, Mar. 2000.
33. O. Faugeras, Three-Dimensional Computer Vision. Cambridge, MA: MIT Press, 1993.
34. R. Koch and L. Van Gool, Eds., 3D Structure from Multiple Images of Large-Scale Environments. Berlin, Germany: Springer-Verlag, 1998, vol. 1506, Lecture Notes in Computer Science.
35. M. Pollefeys, R. Koch, M. Vergauwen, and L. van Gool, Metric 3D Surface Reconstruction from Uncalibrated Image Sequences. Berlin, Germany: Springer-Verlag, 1998, vol. 1506, Lecture Notes in Computer Science, pp. 139-154.
36. W. E. Lorensen and H. E. Cline, "Marching cubes: A high resolution 3D surface construction algorithm," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'87), Anaheim, CA, July 1987, pp. 163-169.
37. H. Hoppe, "Progressive meshes," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'96), New Orleans, CA, Aug. 1996, pp. 99-108.
38. J. Popović and H. Hoppe, "Progressive simplicial complexes," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'97), Los Angeles, CA, Aug. 1997, pp. 217-224.
39. G. Taubin, A. Gueziec, W. Horn, and F. Lazarus, "Progressive forest split compression," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'98), Orlando, FL, July 1998, pp. 123-132.
40. R. Pajarola and J. Rossignac, "Compressed progressive meshes," Georgia Inst. Technol., Atlanta, GA, Rep. GIT-GVU-99-05, 1999.
41. M. Magnor and B. Girod, "Fully embedded coding of triangle meshes," in Proc. Vision, Modeling, and Visualization (VMV'99), Erlangen, Germany, Nov. 1999, pp. 253-259.
42. S. Chen and L. Williams, "View interpolation for image synthesis," in Proc. ACM Conf. Computer Graphics (SIGGRAPH'93), Anaheim, CA, Aug. 1993, pp. 279-288.
43. M. Magnor and B. Girod, "Hierarchical coding of light fields with disparity maps," in Proc. IEEE Int. Conf. Image Processing (ICIP'99), vol. 3, Oct. 1999, pp. 334-338.
44. W. Heidrich, H. Schirmacher, H. Küch, and H.-P. Seidel, "A warping-based refinement of lumigraphs," in Proc. 6th Int. Conf. Central Europe Computer Graphics and Visualization, Plzen, Czech Republic, Feb. 1998, pp. 102-109.
45. M. Magnor, Geometry-adaptive Multi-View Coding Techniques for Image-based Rendering. Aachen, Germany: Shaker-Verlag, 2001.
46. A. Said and W. Pearlman, "A new, fast and efficient image codec based on set partitioning in hierarchical trees," IEEE Trans. Circuits Syst. Video Technol., vol. 6, pp. 243-250, June 1996.
47. M. Magnor, A. Endmann, and B. Girod, "Progressive compression and rendering of light fields," in Proc. Vision, Modeling, and Visualization (VMV-000), Saarbrücken, Germany, Nov. 2000, pp. 199-203.
48. P. Ramanathan and B. Girod, "Theoretical analysis of geometry inaccuracy for light field compression," in Proc. IEEE Int. Conf. Image Processing ICIP-02, Sept. 2002, pp. 229-232.
49. B. Girod, "The efficiency of motion-compensating prediction for hybrid coding of video sequences," IEEE J. Select. Areas Commun., vol. SAC-5, pp. 1140-1154, Aug. 1987.
50. ____, "Efficiency analysis of multihypothesis motion-compensated prediction for video coding," IEEE Trans. Image Processing, vol. 9, pp. 173-183, Feb. 2000.
51. N. S. Jayant and P. Noll, Digital Coding of Waveforms. Englewood Cliffs, NJ: Prentice-Hall, 1984.
52. M. Magnor and B. Girod, "Sensitivity of image-based and texture-based multi-view coding to model accuracy," in Proc. IEEE Int. Conf. Image Processing (ICIP-2001), vol. 3, Oct. 2001, pp. 98-101.