Interactive rendering from compressed light fields

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 13, Issue 11, 2003, Pages 1080-1091

  Tong, Xin ^a   Gray, Robert M ^a  

^a Stanford University   (United States)

Author keywords

Interactive rendering; Light field compression; Vector quantization (VQ)

Indexed keywords

ALGORITHMS; CACHE MEMORY; CAMERAS; COMPUTATIONAL COMPLEXITY; COMPUTATIONAL GEOMETRY; COMPUTER SIMULATION; DECODING; IMAGE CODING; IMAGE RECONSTRUCTION; LAPTOP COMPUTERS; PHOTOMETRY; THREE DIMENSIONAL COMPUTER GRAPHICS;

INTERACTIVE RENDERING; LIGHT FIELD COMPRESSION; TREE STRUCTURED VECTOR QUANTIZATION;

VECTOR QUANTIZATION;

EID: 0242468121     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2003.817626     Document Type: Article

References (77)

1. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. ACM Siggraph, 1996, pp. 31-42.
2. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proc. ACM Siggraph, 1996, pp. 43-54.
3. E. H. Adelson and J. R. Bergen, "The plenoptic function and the elements of early vision," in Computation Models of Visual Processing, M. Landy and J. A. Movshon, Eds. Cambridge: MIT Press, 1991.
4. S. E. Chen, "Quicktime VR - An image-based approach to virtual environment navigation," in Proc. ACM Siggraph, 1995, pp. 29-38.
5. ____, "View interpolation for image synthesis," in Proc. ACM Siggraph, 1993, pp. 279-288.
6. N. Greene and M. Kass, "Approximating Visibility with Environment Maps," Apple Computer, Inc., Cupertino, CA, Tech. Rep. 41, 1993.
7. L. McMillan and G. Bishop, "Head-tracked stereo-scopic display using image warping," Stereosc. Displ. Virtual Reality Syst. II, vol. 2409, pp. 21-30, 1995.
8. ____, "Plenoptic modeling: An image-based rendering system," in Proc. ACM Siggraph, 1995, pp. 39-46.
9. T. Kanade, P. J. Narayanan, and P. W. Rander, "Virtualized reality: Concepts and early results," Proc. IEEE Workshop Representation of Visual Scenes, pp. 69-76, June 1995.
10. S. Seitz and C. Dyer, "Physically-valid view synthesis by image interpolation," Proc. IEEE Workshop Representation of Visual Scenes, pp. 18-25, June 1995.
11. H. Y. Shum and L. W. He, "Rendering with concentric mosaics," in Proc. ACM Siggraph, Aug. 1999, pp. 299-306.
12. M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz, D. Koller, L. Pereira, M. Ginzton, S. Anderson, J. Davis, J. Ginsberg, J. Shade, and D. Fulk, "The digital michelangelo project: 3d scanning of large statues," in Proc. ACM Siggraph, 2000, pp. 131-144.
13. "ITU-T Recommendation H.263: Video Coding for Low Bit Rate Communication," ITU-T, 1998.
14. M. Magnor and B. Girod, "Data compression for light field rendering," IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 338-343, Apr. 2000.
15. M. Magnor, "Geometry-Adaptive Multi-View Coding Techniques for Image-Based Rendering," Ph.D. dissertation, Univ. of Erlangen, Nuremberg, Germany, 2000.
16. T. Wiegand, M. Flierl, and B. Girod, "Entropy-constrained design of quadtree video coding schemes," Proc. IEE Int. Conf. Image Processing and its Applications (IPA '97), pp. 36-40, July 1997.
17. C. Zhang and J. Li, "Compression of lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering," in Proc. Data Compression Conf., Snowbird, UT, Mar. 2000, pp. 253-262.
18. G. Wolberg, Digital Image Warping. New York: IEEE Computer Society Press, 1990.
19. J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes, Computer Graphics: Principles and Practice. Reading, MA: Addison-Wesley, 1990.
20. M. E. Lukacs, "Predictive coding of multi-viewpoint image sets," Proc. ICASSP'86, pp. 521-24, 1986.
21. I. Dinstein, G. Guy, and J. Rabany, "On the compression of stereo images: Preliminary results," Signal Processing, vol. 17, pp. 373-382, 1989.
22. M. Perkins, "Data compression of stereopairs," IEEE Trans. Comm., vol. 40, pp. 684-696, Apr. 1992.
23. X. Tong and R. M. Gray, "Coding of multiview images for immersive viewing," Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, vol. 4, pp. 1879-1882, June 2000.
24. ____, "Interactive view synthesis from compressed light fields," Proc. IEEE Int. Conf. Image Processing, vol. 2, pp. 85-88, Oct. 2001.
25. T. Beier and S. Neely, "Feature-based image metamorphosis," in Proc. ACM Siggraph, July 1992, pp. 35-42.
26. S. Benton, "Survey of holographic stereograms," in Proc. SPIE Processing and Display of Three-Dimensional Data, vol. 367, 1983, pp. 15-19.
27. V. Bhaskaran and K. Konstantinides, Image and Video Compression Standards: Algorithms and Architectures. Norwell, MA: Kluwer, 1995.
28. J. F. Blinn and M. E. Newell, "Texture and reflection in computer generated images," Comm. ACM, vol. 19, no. 10, pp. 542-547, 1976.
29. P. J. Burt and E. H. Adelson, "The laplacian pyramid as a compact image code," IEEE Trans. Comm., vol. COM-31, pp. 552-540, Apr. 1983.
30. R. Bolles, H. Baker, and D. Marimont, "Epipolar-plane image analysis: An approach to determining structure from motion," Int. J. Comp. Vis., vol. 1, no. 1, pp. 7-55, 1987.
31. E. Camahort, A. Lerios, and D. Fussell, "Uniformly sampled light fields," in Proc. Eurographics Rendering Workshop, Saint Etienne, France, June 1997, pp. 117-130.
32. E. Camahort and D. Fussell, "A Geometric Study of Light Field Representations," Univ. Texas at Austin, Dept. Comp. Sci., Austin, 2000.
33. J. X. Chai, X. Tong, S. C. Chan, and H. Y. Shum, "Plenoptic sampling," in Proc. ACM Siggraph, July 2000, pp. 307-318.
34. P. Debevec, C. Taylor, and J. Malik, "Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach," in Proc. ACM Siggraph, Aug. 1996, pp. 11-20.
35. P. Debevec, "Rendering synthetic objects into real scenes," in Proc. ACM Siggraph, Aug. 1998, pp. 189-198.
36. M. Flierl, T. Wiegand, and B. Girod, "A locally optimal design algorithm for block-based multi-hypothesis motion-compensated prediction," in Proc. Data Compression Conf., Snowbird, UT, Apr. 1998, pp. 239-248.
37. H. Fuchs G. Bishop, K. Arthur, L. McMillan, R. Bajcsy, S. W. Lee, H. Farid, and T. kanade, "Virtual space tele-conferencing using a sea of cameras," in Proc. 1st Int. Conf. Medical Robotics and Computer Assisted Surgery, 1994, pp. 161-167.
38. A. Gershun, "The light field," J. Math. Phys., vol. 18, pp. 51-151, 1939.
39. B. Girod, "Efficiency analysis of multihypothesis motion-compensated prediction for video coding," IEEE Trans. Image Processing, vol. 9, pp. 173-183, Feb. 2000.
40. N. Greene, "Environment mapping and other applications of world projections," IEEE Comp. Graphics Applicat., vol. 6, pp. 21-29, Nov. 1986.
41. M. Halle, "Holographic stereograms as discrete imaging systems," Pract. Hologr., vol. 2176, pp. 73-84, 1994.
42. D. A. Patterson and J. L. Hennessy, Computer Architecture: A Quantitative Approach, 2nd ed. San Francisco, CA: Morgan Kaufmann, 1996.
43. I. Ihm, S. Park, and R. K. Lee, "Rendering of spherical light fields," in Proc. 5th Pacific Conf. Computer Graphics and Applications, Seoul, Korea, Oct. 1997, pp. 59-68.
44. "Digital Compression and Coding of Continuous-Tone Still Images - Part 1: Requirements and Guidelines," ISO, Geneva, Switzerland, ISO/IEC JTC1 CD 10918, 1993.
45. A. Katayama K. Tanaka, T. Oshino, and H. Tamura, "Viewpoint-dependent stereoscopic display using interpolation of multi-viewpoint images," Stereoscop. Displ. Virtual Reality Syst. II, vol. 2409, pp. 11-20, 1995.
46. P. Lalonde and A. Fournier, "Interactive rendering of wavelet projected light fields," in Proc. Graphics Interface, Kingston, Canada, June 1999, pp. 107-114.
47. S. Laveau and O. D. Faugeras, "3-D Scene Representation as a Collection of Images and Fundamental Matrices," INRIA, Cedex, France, 2205, 1994.
48. J. Lengyel and J. Snyder, "Rendering with coherent layers," in Proc. ACM Siggraph, Aug. 1997, pp. 233-242.
49. M. Lucente and T. A. Galyean, "Rendering interactive holographic images," in Proc. ACM Siggraph, Aug. 1995, 387-394.
50. G. Miller, "Volumetric hyper-reality: A computer graphics holy grail for the 21st century?," in Proc. Graphics Interface, 1995, pp. 56-64.
51. G. S. P. Miller, S. M. Rubin, P. M. Hubbard, J. Dana, and J. Woodfill, "Head-tracked light field movies: A new multimedia data type," in EuroGraphics '99 Multimedia Workshop, 1999.
52. "Generic Coding of Moving Pictures and Associated Audio," ISO, Geneva, Switzerland, ISO/IEC JTC1 CD 13818, 1994.
53. S. Na and D. L. Neuhoff, "Bennett's integral for vector quantizers," IEEE Trans. Inform. Theory, vol. 41, pp. 886-900, July 1995.
54. K. Nishino, Y. Sato, and K. Ikeuchi, "Eigen-texture method: Appearance compression based on 3D model," in Proc. Computer Vision and Pattern Recognition, vol. 1, June 1999, pp. 618-624.
55. K. Rose, H. Khalil, and S. Regunathan, "Open-loop design of predictive vector quantizers for video coding," Proc. IEEE Int. Conf. Image Processing, ICIP'98, vol. 3, pp. 953-957, Oct. 1998.
56. H. Khalil, I. Rose, and S. Regunathan, "The asymptotic closed-loop approach to predictive vector quantizer design," IEEE Trans. Image Processing, vol. 10, pp. 15-23, Jan. 2001.
57. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1996.
58. Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design," IEEE Trans. Comm., vol. COM-28, pp. 84-95, Jan. 1980.
59. X. Tong, Disparity Compensated Compression of Light Fields. Stanford, CA: Univ. California, 1998.
60. P. C. Chang and R. M. Gray, "Gradient algorithms for designing predictive vector quantizers," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-34, pp. 679-690, Aug. 1986.
61. V. Cuperman and A. Gersho, "Vector predictive coding of speech at 16 kbits/s," IEEE Trans. Commun., vol. COM-33, pp. 685-696, July 1985.
62. M. Magnor and B. Girod, "Hierarchical coding of light fields with disparity maps," Proc. IEEE Int. Conf. Image Processing, pp. 334-338, Oct. 1999.
63. ____, "Adaptive block-based light field coding," in Proc. 3rd Int. Workshop on Synthetic and Natural Hybrid Coding and Three-Dimensional Imaging, Santorini, Greece, Sept. 1999, pp. 140-143.
64. M. T. Orchard and G. J. Sullivan, "Overlapped block motion compensation: An estimation-theoretic approach," IEEE Trans. Image Processing, vol. 3, pp. 693-699, Sept. 1994.
65. P. Rademacher and G. Bishop, "Multiple-center-of-projection images," in Proc. ACM Siggraph, Aug. 1998, pp. 199-205.
66. K. R. Rao and P. Yip, Discrete Cosine Transform - Algorithms, Advantages, Applications. San Diego, CA: Academic, 1990.
67. J. Shade, S. Gortler, L. W. He, and R. Szeliski, "Layered depth images," in Proc. ACM Siggraph, Aug. 1998, pp. 231-242.
68. A. M. Tekalp, Digital Video Processing. Upper Saddle River, NJ: Prentice-Hall, 1995.
69. X. Tong and R. M. Gray, "Compression of light fields using disparity compensation and vector quantization," in Proc. IASTED Conf. Computer Graphics and Imaging, Palm Springs, CA, Oct. 1999, pp. 300-305.
70. G. Tsang, S. Ghali, E. Fiume, and A. N. Venetsanopoulos, "A novel parameterization of the light field," Proc. IEEE Image and Multidimensional Digital Signal Processing, pp. 319-322, July 1998.
71. A. B. Watson, "Perceptual optimization of DCT color quantization matrices," Proc. IEEE Int. Conf. Image Processing, vol. 1, pp. 100-104, Nov. 1994.
72. A. J. Ahumada and A. B. Watson, "A visual detection model for DCT coefficient quantization," Technology, vol. 2003, no. 2, pp. 404-415, 1993.
73. B. Wilburn, "The light field video camera," presented at the Proc. SPIE Electronic Imaging Media Processors, San Jose, CA, 2002.
74. T. Wong, P. Heng, S. Or, and W. Ng, "Image-based rendering with controllable illumination," in Proc. Eurographics Rendering Workshop, Saint Etienne, France, June 1997, pp. 13-22.
75. D. Wood, D. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle, "Surface light fields for 3d photography," in Proc. ACM Siggraph, July 2000, pp. 287-296.
76. S. W. Wu and A. Gersho, "Joint estimation of forward and backward motion vectors for interpolative prediction of video," IEEE Trans. Image Processing, vol. 4, pp. 684-689, Sept. 1994.
77. T. Wiegand, X. Zhang, and B. Girod, "Long-term memory motion-compensated prediction," IEEE Trans. Circuits Syst. Video Technol., vol. 9, pp. 70-84, Feb. 1999.