Rate-distortion optimized interactive light field streaming

IEEE Transactions on Multimedia

Volumn 9, Issue 4, 2007, Pages 813-824

  Ramanathan, Prashant ^b   Kalman, Mark ^c   Girod, Bernd ^a,c  

^a IEEE   (United States)

^b AOL   (United States)

^c STANFORD UNIVERSITY   (United States)

Author keywords

Image coding; Image communication; Image based rendering; Interactive streaming; Light field coding; Light field streaming; Light fields; Rate distortion optimized streaming

Indexed keywords

IMAGE BASED RENDERING; INTERACTIVE STREAMING; LIGHT FIELD CODING; LIGHT FIELD STREAMING; LIGHT FIELD STREAMING SYSTEMS; LIGHT FIELDS; PACKET SCHEDULING; RATE-DISTORTION OPTIMIZED STREAMING;

IMAGE CODING; IMAGE COMPRESSION; PACKET NETWORKS; SCHEDULING; SIGNAL DISTORTION; VIDEO STREAMING;

IMAGE COMMUNICATION SYSTEMS;

EID: 34249328874     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2007.893350     Document Type: Article

References (53)

1. S. Chen, "Quicktime VR - an image-based approach to virtual environment navigation," in Proc. SIGGRAPH95, Aug. 1995, pp. 29-38.
2. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. SIGGRAPH96, Aug. 1996, pp. 31-42.
3. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proc. SIGGRAPH96, Aug. 1996, pp. 43-54.
4. T. Fujii, "A Basic Study on the Integrated 3-D Visual Communication," Ph.D. dissertation, Univ. Tokyo, Tokyo, Japan, 1994.
5. T. Fujii, T. Kimoto, and M. Tanimoto, "Ray space coding for 3-D visual communication," in Proc. Picture Coding Symp. PCS-1996, Melbourne, Australia, Mar. 1996, pp. 447-451.
6. H.-Y. Shum and L.-W. He, "Rendering with concentric mosaics," in Proc. SIGGRAPH99, Aug. 1999, pp. 299-306.
7. L. McMillan, "An Image-Based Approach to Three-Dimensional Computer Graphics," Ph.D. dissertation, Univ. North Carolina, Chapel Hill, NC, 1999.
8. J. Shade, S. Gortler, L.-W. He, and R. Szeliski, "Layered depth images," in Proc. SIGGRAPH98, 1998, pp. 231-242, ACM Press.
9. C.-F. Chang, G. Bishop, and A. Lastra, "LDI tree: A hierarchical representation for image-based rendering," in Proc. SIGGRAPH99, 1999, pp. 291-298.
10. P. E. Debevec, C. J. Taylor, and J. Malik, "Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach," in Proc. SIGGRAPH96, 1996, pp. 11-20, ACM Press.
11. P. E. Debevec, G. Borshukov, and Y. Yu, "Efficient view-dependent image-based rendering with projective texture-mapping," in Proc. Eurographics Workship on Rendering, Vienna, Austria, 1998, pp. 105-116.
12. D. N. Wood, D. I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. H. Salesin, and W. Stuetzle, "Surface light fields for 3-D photography," in Proc. SIGGRAPH 00, Aug. 2000, pp. 287-296.
13. M. Levoy and K. Pulli et al., "The Digital Michelangelo project: 3-D scanning of large statues," in Proc. SIGGRAPH 00, Aug. 2000, pp. 131-144.
14. M. Lukacs, "Predictive coding of multi-viewpoint image sets," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing ICASSP 1986, 1986, pp. 521-524.
15. I. Dinstein, G. Guy, J. Rabany, J. Tzelgov, and A. Henik, "On stereo image coding," in Proc. Int. Conf. Pattern Recognition, Nov. 1988, vol. 1, pp. 357-359.
16. I. Dinstein, G. Guy, and J. Rabany, "On the compression of stereo images: Preliminary results," Signal Process., vol. 17, pp. 373-382, 1989.
17. M. G. Perkins, "Data compression of stereopairs," IEEE Trans. Commun., vol. 40, no. 4, pp. 684-696, Apr. 1992.
18. H. Aydinoglu and M.H.H., III, "Compression of multi-view images," in Proc. IEEE Int. Conf. on Image Processing ICIP-1994, 1994, vol. 2, pp. 385-389.
19. H. Aydinoǧlu, F. Kossentini, and M. H. H., III, "A new framework for multi-view image coding," in Proc. Int. Conf. Acoustics, Speech and Signal Processing ICASSP-95, 1995, vol. 4, pp. 2173-2176.
20. M. Magnor and B. Girod, "Data compression for light field rendering," IEEE Trans. Circuits Syst. Video Technol., vol. 10, no. 3, pp. 338-343, Apr. 2000.
21. C. Zhang and J. Li, "Compression of lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering," in Proc. Data Compression Conf. 2000, Snowbird, UT, Mar. 2000, pp. 253-262.
22. P. Ramanathan, M. Flierl, and B. Girod, "Multi-hypothesis disparity-compensated light field compression," in Proc. IEEE Int. Conf. Image Processing ICIP-2001, Oct. 2001.
23. M. Magnor, P. Eisert, and B. Girod, "Model-aided coding of multi-viewpoint image data," in Proc. IEEE Int. Conf. Image Processing ICIP-2000, Vancouver, BC, Canada, Sep. 2000, vol. 2, pp. 919-922.
24. M. Magnor, P. Ramanathan, and B. Girod, "Multi-view coding for image-based rendering using 3-D scene geometry," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1092-1106, Nov. 2003.
25. C.-L. Chang, X. Zhu, P. Ramanathan, and B. Girod, "Shape adaptation for light field compression," in Proc. IEEE Int. Conf. Image Processing ICIP-2003, Barcelona, Spain, Sep. 2003.
26. I. Peter and W. Strasser, "The wavelet stream: Progressive transmission of compressed light field data," in Proc. IEEE Visualization 1999 Late Breaking Hot Topics, Oct. 1999, pp. 69-72.
27. M. Magnor, A. Endmann, and B. Girod, "Progressive compression and rendering of light fields," in Proc. Vision, Modelling and. Visualization 2000, Nov. 2000, pp. 199-203.
28. C.-L. Chang, X. Zhu, P. Ramanathan, and B. Girod, "Inter-view wavelet compression of light fields with disparity-compensated lifting," in Proc. SPIE Visual Comm. and Image Processing VCIP-2003, Lugano, Switzerland, Jul. 2003.
29. B. Girod, C.-L. Chang, P. Ramanathan, and X. Zhu, "Light field compression using disparity-compensated lifting," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing 2003, Hong Kong, China, Apr. 2003, vol. IV, pp. 761-764.
30. X. Tong and R. M. Gray, "Interactive rendering from compressed light fields," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1080-1091, Nov. 2003.
31. ITU-T, ISE/IEC 15444-1:2000, Information Technology: JPEG 2000 Image Coding System 2002.
32. D. Taubman and R. Rosenbaum, "Rate-distortion optimized interactive browsing of JPEG2000 images," in Proc. IEEE Int. Conf. Image Processing ICIP-2003, Barcelona, Spain, Sep. 2003, vol. 2, pp. 765-768.
33. D. S. Cruz, T. Ebrahimi, M. Larsson, J. Askelof, and C. Christopoulos, "Region of interest coding in JPEG2000 for interactive client/server applications," in Proc. Multimedia Signal Processing Workshop MMSP-1999, Copenhagen, Denmark, Sep. 1999, pp. 389-394.
34. I. Peter and W. Strasser, "The wavelet stream: Interactive multi resolution light field rendering," in Proc. 12th Eurographics Workshop on Rendering, Jun. 2001, pp. 262-273.
35. C.-L. Chang and B. Girod, "Rate-distortion optimized interactive streaming for scalable bitstreams of light fields," in Proc. SPIE Visual Comm. and Image Processing VCIP-2004, Jan. 2004.
36. _, "Receiver-based rate-distortion optimized interactive streaming for scalable bitstreams of light fields," in Proc. IEEE Int. Conf. Multimedia Expo ICME-2004, Jun. 2004.
37. C. Zhang and J. Li, "Interactive browsing of 3-D environment over the Internet," in Proc. SPIE Visual Comm. and Image Processing VCIP-2001, San Jose, CA, Jan. 2001, pp. 509-520.
38. _, "Compression and rendering of concentric mosaics with reference block codec (RBC)," in Proc. SPIE Visual Comm. and Image Processing VCIP-2000, 2000, pp. 43-54.
39. D. Gotz and K. Mayer-Patel, "A general framework for multidimensional adaptation," in Proc. ACM Multimedia 2004, 2004, pp. 612-619.
40. _, "A framework for scalable delivery of digitized spaces," Int. J. Digital Libraries, vol. 5, no. 3, pp. 205-218, May 2005.
41. A. Albanese, J. Blömer, J. Edmonds, M. Luby, and M. Sudan, "Priority encoding transmission," IEEE Trans. Inform. Theory, vol. 42, no. 6, pp. 1737-1744, Nov. 1996.
42. M. Khansari, A. Jalali, E. Dubois, and P. Mermelstein, "Low bit-rate video transmission over fading channels for wireless microcellular systems," IEEE Trans. Circuits Syst. Video Technol., vol.6, no. 1, pp. 1-11, Feb. 1996.
43. H. Liu and M. E. Zarki, "Performance of H.263 video transmission over wireless channels using hybrid ARQ," IEEE J. Select, Areas Commun., vol. 15, no. 9, pp. 1775-86, Dec. 1997.
44. P. A. Chou and Z. Miao, "Rate-distortion optimized streaming of packetized media," IEEE Trans. Multimedia, vol. 8, no. 2, pp. 390-404, Apr. 2006.
45. P. A. Chou and A. Seghal, "Rate-distortion optimized receiver-driven streaming over best-effort networks," in Packet Video Workshop, Pittsburgh, PA, Apr. 2002.
46. ITU-T, One-Way Transmission Time ITU-T Recommend. G.I 14, Feb. 1996.
47. R. Azuma and G. Bishop, "A frequency-domain analysis of head-motion prediction," in Proc. SIGGRAPH95, Aug. 1995, pp. 401-408.
48. S. K. Singhal and D. R. Cheriton, "Exploiting position history for efficient remote rendering in networked virtual reality," Presence: Teleoperators Virtual Environ., vol. 4, pp. 169-193, Spring 1995.
49. P. Ramanathan, Light Field Viewer Software [Online]. Available: http://www.ivms.stanford.edu/pramanat/lfviewer.html 2003
50. P. Ramanathan, M. Kalman, and B. Girod, "Rate-distortion optimized streaming of compressed light fields," in Proc. IEEE Int. Conf. on Image Processing ICIP-2003, Barcelona, Spain, Sep. 2003, vol. 3, pp. 277-280.
51. P. Ramanathan and B. Girod, "Theoretical analysis of the rate-distortion performance of a light field streaming system," in Proc. Picture Coding Symp. (PCS-2004), San Francisco, CA, Dec. 2004.
52. _, "Rate-distortion analysis of random access for compressed light fields," in Proc. IEEE Int. Conf. Image Processing ICIP-2004, Singapore, Oct. 2004.
53. P. Ramanathan, "Compression and Interactive Streaming of Light Fields," Ph.D. dissertation, Stanford University, Stanford, CA, 2005.