Image-based rendering and synthesis

IEEE Signal Processing Magazine

Volumn 24, Issue 6, 2007, Pages 22-33

  Chan, S C ^a,b   Shum, Heung Yeung ^a   Ng, King To ^b  

^a MICROSOFT   (United States)

^b UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SIMULATION; IMAGE RECONSTRUCTION; TELEVISION; THREE DIMENSIONAL COMPUTER GRAPHICS; VIRTUAL REALITY; VISUALIZATION;

IMAGE-BASED RENDERING; MULTIVIEW IMAGING; THREE-DIMENSIONAL TELEVISION; VIDEO GAMES;

VOLUME RENDERING;

EID: 85032751454     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2007.905702     Document Type: Article

References (65)

1. E.H. Adelson and J. Bergen, "The plenoptic function and the elements of early vision," in Computational Models of Visual Processing. Cambridge, MA: MIT Press, pp. 3-20, 1991.
2. H.Y. Shum, S.B. Kang, and S.C. Chan, "Survey of image-based representations and compression techniques," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1020-1037, Nov. 2003.
3. C. Zhang and T. Chen, "A survey on image-based rendering - Representation, sampling and compression," in EURASIP Signal Processing: Image Commun., vol. 19, no. 1, pp. 1-28, Jan. 2004.
4. H.Y. Shum, S.C. Chan, and S.B. Kang, Image-Based Rendering. New York: Springer-Verlag, 2006.
5. L. McMillan and G. Bishop, "Plenoptic modeling: An image-based rendering system," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 1995, pp. 39-46.
6. R. Szeliski and H.Y. Shum, "Creating full view panoramic image mosaics and environment maps," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 1997, pp. 251-258.
7. H.Y. Shum and L.W. He, "Rendering with concentric mosaics," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 1999, pp. 299-306.
8. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 1996, pp. 31-42.
9. S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen, "The lumigraph," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 1996, pp. 43-54.
10. H.Y. Shum, J. Sun, S. Yamazaki, Y. Lin, and C.K. Tang, "Pop-up light field: An interactive image-based modeling and rendering system," ACM Trans. Graphics, vol. 23, no. 2, pp. 143-162, Apr. 2004.
11. C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 2004, pp. 600-609.
12. S.C. Chan, K.T. Ng, Z.F. Gan, K.L. Chan, and H.Y. Shum, "The plenoptic videos," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 12, pp. 1650-1659, Dec.2005.
13. B. Wilburn, M. Smulski, H.H. Lee, and M. Horowitz, "The light field video camera," in Proc. SPIE Electronic Imaging: Media Processors'2002, vol. 4674, Jan. 2002, pp. 29-36.
14. T. Naemura, J. Tago and H. Harashima, "Real-time video-based modeling and rendering of 3D scenes," IEEE Comput. Graphics Applicat., vol. 22, no. 2, pp. 66-73, Mar.-Apr. 2002.
15. J.C. Yang, M. Everett, C. Buehler, and L. McMillan, "A real-time distributed light field camera," in Proc. Eurographics Workshop on Rendering, 2002, pp. 77-86.
16. B. Goldlücke, M. Magnor, and B. Wilburn, "Hardware-accelerated dynamic light field rendering," in Proc. VMV'2002, pp. 455-462.
17. C. Zhang and T. Chen, "Active rearranged capturing of image-based rendering scenes-Theory and practice," in IEEE Trans. Multimedia, vol. 9, no. 3, pp. 520-531, Apr. 2007.
18. K.T. Ng, S.C. Chan, and H.Y. Shum, "The data compression and transmission aspects of panoramic videos," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 1, pp. 82-95, Jan. 2005.
19. M. Agrawala, R. Ramamoorthi, A. Heirich, and L. Moll, "Efficient image-based methods for rendering soft shadows," in Proc. SIGGRAPH (ACM Trans. Graphics), 2000, pp. 372-384.
20. R. Ng, R. Ramamoorthi, and P. Hanrahan, "Triple product wavelet integrals for all-frequency relighting," in Proc. SIGGRAPH (ACM Trans. Graphics), 2004, pp. 477-487.
21. P. Sloan, J. Kautz, and J. Snyder, "Precomputed radiance transfer for real-time rendering in dynamic, low- frequency lighting environment," in Proc. SIGGRAPH (ACM Trans. Graphics), 2002, pp. 527-536.
22. K. Zhou, Y. Hu, S. Lin, B. Guo, and H.Y. Shum, "Precomputed shadow fields for dynamic scenes," in Proc. SIGGRAPH (ACM Trans. Graphics) 2005, pp. 1196-1201.
23. Z.F. Gan, S.C. Chan, K.T. Ng, and H.Y. Shum, "An object-based approach to plenoptic videos," in Proc. IEEE Int. Symp. Circuits and Systems May 2005, pp. 3435-3438.
24. S.C. Chan, Z.F. Gan, K.T. Ng, and H.Y. Shum, "An object-based approach to a class of dynamic image-based representations," submitted for publication.
25. J. Shade, S. Gortler, L.W. He, and R. Szeliski, "Layered depth images," in Proc. SIGGRAPH (ACM Trans. Graphics), Orlando, FL, July 1998, pp. 231-242.
26. C. Chang, G. Bishop, and A. Lastra, "LDI tree: A hierarchical representation for image-based rendering," in Proc. SIGGRAPH (ACM Trans. Graphics), Aug. 1999, pp. 291-298.
27. P.E. Debevec, Y. Yu, and G. Borshukov, "Efficient view-dependent image-based rendering with projective texture-mapping," in Proc. Eurographics Workshop on Rendering, 1998, pp. 150-116.
28. L. Wang, W. Wang, J. Dorsey, X. Yang, B. Guo, and H.Y. Shum, "Real-time rendering of plant leaves," in Proc. SIGGRAPH (ACM Trans. Graphics) July 2005, pp. 712-719.
29. J.X. Chai, X. Tong, S.C. Chan and H.Y. Shum, "Plenoptic sampling," in Proc. SIGGRAPH (ACM Trans. Graphics), July 2000, pp. 307-318.
30. Y.Y. Chuang, B. Curless, D.H. Salesin, and R. Szelishi, "A Bayesian approach to digital matting," in Proc. IEEE Conf. CVPR, vol. 5, 2001, pp. 264-271.
31. Q. Wu, K.T. Ng, S.C. Chan and H.Y. Shum, "On object-based compression for a class of dynamic image-based representations," in Proc. IEEE Int. Conf. Image Processing, Italy, vol. 3, Sept. 11-14, 2005, pp. 405-408.
32. Y. Li, J. Sun, C.K. Tang, and H.Y. Shum, "Lazy snapping," in Proc. SIGGRAPH (ACM Trans. Graphics), 2004, pp. 303-308.
33. S.J. Osher and JA. Sethian, "Fronts propagation with curvature dependent speed: Algorithms based on Hamilton-Jacobi formulations," J. Compute. Phys., vol. 79, no. 1, pp. 12-49, 1988.
34. T.F. Chan and L.A. Vese, "Active contours without edges," IEEE Trans. Image Processing, vol. 10, no. 2, pp. 266-277, Feb. 2001.
35. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Computer Vision vol. 47, no. 1/2/3, pp. 7-42, 2002 [Online]. Available: http://www.middlebury.edu/stereo/
36. A. Klaus, M. Sormann, and K. Karner, "Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure," in Proc. ICPR, vol. 3, 2006, pp. 15-18.
37. Q. Yang, L. Wang, R. Yang, H. Stewenius, and D. Nister, "Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling," in Proc. IEEE Conf. CVPR, June 2006, vol. 2, pp. 2347-2354.
38. J. Sun, Y. Li, S.B. Kang, and H.Y. Shum, "Symmetric stereo matching for occlusion handling," in Proc. IEEE Conf. CVPR, vol. 2, 2005, pp. 399-406.
39. H. Hirschmueller, "Stereo vision in structured environments by consistent semi-global matching," in Proc. IEEE Conf. CVPR, 2006, pp. 2386-2393.
40. C. Lei, J. Selzer, and Y. Yang, "Region-tree based stereo using dynamic programming optimization" in Proc. IEEE Conf. CVPR, 2006, pp. 2378-2385.
41. K.J. Yoon and I.S. Kweon, "Adaptive support-weight approach for correspondence search," IEEE Trans. Pattern Anal. Machine Intell. vol. 28, no. 4, pp. 650-656, 2006.
42. V. Kolmogorov and R. Zabih, "Computing visual correspondence with occlusions using graph cuts," in Proc. ICCV, 2001, vol. 2, pp. 508-515.
43. S. Rusinkiewicz, "Survey of BRDF representation for computer graphics," 1997 [Online]. Available: http://www.cs.princeton.edu/ ~smr/cs348c-97/surveypaper.html
44. A. Wenger, A. Gardner, C. Tchou, J. Unger, T. Hawkins, and P. Debevec, "Performance relighting and reflectance transformation with time-multiplexed illumination," in Proc. SIGGRAPH (ACM Trans. Graphics), 2005, pp. 756-764.
45. K.Y.K. Wong and R. Cipolla, "Reconstruction of sculpture from its profiles with unknown camera positions," IEEE Trans. Image Process. vol. 13, no. 3, pp. 381-389, Mar. 2004.
46. S. Osher and N. Paragios, Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer-Verlag, 2003.
47. J.E. Lengyel, "Compression of time-dependent geometry," in ACM Symp. Interactive 3D Graphics, pp. 89-96, 1999.
48. S. Gupta, K. Sengupta, and A. Kassim, "Registration and partitioning-based compression of 3-D dynamic data," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1144-1155, 2003.
49. W. Martin and J. Aggarwal, "Volumetric descriptions of objects from multiple views," IEEE Trans. Pattern Anal Machine Intell., vol. 5, no. 2, pp. 150-158, 1983.
50. B. García and B. Brunet, "3D reconstruction with projective octrees and epipolar geometry," in Int. Conf. Computer Vision, pp. 1067-1072, Jan. 1998.
51. A. Laurentini, "The visual hull concept for silhouette-based image understanding," IEEE Trans. Pattern Anal. Machine Intell., vol. 16, no. 2, pp. 150-162, Feb. 1994.
52. W. Matusik, C. Buehler, R. Raskar, S. Gortler, and L. McMillan, "Image-based visual hulls," in Proc. SIGGRAPH (ACM Trans. Graphics) July 2000, pp. 369-374.
53. R. Cipolla and A. Blake, "Surface shape from the deformation of apparent contours," Int. J. Computer Vision, vol. 9, no. 2, pp. 83-112, 1992.
54. R. Vaillant and O. Faugeras, "Using extremal boundaries for 3-D object modeling," IEEE Rans. Pattern Anal. Machine Intell, vol. 14, no. 2, pp. 157-173, Feb. 1992.
55. C. Liang and K.Y. Wong, "Robust recovery of shapes with unknown topology from dual space," IEEE Trans. Pattern Anal. Machine Intell. to be published.
56. T. Kanade, P. W. Rander, and P. J. Narayanan, "Virtualized reality: constructing virtual worlds from real scenes," IEEE Multimedia, Immersive Telepresence, vol. 4, no. 1, pp. 34-47, Jan. 1997.
57. S. Vedula, S. Baker, and T. Kanade, "Image-based spatio-temporal modeling and view interpolation of dynamic events," ACM Trans. Graphics, vol. 24, no. 2, pp. 240-261, Apr. 2005.
58. J. Carranza, C. Theobolt, M.A. Magnor, and H.P. Seidel, "Free-viewpoint video of human actors," in Proc. SIGGPAPH (ACM Trans. Graphics), July 2003, pp. 569-577.
59. G. Cheung, S. Baker, J. Hodgins, and T. Kanade, "Markerless human motion transfer," in Proc. 2nd Int. Symp. 3D Data Processing Visualization Transmission, Thessaloniki, Greece, Sept. 2004, pp. 373-378.
60. G. Cheung, S. Baker, and T. Kanade, "Visual hull alignment and refinement across time: A 3D reconstruction algorithm combining shape-from-silhouette with stereo," in Proc. IEEE Conf. CVPR, vol. 2, June 2003, pp. 375-382.
61. M. Magnor, Video-Based Rendering. Wellesley, MA: A.K. Peters, 2005.
62. Fujii, "A basic study on integrated 3-D visual communication," Ph.D dissertation, The University of Tokyo, 1994 (in Japanese).
63. T. Fujii, T. Kimoto, and M. Tanimoto, "Ray space coding for 3D visual communication," in Proc. Picture Coding Symp. '96, Mar. 1996, pp. 447-451.
64. M. Bertalmino, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," in Proc. SIGGRAPH ACM Trans. Graphics, 2000, pp. 417-424.
65. A. Criminisi, P. Perez, and K. Toyama, "Object removal by exemplar-based inpainting," in Proc. IEEE Conf. CVPR, 2003, vol. 2, pp. 721-728.