The multiple-camera 3-D production studio

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 19, Issue 6, 2009, Pages 856-869

  Starek, Jonathan ^a   Maki, Atsuto ^b,c   Nobuhara, Shohei ^b   Hilton, Adrian ^a   Matsuyama, Takashi ^b  

^a UNIVERSITY OF SURREY   (United Kingdom)

^b KYOTO UNIVERSITY   (Japan)

^c TOSHIBA CORPORATION   (Japan)

Author keywords

Camera; Free viewpoint video; Image based reconstruction; Image based rendering; Machine vision; Multiple camera systems; Studio system design; Virtual reality

Indexed keywords

FREE-VIEWPOINT VIDEO; IMAGE-BASED RECONSTRUCTION; IMAGE-BASED RENDERING; MACHINE-VISION; MULTIPLE-CAMERA SYSTEMS;

IMAGE RECONSTRUCTION; MACHINE DESIGN; RESEARCH LABORATORIES; RESTORATION; STUDIOS; SYNTHESIS (CHEMICAL); SYSTEMS ANALYSIS; THREE DIMENSIONAL; VIDEO AMPLIFIERS; VIRTUAL REALITY; VISUAL COMMUNICATION;

CAMERAS;

EID: 67649534496     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2009.2017406     Document Type: Article

References (49)

1. T. Kanadě, P. Rander, and P. Narayanan, "Virtualized reality: Constructing virtual worlds from real scenes," IEEE Multimedia, vol. 4, no. 1, pp. 34-47, Jan.-Mar. 1997.
2. J. Starek and A. Hilton, "Surface capture for performance based animation," IEEE Comput. Graph. Appl., vol. 27 no. 3, pp. 21-31, May-Jun. 2007.
3. T. Matsuyama, X. Wu, T. Takai, and S. Nobuhara, "Real-time 3-D shape reconstruction, dynamic 3-D mesh deformation and high fidelity visualization for 3-D video," Comput. Vision Image Understand., vol. 96. no. 3, pp. 393-434, Dec. 2004.
4. T. Moeslund, A. Hilton, and V. Kruger, "A survey of advances in videobased human motion capture and analysis," Comput. Vision Image Understand., vol. 104, no. 2, pp. 90-126, Nov. 2006.
5. J. Starck and A. Hilton, "Model-based multiple view reconstruction of people," in Proc. IEEE Int. Conf. Comput. Vision, vol. 2. Nice, France, 2003, pp. 915-922.
6. J. Carranza, C. M. Theobalt, M. Magnor, and H. Seidel, "Free-viewpoint video of human actors," ACM Trans. Gmph., vol. 22, no. 3, pp. 569-577, Jul. 2003.
7. C. Theobalt, G. Ziegler, M. Magnor, and H.-P. Seidel, "Model-based free-viewpoint video: Acquisition, rendering and encoding," in Proc. Picture Coding Symp., vol. 5, San Francisco, CA, 2004, pp. 1-6.
8. P. Sand, L. McMillan, and J. Popovic, "Continuous capture of skin deformation," ACM Trans. Graph., vol. 22, no. 3, pp. 578-586, Jul. 2003.
9. S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A comparison and evaluation of multi-view stereo reconstruction algorithms," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognition, vol. 1. New York, 2006, pp. 519-526.
10. A. Laurentini, "The visual hull concept for silhouette based image understanding," IEEE Trans. Pattern Anal. Mach. Intell., vol. 16, no. 2, pp. 150-162, Feb. 1994.
11. K. Kutulakos and S. Seitz, "A theory of shape by space carving," Int. J. Comput. Vision, vol. 38, no. 3, pp. 199-218, Jul.-Aug. 2000.
12. O. Faugeras and R. Keriven, "Variational principles, surface evolution, PDEs, level set methods and the stereo problem," IEEE Trans. Image Process., vol. 7, no. 3, pp. 336-344, Mar. 1998.
13. B. Goldluecke and M. Magnor, "Space-time isosurface evolution for temporally coherent 3-D reconstruction," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognition, vol. 1. Hilton Head, SC, 2000, pp. 350-355.
14. C. Esteban and F. Schmitt, "Silhouette and stereo fusion for 3-D object modeling," Comput. Vision Image Understand., vol. 96, no. 3, pp. 367-392, Dec. 2004.
15. J. Starck and A. Hilton, "Virtual view synthesis of people from multiple view video sequences," Graph. Models, vol. 67, no. 6, pp. 600-620, Nov. 2005.
16. Y. Furukawa and J. Ponce, "Carved visual hulls for image-based modeling," in Proc. Eur. Conf. Comput. Vision, Graz, Austria, 2006, pp. 1:564-1:577.
17. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. 23rd Annu. Conf. Comput. Graph. Interactive Techniques, vol. 30. pp. 31-42, 1996.
18. P. Debevec, Y. Yu, and G. Borshukov, "Efficient view-dependent image-based rendering with projective texture-mapping," in Proc. Eurograh. Workshop Rendering, Vienna, Austria, 1998, pp. 105-116.
19. C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen, "Unstructured lumigraph rendering," in Proc. ACM Trans. Graph., New York, 2001, pp. 425-432.
20. C. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation," ACM Trans. Graph., vol. 23, no. 3, pp. 600-608, Aug. 2004.
21. G. Cheung, T. Kanade, J. Bouguet, and M. Holler, "A real time system for robust 3-D voxel reconstruction of human motions," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognition, Hilton Head Island, SC, 2000, pp. 714-720.
22. O. Grau, T. Pullen, and G. Thomas, "A combined studio production system for 3-D capturing of live action and immersive actor feedback," IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 3, pp. 370-380, Mar. 2004.
23. J.-M. Hasenfratz, M. Lapierre, and F. Sillion, "A real-time system for full body interaction with virtual worlds," in Proc. Eurograph. Symp. Virtual Envimnments, Grenoble, France, 2004, pp. 147-156.
24. W. Matusik, C. Buehler, and L. Mcmillan, "Polyhedral visual hulls for real-time rendering," in Proc. Eurograph. Workshop Rendering, London, U.K., 2001, pp. 115-126.
25. J. Allard, J.-S. Franco, C. Menier, E. Boyer, and B. Raffin, "The grimage platform: A mixed reality environment for interactions," in Proc. IEEE Int. Conf. Comput. Vision Syst., New York, 2006, pp. 46.
26. M. Waschbusch, S. Wurmlin, D. Cotting, F. Sadlo, and M. Gross, "Scalable 3-D video of dynamic scenes," Vision Comput., vol. 21 no. 8-10, pp. 629-638, Feb. 2005.
27. C. Theobalt, M. Li, M. Magnor, and H.-P. Seidel, "A flexible and versatile studio for synchronized multi-view video recording," in Proc. Vision, Video Graph., Bath, U.K., Jul. 2003, pp. 9-16.
28. J.-S. Franco, M. Lapierre, and E. Boyer, "Visual shapes of silhouette sets," in Proc. Int. Symp. 3-D Data Process., Visualization Transm., Chapel Hill, NC, 2006, pp. 397-404.
29. K. Tomiyama, Y. Orihara, M. Katayama, and Y. Iwadate, "Algorithm for dynamic 3-D object generation from multi-viewpoint images," in Proc. SPIE, vol. 5599, no. 19, pp. 153-161, Nov. 2004.
30. Y. Boykov and V. Kolmogorov, "Computing geodesies and minimal surfaces via graph, cuts," in Proc. IEEE Int. Conf. Comput. Vision, Nice, France, 2003, pp. 26-33.
31. S. Vedula, S. Baker, and T. Kanadě, "Image-based spatio-temporal modeling and view interpolation of dynamic events," ACM Trans. Graph., vol. 24, no. 2, pp. 240-261, Apr. 2005.
32. W. Matusik and H. Pfister, "3-D TV: A scalable system, for real-time acquisition, transmission and autostereoscopic display of dynamic scenes," in Proc. ACM SIGGRAPH 2004, Los Angeles, CA, pp. 814-824.
33. C. Zhang and T. Chen, "A self-reconfigurable camera array," in Proc. Eurograph. Symp. Rendering, Norrköping, Sweden, Jun. 2004, pp. 243-254.
34. T. Kanade, H. Saito, and S. Vedula, "Digitizing time-varying 3-D events by synchronized multiple video streams," Robotics Inst., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-RITR-98-34, 1998.
35. K. Cheung, S. Baker, and T. Kanade, "Silhouette across time: Part II: Applications to human modeling and markerless motion tracking," Int. J. Comput. Vision, vol. 63, no. 3, pp. 225-245, Jul. 2005.
36. M. Tanimoto, "FTV (free-viewpoint television) for 3-D scene reproduction and creation," in Proc. IEEE Conf. Comput. Vision Pattern Recognition Workshop, New York, 2006, pp. 172.
37. Y.-Y. Chuang, B. Curless, D. Salesin, and R. Szeliski, "A Bayesian approach to digital matting," in Proc. IEEE Conf. Comput. Vision Pattern Recognition, HI, 2001, pp. 264-271.
38. N. Ukita and T. Matsuyama, "Real-time cooperative multi-target tracking by communicating active vision agents," Comput. Vision Image Understand., vol. 97, no. 2, pp. 137-179, Feb. 2005.
39. R. Tsai, "An efficient and accurate camera calibration technique for 3-D machine-vision," in Proc. IEEE Conf. Comput. Vision Pattern Recognition, Miami, FL, 1986, pp. 364-374.
40. Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans Pattern Anal Mach. Intell., vol. 22, no. 11, pp. 1330-1334, Nov. 2000.
41. R. I. Hartley and A. Zisserman, Multiple View Geometry Comput. Vision, 2nd ed., Cambridge, U.K.: Cambridge Univ. Press, 2004.
42. T. Svoboda, D. Martinec, and T. Pajdla, "A convenient multicamera self-calibration for virtual environments," Presence: Teleoper. Virtual, Envrion., vol. 14, no. 4, pp. 407-422, Aug. 2005.
43. J. Mitchelson and A. Hilton, "Wand-based multiple-camera studio calibration," Center Vision, Speech and Signal Process., Univ. Surrey, Surrey, U.K., Tech. Rep., vol. VSSP-TR-2/2003, 2003.
44. A. Wenger, T. Hawkins, and P. Debevec, "Optimizing color matching in a lighting reproduction system for complex subject and illuminant spectra," in Proc. Eurograph. Workshop Rendering, Leuven, Belgium, 2003, pp. 249-259.
45. P. Cignoni, C. Rocchini, and R. Scopigno, "Metro: Measuring error on simplified surfaces," Comput. Graph. Forum, vol. 17, no. 2, pp. 167-174, Jun. 1998.
46. O. Faugeras, B. Hotz, H. Mathieu, T. Vieville, Z. Zhang, P. Fua, E. Theron, L. Moll, G. Berry, J. Vuillemin, P. Bertin, and C. Proy, "Real time correlation based stereo: Algorithm implementations and applications," French Nat. Res. Inst. Comput. Sci., France, Rep. 2013, 1993.
47. A. Fusiello, E. Trucco, and A. Verri, "Rectification with unconstrained stereo geometry," in Proc. Brit. Machine-Vision Conf., Essex, U.K., 1997, pp. 400-409.
48. A. Fusiello, V Roberto, and E. Trucco, "Symmetric stereo with multiple windowing," Int. J. Pattern Recognit. Artificial Intell., vol. 14, no. 8, pp. 1053-1066, Dec. 2000.
49. A. R. Greenleaf, Photographic Optics. New York: MacMillan, 1950, pp. 25-27.