3D-TV production from conventional cameras for sports broadcast

IEEE Transactions on Broadcasting

Volumn 57, Issue 2 PART 2, 2011, Pages 462-476

  Hilton, Adrian ^a   Guillemaut, Jean Yves ^a   Kilner, Joe ^a   Grau, Oliver ^b   Thomas, Graham ^b  

^a UNIVERSITY OF SURREY   (United Kingdom)

^b BBC RESEARCH AND DEVELOPMENT   (United Kingdom)

Author keywords

3D video; 3DTV; Camera calibration; free viewpoint video; image segmentation; multiple view reconstruction

Indexed keywords

3D VIDEO; 3DTV; CAMERA CALIBRATION; FREE-VIEWPOINT VIDEO; MULTIPLE VIEW RECONSTRUCTION;

CAMERAS; DIGITAL TELEVISION; IMAGE SEGMENTATION; THREE DIMENSIONAL; VIDEO SIGNAL PROCESSING;

TELEVISION BROADCASTING;

EID: 79957597528     PISSN: 00189316     EISSN: None     Source Type: Journal    
DOI: 10.1109/TBC.2011.2131870     Document Type: Article

References (46)

1. S. Chen and L. Williams, "View interpolation for image synthesis," in Proc. ACM SIGGRAPH, 1993.
2. S. Seitz and C. Dyer, "View morphing: Synthesizing 3D metamorphosis using image transforms," in Proc. ACM SIGGRAPH, 1996, pp. 21-30.
3. D. Kim, D. Min, and K. Sohn, "A stereoscopic video generation method using stereoscopic display characterization and motion analysis," IEEE Trans. Broadcast., vol. 54, no. 2, pp. 188-197, Jun. 2008. (Pubitemid 351780176)
4. M. Irani, T. Hassner, and P. Anandan, "What does the scene look like from a scene point?," in European Conf. Comput. Vis., 2002.
5. H.-Y. Shum, S. Kang, and S.-C. Chan, "Survey of image-based representations and compression techniques," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, 2003.
6. A. Laurentini, "The visual hull concept for silhouette based image understanding," in IEEE Trans. Pattern Anal. Mach. Intell., 1994, vol. 16, no. 2, pp. 150-162.
7. S. Moezzi, A. Katkere, D. Y. Kuramura, and R. Jain, "Reality modeling and visualization from multiple video sequences," IEEE Comput. Graphics Appl., pp. 58-63, Nov. 1996. (Pubitemid 126586168)
8. W. Matusik, C. Buehler, R. Raskar, S. Gortler, and L. McMillan, "Image-based visual hulls," in Proc. ACM SIGGRAPH, 2000, pp. 369-374. (Pubitemid 32394001)
9. F. Franco and E. Boyer, "Exact polyhedral visual hulls," in British Mach. Vis. Conf., 2003, pp. 329-338.
10. G. Miller, A. Hilton, and J. Starck, "Interactive free-viewpoint video," in IEE Eur. Conf. Vis. Media Prod., 2005, pp. 50-59.
11. S. Seitz and C. Dyer, "Photorealistic scene reconstruction by voxel coloring," Int. J. Comput. Vis., vol. 35, no. 2, pp. 1-23, 1997.
12. T. Kanade and P. Rander, "Virtualized reality: Constructing virtual worlds from real scenes," IEEE Multimedia, vol. 4, no. 2, pp. 34-47, 1997. (Pubitemid 127583991)
13. C. Zitnick, S. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation, " in Proc. ACM SIGGRAPH, 2004, pp. 600-608.
14. J. Starck and A. Hilton, "Virtual viewsynthesis of people from multiple view video," Graphical Models, vol. 67, no. 6, pp. 600-620, 2005. (Pubitemid 41562261)
15. G. Miller, J. Starck, and A. Hilton, "Projective surface refinement for free-viewpoint video," in IET Eur. Conf. Vis. Media Prod., 2006, pp. 153-162.
16. S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A comparison and evaluation of multi-viewstereo reconstruction algorithms," in Conf. Comput. Vis. Pattern Recog., 2006, pp. 519-528.
17. W. Matusik, C. Buehler, R. Raskar, S. Gortler, and L. McMillan, "Image-based visual hulls," in Proc. ACM SIGGRAPH, 2000, pp. 369-374. (Pubitemid 32394001)
18. S. Moezzi, L. Tai, and P. Gerard, "Virtual viewgeneration for 3d digital video," IEEE Multimedia, vol. 4, no. 1, pp. 18-25, 1997.
19. S. Vedula, S. Baker, P. Rander, R. Collins, and T. Kanade, "Threedimensional scene flow," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 3, 2005.
20. F. Franco, C. Menier, E. Boyer, and B. Raffin, "A distributed approach for real-time 3D modeling," in CVPR Workshop on Real-Time 3D Sensors and Their Applications, 2004.
21. J. Starck and A. Hilton, "Surface capture for performance-based animation," in IEEE Comput. Graphics Appl., 2007, vol. 27, no. 3, pp. 21-31. (Pubitemid 46767722)
22. T. Kanade, P. Rander, and P. Narayanan, "Virtualized reality: Constructing virtual worlds from real scenes," IEEE Multimedia, vol. 4, no. 1, pp. 34-47, 1997. (Pubitemid 127583991)
23. O. Grau, M. Prior-Jones, and G. Thomas, "3d modelling and rendering of studio and sport scenes for tv applications," in Proc. of WIAMIS.
24. K. Connor and I. Reid, "A multiple view layered representation for dynamic novel view synthesis," in British Mach. Vis. Conf., 2003.
25. N. Inamoto and H. Saito, "Virtual viewpoint replay for a soccer match by view interpolation from multiple cameras," IEEE Trans. Multimedia, vol. 9, no. 6, pp. 1155-1166, 2007. (Pubitemid 47460397)
26. K. Kimura and H. Saito, "Player viewpoint video synthesis using multiple cameras," in IEE Eur. Conf. Vis. Media Prod., 2005, pp. 112-121.
27. M. Germann, A. Hornung, R. Keiser, R. Siegler, S. Wurmlin, and M. Gross, "Articulated billboards for video-based rendering," Comput. Graphics Forum, vol. 29, no. 2, pp. 585-594, 2010.
28. G. Thomas, "Real-time camera pose estimation for augmenting sports scenes," in Eur. Conf. Vis. Media Prod., 2006, pp. 10-19.
29. G. Thomas, "Real-time camera tracking using sports pitch markings," J. Real Time Image Process. vol. 2, no. 2-3, pp. 117-132, 2007 [Online]. Available: http://www.bbc.co.uk/rd/publications/whitepaper168. shtml, (Available as BBC R&D White Paper 168. ) (Pubitemid 350027504)
30. P. Hillman, J. Hannah, and D. Renshaw, "Foreground/background segmentation of motion picture images and image sequences," IEE Trans. Vis., Image Signal Process., vol. 142, no. 4, pp. 387-397, Aug. 2005. (Pubitemid 41229589)
31. O. Grau, G. Thomas, A. Hilton, J. Kilner, and J. Starck, "A robust free-viewpoint video system for sport scenes," in Proc. 3DTV Conf., Kos, Greece, 2007.
32. O. Grau and J. Easterbrook, "Effects of camera aperture correction on keying of broadcast video," in Proc. 5rd Eur. Conf. Vis. Media Prod. (CVMP), 2008.
33. J. Kilner, J. Starck, A. Hilton, J. Guillemaut, and O. Grau, "Dual mode deformable models for free-viewpoint video of outdoor sports events," in IEEE Int.Conf. 3D Imaging Modeling, 2007.
34. J. Guillemaut, A. Hilton, J. Starck, J. Kilner, and O. Grau, "A baysian framework for simultaneous reconstruction and matting," in IEEE Int. Conf. 3D Imaging and Modeling, 2007.
35. J.-Y. Guillemaut, J. Kilner, and A. Hilton, "Robust graph-cut scene segmentation and reconstruction for free-viewpoint video of complex dynamic scenes," in IEEE Int.Conf. Comput. Vis. (ICCV), 2009.
36. J. Sun, W. Zhang, X. Tang, and H.-Y. Shum, in Background Cut, 2006, vol. 3954, pp. 628-641.
37. K. Kutulakos, "Approximate N-view stereo," in ECCV, 2000, vol. I, pp. 67-83.
38. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool, "A comparison of affine region detectors," IJCV, vol. 65, no. 1-2, pp. 43-72, 2005. (Pubitemid 43281159)
39. K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," PAMI, vol. 27, no. 10, pp. 1615-1630, 2005. (Pubitemid 41555406)
40. Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," PAMI, vol. 23, no. 11, pp. 1222-1239, 2001. (Pubitemid 33137959)
41. V.Kolmogorov and R. Zabih, "What energy function can be minimized via graph cuts?," PAMI, vol. 26, no. 2, pp. 147-159, 2004.
42. Y. Boykov and V. Kolmogorov, "An experimental comparison of mincut/ max-flow algorithms for energy minimization in vision," PAMI, vol. 26, no. 9, pp. 1124-1137, 2004.
43. K. Alahari, P. Kohli, and P. Torr, "Reduce, reuse & recycle: Efficiently solving multi-label MRFs," in CVPR, 2008.
44. A. Smolic, K. Mueller, P. Merkle, and A. Vetro, "Development of a new mpeg standard for advanced 3d video applications," in Proc. Int. Symp. Image Signal Process. Anal., 2009, pp. 400-407.
45. A. Vetro, A. M. Tourapis, K. Muller, and T. Chen, "3D-TV content storage and transmission," IEEE Trans. Broadcast., 2011, in press.
46. L. Lipton, Foundations of the Stereoscopic Cinema. New York: Van Nostrand Reinhold, 1982.