Unstructured video-based rendering: Interactive exploration of casually captured videos

ACM Transactions on Graphics

Volumn 29, Issue 4, 2010, Pages

  Ballan, Luca ^a   Brostow, Gabriel J ^b   Puwein, Jens ^a   Pollefeys, Marc ^a  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

3-D SHAPE; CAMERA MOTIONS; COMPLICATED BACKGROUNDS; HAND-HELD CAMERAS; INTERACTIVE EXPLORATION; INTERACTIVE RENDERING; MULTIVIEW VIDEO; OFFLINE; REAL-WORLD; SPATIAL NAVIGATION; VIDEO-BASED RENDERING; VIEW INTERPOLATION; VIEW-DEPENDENT; VISUAL CUES; WHOLE SYSTEMS;

ALGORITHMS; CAMERAS; INTERPOLATION; NAVIGATION;

THREE DIMENSIONAL;

EID: 77956353237     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/1778765.1778824     Document Type: Article

References (64)

1. ARULAMPALAM, M. S., MASKELL, S., and GORDON, N. 2002. A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. Signal Processing 50, 174-188.
2. BAI, X., WANG, J., SIMONS, D., and SAPIRO, G. 2009. Video snapcut: robust video object cutout using localized classifiers. ACM Trans. Graph. 28, 3.
3. BALLAN, L., and CORTELAZZO, G. M. 2008. Marker-less motion capture of skinned models in a four camera set-up using optical flow and silhouettes. In 3DPVT.
4. BOYKOV, Y., and KOLMOGOROV, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9, 1124-1137.
5. BUEHLER, C., BOSSE, M., MCMILLAN, L., GORTLER, S. J., and COHEN, M. F. 2001. Unstructured lumigraph rendering. In SIGGRAPH, 425-432.
6. CAMPBELL, N. D., VOGIATZIS, G., HERNáNDEZ, C., and CIPOLLA, R. 2007. Automatic 3d object segmentation in multiple views using volumetric graph-cuts. In 18th British Machine Vision Conference, vol. 1, 530-539.
7. CARRANZA, J., THEOBALT, C., MAGNOR, M. A., and PETER SEIDEL, H. 2003. Free-viewpoint video of human actors. In ACM Transactions on Graphics, 569-577.
8. CHEN, S. E., and WILLIAMS, L. 1993. View interpolation for image synthesis. In SIGGRAPH'93: Proceedings of the 20th annual conference on Computer graphics and interactive techniques, 279-288.
9. CHUANG, Y.-Y., CURLESS, B., SALESIN, D. H., and SZELISKI, R. 2001. A bayesian approach to digital matting. In Proceedings of IEEE CVPR 2001, vol. 2, 264-271.
10. CHUANG, Y.-Y., AGARWALA, A., CURLESS, B., SALESIN, D. H., and SZELISKI, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3 (July), 243-248.
11. DE AGUIAR, E., STOLL, C., THEOBALT, C., AHMED, N., SEIDEL, H. P., and THRUN, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3, 1-10.
12. DEBEVEC, P. E., TAYLOR, C. J., and MALIK, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Proceedings of SIG-GRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 11-20.
13. DEBEVEC, P., BORSHUKOV, G., and YU, Y. 1998. Efficient view-dependent image-based rendering with projective texturemapping. In 9th Eurographics Workshop on Rendering.
14. DRAGICEVIC, P., RAMOS, G., BIBLIOWITCZ, J., NOWROUZEZAHRAI, D., BALAKRISHNAN, R., and SINGH, K. 2008. Video browsing by direct manipulation. In CHI'08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 237-246.
15. EISEMANN, M., DECKER, B. D., MAGNOR, M., BEKAERT, P., DE AGUIAR, E., AHMED, N., THEOBALT, C., and SELLENT, A. 2008. Floating Textures. Computer Graphics Forum (Proc. Eurographics EG'08) 27, 2 (4), 409-418.
16. FRANCO, J.-S., and BOYER, E. 2005. Fusion of multi-view silhouette cues using a space occupancy grid. In ICCV, 1747-1753.
17. GOESELE, M., SNAVELY, N., CURLESS, B., HOPPE, H., and SEITZ, S. M. 2007. Multi-view stereo for community photo collections. In ICCV, 1-8.
18. GOLDMAN, D. B., GONTERMAN, C., CURLESS, B., SALESIN, D., and SEITZ, S. M. 2008. Video object annotation, navigation, and composition. In UIST'08: Proceedings of the 21st annual ACM symposium on User interface software and technology, 3-12.
19. GOLDMAN, D. B. 2007. A framework for video annotation, visualization, and interaction. PhD thesis.
20. GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., and COHEN, M. F. 1996. The lumigraph. In SIGGRAPH, 43-54.
21. GRUNDLAND, M., VOHRA, R., WILLIAMS, G. P., and DODGSON, N. A. 2006. Cross dissolve without cross fade: Preserving contrast, color and salience in image compositing. In Proceedings of EUROGRAPHICS, Computer Graphics Forum, 577-586.
22. GUILLEMAUT, J.-Y., HILTON, A., STARCK, J., KILNER, J., and GRAU, O. 2007. A bayesian framework for simultaneous matting and 3d reconstruction. In 3DIM'07: Proceedings of the Sixth International Conference on 3-D Digital Imaging and Modeling, 167-176.
23. GUILLEMAUT, J.-Y., KILNER, J., and HILTON, A. 2009. Robust graph-cut scene segmentation and reconstruction for freeviewpoint video of complex dynamic scenes. In Proc. International Conference on Computer Vision (ICCV 2009).
24. HARTLEY, R. I., and ZISSERMAN, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN:0521623049.
25. HASLER, N., ROSENHAHN, B., THORMäHLEN, T., WAND, M., GALL, J., and SEIDEL, H.-P. 2009. Markerless motion capture with unsynchronized moving cameras. In CVPR, 224-231.
26. HAYASHI, K., and SAITO, H. 2006. Synthesizing free-viewpoint images from multiple view videos in soccer stadium. In CGIV'06: Proceedings of the International Conference on Computer Graphics, Imaging and Visualisation, 220-225.
27. HAYS, J., and EFROS, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (SIG-GRAPH 2007) 26, 3.
28. HEIGL, B., KOCH, R., POLLEFEYS, M., DENZLER, J., and VAN GOOL, L. 1999. Plenoptic modeling and rendering from image sequences taken by hand-held camera. In Patter Recognition 1999, 21. DAGM-Symposium, 94-101.
29. KANADE, T., 2001. Carnegie mellon goes to the superbowl. http://www.ri.cmu.edu/events/sb35/tksuperbowl.html.
30. KARRER, T., WEISS, M., LEE, E., and BORCHERS, J. 2008. Dragon: a direct manipulation interface for frame-accurate inscene video navigation. In CHI'08, 247-250.
31. KILNER, J., STARCK, J., and HILTON, A. 2006. A comparative study of free-viewpoint video techniques for sports events. European Conference on Visual Media Production (CVMP).
32. KILNER, J., STARCK, J., HILTON, A., and GRAU, O. 2007. Dual-mode deformable models for free-viewpoint video of sports events. In 3DIM07, 177-184.
33. KOPF, J., NEUBERT, B., CHEN, B., COHEN, M., COHEN-OR, D., DEUSSEN, O., UYTTENDAELE, M., and LISCHINSKI, D. 2008. Deep photo: model-based photograph enhancement and viewing. ACM Trans. Graph. 27, 5, 116.
34. LEVOY, M., and HANRAHAN, P. 1996. Light field rendering. In SIGGRAPH, 31-42.
35. LHUILLIER, M., and QUAN, L. 2005. A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Trans. Pattern Anal. Mach. Intell. 27, 3, 418-433.
36. LIU, F., GLEICHER, M., JIN, H., and AGARWALA, A. 2009. Content-preserving warps for 3d video stabilization. In ACM SIGGRAPH 2009, 1-9.
37. LOWE, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2, 91-110.
38. MATUSIK, W., BUEHLER, C., RASKAR, R., GORTLER, S. J., and MCMILLAN, L. 2000. Image-based visual hulls. In Proceedings of ACM SIGGRAPH, 369-374.
39. POLLEFEYS, M., VAN GOOL, L., VERGAUWEN, M., VERBIEST, F., CORNELIS, K., TOPS, J., and KOCH, R. 2004. Visual modeling with a hand-held camera. IJCV 59, 3, 207-232.
40. RAV-ACHA, A., KOHLI, P., ROTHER, C., and FITZGIBBON, A. 2008. Unwrap mosaics: A new representation for video editing. ACM Transactions on Graphics (SIGGRAPH 2008) (August).
41. RONG, G., and TAN, T.-S. 2006. Jump flooding in gpu with applications to voronoi diagram and distance transform. In ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D), ACM, 109-116.
42. SCHINDLER, G., and DELLAERT, F. 2010. Probabilistic temporal inference on reconstructed 3D scenes. In CVPR, 1-8.
43. SCHöDL, A., SZELISKI, R., SALESIN, D. H., and ESSA, I. 2000. Video textures. In SIGGRAPH'00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 489-498.
44. SCHöNEMANN, P. 1966. A generalized solution of the orthogonal procrustes problem. Psychometrika 31, 1 (March), 1-10.
45. SEITZ, S. M., and DYER, C. R. 1996. View morphing. In Proceedings of ACM SIGGRAPH, 21-30.
46. SEITZ, S. M., CURLESS, B., DIEBEL, J., SCHARSTEIN, D., and SZELISKI, R. 2006. A comparison and evaluation of multiview stereo reconstruction algorithms. In 2006 Conference on Computer Vision and Pattern Recognition (CVPR 2006), 519-528.
47. SINHA, S. N., and POLLEFEYS, M. 2004. Synchronization and calibration of camera networks from silhouettes. In ICPR'04: Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 1, 116-119.
48. SINHA, S. N., STEEDLY, D., SZELISKI, R., AGRAWALA, M., and POLLEFEYS, M. 2008. Interactive 3d architectural modeling from unordered photo collections. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2008) 27, 5, 159.
49. SIVIC, J., and ZISSERMAN, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision, vol. 2, 1470-1477.
50. SNAVELY, N., SEITZ, S. M., and SZELISKI, R. 2006. Photo tourism: Exploring photo collections in 3d. In SIGGRAPH Conference Proceedings, 835-846.
51. SNAVELY, N., GARG, R., SEITZ, S. M., and SZELISKI, R. 2008. Finding paths through the world's photos. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008) 27, 3, 11-21.
52. STARCK, J., and HILTON, A. 2007. Surface capture for performance based animation. IEEE Computer Graphics and Applications 27 (3), 21-31.
53. STICH, T., LINZ, C., ALBUQUERQUE, G., and MAGNOR, M. 2008. View and time interpolation in image space. Computer Graphics Forum (Proc. Pacific Graphics) 27, 7.
54. SUN, J., ZHANG, W., TANG, X., and SHUM, H.-Y. 2006. Background cut. In ECCV (2), 628-641.
55. TUYTELAARS, T., and VAN GOOL, L. 2004. Synchronizing video sequences. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on 1, 762-768.
56. VAN DEN HENGEL, A., DICK, A., THORMäHLEN, T., WARD, B., and TORR, P. H. S. 2007. Videotrace: Rapid interactive scene modelling from video. ACM Transactions on Graphics 26, 3 (July), 86:1-86:5.
57. VEDULA, S., BAKER, S., and KANADE, T. 2005. Imagebased spatio-temporal modeling and view interpolation of dynamic events. ACM Transactions on Graphics 24, 2 (Apr.), 240-261.
58. VLASIC, D., BARAN, I., MATUSIK, W., and POPOVIĆ, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Transactions on Graphics 27, 3, 97:1-97:9.
59. WANG, J., and BODENHEIMER, B. 2008. Synthesis and evaluation of linear motion transitions. ACM Trans. Graph. 27, 1, 1-15.
60. WANG, J., BHAT, P., COLBURN, R. A., AGRAWALA, M., and COHEN, M. F. 2005. Interactive video cutout. ACM Trans. Graph. 24, 3, 585-594.
61. WASCHBüSCH, M., WüRMLIN, S., and GROSS, M. H. 2007. 3d video billboard clouds. Computer Graphics Forum (Proc. Eurographics EG'07) 26, 3, 561-569.
62. WüRMLIN, S., and NIEDERBERGER, C., 2010. Realistic virtual replays for sports broadcasts. http://www.liberovision.com/.
63. ZACH, C., POCK, T., and BISCHOF, H. 2007. A globally optimal algorithm for robust tv-l1 range image integration. In IEEE International Conference on Computer Vision (ICCV).
64. ZITNICK, C. L., KANG, S. B., UYTTENDAELE, M., WINDER, S., and SZELISKI, R. 2004. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics 23, 3 (Aug.), 600-608.