Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos

Computer Graphics Forum

Volumn 31, Issue 2, 2012, Pages 247-256

  Richardt, Christian ^a,b   Stoll, Carsten ^a   Dodgson, Neil A ^b   Seidel, Hans Peter ^a   Theobalt, Christian ^a  

^a MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CAMERAS; COMPUTER GRAPHICS; SIGNAL SAMPLING; STEREO IMAGE PROCESSING; THREE DIMENSIONAL COMPUTER GRAPHICS; VIDEO CAMERAS; VIDEO SIGNAL PROCESSING;

BACKGROUND SEGMENTATION; GEOMETRY INFORMATION; INTERACTIVE FRAME RATES; NOISE CHARACTERISTIC; SPATIO TEMPORAL FILTERING; SPATIO-TEMPORAL DENOISING; VIDEO PROCESSING; VIDEO RELIGHTING;

RENDERING (COMPUTER GRAPHICS);

EID: 84872029054     PISSN: 01677055     EISSN: 14678659     Source Type: Journal    
DOI: 10.1111/j.1467-8659.2012.03003.x     Document Type: Conference Paper

References (34)

1. ADAMS A., GELFAND N., DOLSON J., LEVOY M.: Gaussian KD-trees for fast high-dimensional filtering. ACM Transactions on Graphics (Proc. SIGGRAPH) 28, 3 (2009), 21.
2. BEDER C., BARTCZAK B., KOCH R.: A combined approach for estimating patchlets from PMD depth images and stereo intensity images. In Proc. DAGM Symposium (2007), no. 4713 in LNCS, pp. 11-20.
3. BROX T., BRUHN A., PAPENBERG N., WEICKERT J.: High accuracy optical flow estimation based on a theory for warping. In Proc. ECCV (2004), vol. 3024 of LNCS, pp. 25-36.
4. BASRI R., JACOBS D., KEMELMACHER I.: Photometric stereo with general, unknown lighting. International Journal of Computer Vision 72, 3 (2007), 239-257.
5. CHAN D., BUISMAN H., THEOBALT C., THRUN S.: A noise-aware filter for real-time depth upsampling. In ECCV Workshop on multi-camera & multi-modal sensor fusion (2008).
6. CHEN J., PARIS S., DURAND F.: Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics (Proc. SIGGRAPH) 26, 3 (2007), 103.
7. DOLSON J., BAEK J., PLAGEMANN C., THRUN S.: Upsampling range data in dynamic environments. In Proc. CVPR (2010), pp. 1141-1148.
8. DIEBEL J., THRUN S.: An application of Markov Random Fields to range sensing. In Proc. NIPS (2006), pp. 291-298.
9. EISEMANN M., DECKER B. D., MAGNOR M., BEKAERT P., DE AGUIAR E., AHMED N., THEOBALT C., SELLENT A.: Floating textures. Computer Graphics Forum (Proc. Eurographics) 27, 2 (April 2008), 409-418.
10. GORTLER S. J., GRZESZCZUK R., SZELISKI R., COHEN M. F.: The lumigraph. In Proc. SIGGRAPH (1996), pp. 43-54.
11. HERZOG R., EISEMANN E., MYSZKOWSKI K., SEIDEL H.-P.: Spatio-temporal upsampling on the GPU. In Proc. I3D (2010), pp. 91-98.
12. KOLB A., BARTH E., KOCH R., LARSEN R.: Time-offlight cameras in computer graphics. Computer Graphics Forum 29, 1 (2010), 141-159.
13. KOPF J., COHEN M. F., LISCHINSKI D., UYTTENDAELE M.: Joint bilateral upsampling. ACM Transactions on Graphics (Proc. SIGGRAPH) 26, 3 (2007), 96.
14. LINDNER M., KOLB A., HARTMANN K.: Data-fusion of PMD-based distance-information and high-resolution RGB-images. In Proc. ISSCS (July 2007), pp. 121-124.
15. LEE Y., MARKOSIAN L., LEE S., HUGHES J. F.: Line drawings via abstracted shading. ACM Transactions on Graphics (Proc. SIGGRAPH) 26, 3 (2007), 18.
16. LU J., SANDER P. V., FINKELSTEIN A.: Interactive painterly stylization of images, videos and 3D animations. In Proc. I3D (2010), pp. 127-134.
17. LANMAN D., TAUBIN G.: Build your own 3D scanner: 3D photography for beginners. In SIGGRAPH Courses (2009).
18. MALZBENDER T., WILBURN B., GELB D., AMBRISCO B.: Surface enhancement using real-time photometric stereo and reflectance transformation. In Proc. EGSR (2006), pp. 245-250.
19. OKABE M., ZENG G., MATSUSHITA Y., IGARASHI T., QUAN L., SHUM H.-Y.: Single-view relighting with normal map painting. In Proc. Pacific Graphics (2006), pp. 27-34.
20. PRAUN E., HOPPE H., WEBB M., FINKELSTEIN A.: Real-time hatching. In Proc. SIGGRAPH (2001), pp. 581-586.
21. RAMAMOORTHI R., HANRAHAN P.: An efficient representation for irradiance environment maps. In Proc. SIGGRAPH (2001), pp. 497-500.
22. RAMAMOORTHI R., HANRAHAN P.: A signal-processing framework for inverse rendering. In Proc. SIGGRAPH (2001), pp. 117-128.
23. RASKAR R., TAN K.-H., FERIS R., YU J., TURK M.: Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3 (2004), 679-688.
24. SCHARSTEIN D., SZELISKI R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47, 1-3 (2002), 7-42.
25. SNAVELY N., ZITNICK C. L., KANG S. B., COHEN M.: Stylizing 2.5-D video. In Proc. NPAR (2006), pp. 63-69.
26. TOLER-FRANKLIN C., FINKELSTEIN A., RUSINKIEWICZ S.: Illustration of complex real-world objects using images with normals. In Proc. NPAR (2007).
27. TOMASI C., MANDUCHI R.: Bilateral filtering for gray and color images. In Proc. ICCV (1998), pp. 839-846.
28. VEERARAGHAVAN A., RASKAR R., AGRAWAL A., MOHAN A., TUMBLIN J.: Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Transactions on Graphics (Proc. SIGGRAPH) 26 (2007), 69.
29. WANG O., FUCHS M., FUCHS C., DAVIS J., SEIDEL H.-P., LENSCH H. P. A.: A context-aware light source. In Proc. ICCP (2010).
30. WENGER A., GARDNER A., TCHOU C., UNGER J., HAWKINS T., DEBEVEC P.: Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics (Proc. SIGGRAPH) 24, 3 (2005), 756-764.
31. WINNEMÖLLER H., OLSEN S. C., GOOCH B.: Realtime video abstraction. ACM Transactions on Graphics (Proc. SIGGRAPH) 25, 3 (2006), 1221-1226.
32. YANG Q., YANG R., DAVIS J., NISTÉR D.: Spatial-depth super resolution for range images. In Proc. CVPR (2007).
33. ZHANG R., TSAI P.-S., CRYER J., SHAH M.: Shape-from-shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 8 (1999), 690-706.
34. ZHU J., WANG L., YANG R., DAVIS J.: Fusion of time-of-flight depth and stereo for high accuracy depth maps. In Proc. CVPR (2008).