Temporal-dense dynamic 3-D reconstruction with low frame rate cameras

IEEE Journal on Selected Topics in Signal Processing

Volumn 6, Issue 5, 2012, Pages 447-459

  Li, Kun ^a   Dai, Qionghai ^b   Xu, Wenli ^b   Yang, Jingyu ^a  

^a TIANJIN UNIVERSITY   (China)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Spatio temporal fusion; spatio temporal interpolation; spatio temporal sampling; temporal dense dynamic 3 D reconstruction

Indexed keywords

3D RECONSTRUCTION; DUAL-TREE DISCRETE WAVELET TRANSFORMS; DYNAMIC SCENES; FRAME RATE; HIGH FRAME RATE; HIGH PRICE; POISSON IMAGE EDITING; RESEARCH TOPICS; SHAPE CONTEXTS; SPATIO-TEMPORAL; TIME VARYING;

CAMERAS; DISCRETE WAVELET TRANSFORMS; INTERPOLATION; SIGNAL PROCESSING;

THREE DIMENSIONAL;

EID: 84865429550     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2012.2194475     Document Type: Article

References (61)

1. M. Tanimoto, M. Tehrani, T. Fujii, and T. Yendo, "Free-viewpoint TV," IEEE Signal Process. Mag., vol. 28, no. 1, pp. 67-76, Jan. 2011.
2. C. Zhang, D. Florencio, and Z. Zhang, "Improving immersive experiences in telecommunication with motion parallax," IEEE Signal Process. Mag., vol. 28, no. 1, pp. 139-144, Jan. 2011.
3. J. Gall, C. Stoll, E. de Aguiar, C. Theobalt, B. Rosenhahn, and H.-P. Seidel, "Motion capture using joint skeleton tracking and surface estimation," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., Miami, FL, 2009, pp. 1746-1753.
4. D. Cremers and K. Kolev, "Multiview stereo and silhouette consistency via convex functionals over convex domains," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 6, pp. 1161-1174, Jun. 2011.
5. Y. M. Kim, C. Theobalt, J. Diebel, J. Kosecka, B. Miscusik, and S. Thrun, "Multi-view image and ToF sensor fusion for dense 3D reconstruction," in Proc. Int. Conf. Comput. Vis., 2009, pp. 1542-1549.
6. B. Wilburn, N. Joshi, V. Vaish, M. Levoy, and M. Horowitz, "Highspeed videography using a dense camera array," in Proc. CVPR'04, 2004, pp. 294-301.
7. B. Wilburn, N. Joshi, V. Vaish, E. V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, "High performance imaging using large camera arrays," ACM Trans. Graphics, vol. 24, no. 3, pp. 765-776, 2005.
8. Y. Pekelny and C. Gotsman, "Articulated object reconstruction and markerless motion capture from depth video," Comput. Graphics Forum, vol. 27, no. 2, pp. 399-408, 2008. (Pubitemid 351612857)
9. M. Liao, Q. Zhang, H. Wang, R. Yang, and M. Gong, "Modeling deformable objects from a single depth camera," in Proc. Int. Conf. Comput. Vis., 2009, pp. 167-174.
10. P. Rander, P. J. Narayanan, and T. Kanade, "Virtualized reality: Constructing time-varying virtual worlds from real world events," in Proc. IEEE Conf. Visualization, Phoenix, AZ, 1997, pp. 277-284.
11. T. Kanade, H. Saito, and S. Vedula, "The 3D room: Digitizing timevarying 3D events by synchronized multiple video streams," Robotics Institute, Carnegie Mellon Univ., Pittsburgh, PA, CMU-RI-TR-98-34, 1998, Tech. Rep.
12. J. C. Yang, M. Everett, C. Buehler, and L. McMillan, "A real-time distributed light field camera," in Proc. Eurographics Workshop on Rendering, Pisa, Italy, Jun. 2002, pp. 77-85.
13. C. Zhang and T. Chen, "A self-reconfigurable camera array," in Proc. Eurographics Symp. Rendering, Sweden, 2004, pp. 243-254.
14. K. Li, Q. Dai, and W. Xu, "Markerless shape and motion capture from multiview video sequences," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 3, pp. 320-334, Mar. 2011.
15. E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H. P. Seidel, and S. Thrun, "Performance capture from sparse multi-view video," in Proc. ACM SIGGRAPH, Los Angeles, CA, 2008, pp. 1-10.
16. M. Levoy and P. Hanrahan, "Light field rendering," in Proc. ACMSIGGRAPH, 2006, pp. 31-42.
17. L. C. Zitnicka, S. B. Kang, M. Uyttendaele, S.Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation, " ACM Trans. Graph., vol. 23, no. 3, pp. 600-608, 2004.
18. J. Starck and A. Hilton, "Surface capture for performance based animation," IEEE Comput. Graphics Applicat., vol. 27, no. 3, pp. 21-31, 2007. (Pubitemid 46767722)
19. A. Hilton, J. Guillemaut, J. Kilner, O. Grau, and G. Thomas, "3D-TV production from conventional cameras for sports broadcast," IEEE Trans. Broadcasting, vol. 57, no. 2, pp. 462-476, Jun. 2011.
20. M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, "Ambient point coulds for viewinterpolation, " in Proc. ACM SIGGRAPH, Los Angeles, CA, 2010, pp. 25-29.
21. L. Ballan, G. J. Brostow, J. Puwein, and M. Pollefeys, "Unstructured video-based rendering: interactive exploration of casually captured videos," in Proc. SIGGRAPH'10, Jul. 2010, pp. 1-11.
22. M. Germann, A. Hornung, R. Keiser, R. Ziegler, S. Würmlin, and M. Gross, "Articulated billboards for video-based rendering," Comput. Graphics Forum (Proc. Eurographics), vol. 29, no. 2, pp. 585-594, 2010.
23. A. Hornung and L. Kobbelt, "Interactive pixel-accurate free viewpoint rendering from images with silhouette aware sampling," Comput. Graphics Forum, vol. 28, no. 8, pp. 2090-2103, 2009.
24. T. Stich, C. Linz, G. Albuquerque, and M. A. Magnor, "View and time interpolation in image space," Comput. Graph. Forum.
25. C. Lipski, C. Linz, K. Berger, A. Sellent, and M. Magnor, "Virtual video camera: Image-based viewpoint navigation through space and time," Comput. Graphics Forum, vol. 29, no. 8, pp. 2555-2568, Dec. 2010.
26. F. Klose, C. Lipski, and M. Magnor, "Reconstructing shape and motion from asynchronous cameras," in Proc. Vis., Modeling, Visualization (VMV) 2010, Siegen, Germany, Nov. 2010, pp. 171-177.
27. D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popović, S. Rusinkiewicz, and W. Matusik, "Dynamic shape capture using multi-view photometric stereo," ACM Trans. Graphics, vol. 28, no. 5, pp. 1-11, 2009.
28. E. Shechtman, Y. Caspi, and M. Irani, "Increasing space-time resolution in video," in Proc. Eur. Conf. Computer Vis., 2002, pp. 753-768.
29. E. Shechtman, Y. Caspi, and M. Irani, "Space-time super-resolution," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 531-545, Apr. 2005. (Pubitemid 40556109)
30. A. Veeraraghavan, D. Reddy, and R. Raskar, "Coded strobing photography: Compressive sensing of high speed periodic videos," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 671-686, Apr. 2011.
31. C. Harris and M. Stephens, "A combined corner and edge detection," in Proc. 4th Alvey Vis. Conf., 1988, pp. 147-151.
32. T. Lindeberg, "Feature detection with automatic scale selection," Int. J. Comput. Vis., vol. 30, no. 2, pp. 79-116, 1998. (Pubitemid 128517917)
33. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
34. J. Gao, H. Sultan, J. Hu, and W.-W. Tung, "Denoising nonlinear time series by adaptive filtering and wavelet shrinkage: A comparison," IEEE Signal Process. Lett., vol. 17, no. 3, pp. 237-240, 2010.
35. J. Shapiro, "Embedded image coding using zerotrees of wavelet coefficients," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3445-3462, Dec. 1993.
36. Y. Xu, X. Yang, H. Ling, and H. Ji, "A new texture descriptor using multifractal analysis in multi-orientation wavelet pyramid," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., San Francisco, CA, 2010, pp. 161-168.
37. E. Loupias, N. Sebe, S. Bres, and J.-M. Jolion, "Wavelet-based salient points for image retrieval," in Proc. IEEE Int. Conf. Image Process., 2000, pp. 518-521. (Pubitemid 32465677)
38. C.-L. Liu, M. Koga, and H. Fujisawa, "Gabor feature extraction for character recognition: Comparison with gradient feature," in Proc. Int. Conf. Doc. Anal. Recogn., Washington, DC, 2005, pp. 121-125. (Pubitemid 46442303)
39. N. G. Kingsbury, "The dual-tree complex wavelet transform: A new technique for shift invariance and directional filters," in Proc. IEEE Digital Signal Process. Workshop, Bryce Canyon, UT, 1998.
40. J. Yang, Y. Wang, W. Xu, and Q. Dai, "Image coding using dual-tree discrete wavelet transform," IEEE Trans. Image Process., vol. 17, no. 9, pp. 1555-1569, Sep. 2008.
41. K. Li, Q. Dai, and W. Xu, "Color transfer based on wavelet transform," in Proc. Vis. Commun. Image Process., 2008, no. 682223.
42. J. Yang, Y. Wang, W. Xu, and Q. Dai, "Image and video denoising using adaptive dual-tree discrete wavelet packets," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 5, pp. 642-655, May 2009.
43. J. Fauqueur, N. Kingsbury, and R. Anderson, "Multiscale keypoint detection using the dual-tree complex wavelet transform," in Proc. IEEE Int. Conf. Image Process., Atlanta, GA, 2006, pp. 1625-1628.
44. K. Li, Q. Dai, and W. Xu, "A system and a geometric calibration method for circular camera arrays," China Patent 200710121782.4, Sep. 2007.
45. K. Li, Q. Dai, and W. Xu, "Collaborative color calibration for multicamera systems," Signal Process.: Image Commun., vol. 26, no. 1, pp. 48-60, 2011.
46. J. S. Franco and E. Boyer, "Exact polyhedral visual hulls," in Proc. British Mach. Vis. Conf., Norwich, U.K., 1994.
47. T. Brox, A. Bruhn, N. Papenberg, and J.Weickert, "High accuracy optical flowestimation based on a theory for warping," in Proc. Eur. Conf. Computer Vision, 2004, vol. 3024, pp. 25-36.
48. A. Telea, "An image inpainting technique based on the fast marching method," J. Graphics, GPU, and Game Tools, vol. 9, no. 1, pp. 23-34, 2004.
49. P. Prez, M. Gangnet, and A. Blake, "Poisson image editing," in Proc. ACM SIGGRAPH, 2003, pp. 313-318.
50. S. Belongie and J. Malik, "Matching with shape contexts," in Proc. IEEE Workshop Content-Based Access of Image and Video Libraries, Washington, DC, 2000, pp. 20-26.
51. H. W. Kuhn, "The Hungarian method for the assignment problem," Naval Res. Logist. Quart., vol. 2, no. 1-2, pp. 83-97, 1955.
52. S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A comparison and evaluation of multi-view stereo reconstruction algorithms," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2006, pp. 519-528.
53. Y. Liu, Q. Dai, and W. Xu, "A point-cloud-based multiview stereo algorithm for free-viewpoint video," IEEE Trans. Visual. Comput. Graph., vol. 16, no. 3, pp. 407-418, 2010.
54. X. Luo, J. Zhang, and Q. Dai, "A classification-based image fusion scheme using wavelet transform," in Proc. SPIE 8064, 2011, no. 80640O.
55. Y. Zheng and Z. Qin, "Region-based image fusion method using bidimensional empirical mode decomposition," J. Electron. Imag., vol. 18, no. 1, p. 013008, 2009.
56. T. Brox and J. Malik, "Large displacement optical flow: Descriptor matching in variational motion estimation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 3, pp. 500-513, 2011.
57. K. Li, Q. Dai, W. Xu, J. Yang, and J. Jiang, "Three-dimensional motion estimation via matrix completion," IEEE Trans. Syst., Man, Cybern., B: Cybern., vol. 42, no. 2, pp. 539-551, Apr. 2012.
58. E. J. Candès and B. Recht, "Exact matrix completion via convex optimization," Found. Comput. Math., vol. 9, no. 6, pp. 717-772, 2009.
59. J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, "Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization," in Proc. Neural Inf. Process. Syst., 2009.
60. L.Wu, A. Ganesh, B. Shi, Y. Matsushita, Y.Wang, and Y. Ma, "Robust photometric stereo via low-rank matrix completion and recovery," in Proc. Asian Conf. Comput. Vis., Queenstown, New Zealand, 2010.
61. H. Ji, C. Liu, Z. Shen, and Y. Xu, "Robust video denoising using low rank matrix completion," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., San Francisco, CA, 2010, pp. 1791-1798.