Stereoscopic 3D from 2D video with super-resolution capability

Signal Processing: Image Communication

Volumn 23, Issue 9, 2008, Pages 665-676

  Knorr, Sebastian ^a   Kunter, Matthias ^a   Sikora, Thomas ^a  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

2D 3D conversion; Image based rendering; Stereoscopic imaging; Structure from motion; Super resolution stereo

Indexed keywords

DEPTH PERCEPTION; IMAGE QUALITY; IMAGE RECONSTRUCTION; OPTICAL RESOLVING POWER; VIDEO AMPLIFIERS; VIDEO SIGNAL PROCESSING;

2D/3D CONVERSION; IMAGE-BASED RENDERING; STEREOSCOPIC IMAGING; STRUCTURE-FROM-MOTION; SUPER-RESOLUTION STEREO;

THREE DIMENSIONAL;

EID: 53849098019     PISSN: 09235965     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.image.2008.07.004     Document Type: Article

References (49)

1. P. Beardsley, P.H.S. Torr, A. Zisserman, 3D model acquisition from extended image sequences, In: Proceedings of European Conference on Computer Vision (ECCV), 1996, pp. 683-695.
2. Borman S., and Stevenson R.L. Super-Resolution from image sequences-a review. Midwest Symp. Syst. Circuits (1998) 374-378
3. J.-Y. Bouguet, Pyramidal implementation of the Lucas Kanade feature tracker, Intel Corporation, Microprocessor Research Labs, 2000 〈http://www.intel.com/research/mrl/research/opencv/〉.
4. S. Curti, D. Sirtori, F. Vella, 3D effect generation from monocular view, in: Proceedings of the International Symposium on 3D Data Processing Visualization and Transmission (3DPVT), Padova, Italy, 2002.
5. J.-F. Evers-Senne, A. Niemann, R. Koch, Visual reconstruction using geometry guided photo consistency, in: International Workshop on Vision, Modeling, and Visualization (VMV), Aachen, Germany, 2006.
6. C. Fehn, Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV, in: Proceedings of the SPIE: Stereoscopic Displays and Virtual Reality Systems XI, San José, USA, 2004.
7. Fehn C., Kauff P., Op de Beeck M., Ernst F., IJsselsteijn W., Pollefeys M., van Gool L., Ofek E., and Sexton I. An evolutionary and optimised approach on 3D-TV. Int. Broadcast. Conv. (IBC) (2002)
8. Fischler M., and Bolles R. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM (1981) 381-385
9. B.J. Garcia, Approaches to stereoscopic video based on spatio-temporal interpolation, in: Proceedings of SPIE: Stereoscopic Displays and Virtual Reality Systems IV, 2653, 1996, pp. 85-95.
10. P. Harman, J. Flack, S. Fox, M. Dowley, Rapid 2D to 3D conversion, in: Proceedings of SPIE: Stereoscopic Displays and Virtual Reality Systems X, vol. 4660, 2002, pp. 78-86.
11. C. Harris, M. Stephens, A combined corner and edge detector, in: Fourth Alvey Vision Conference, 1988, pp. 147-151.
12. Hartley R., and Sturm P. Triangulation. Comput. Vision Image Understanding 68 2 (1997) 146-157
13. Hartley R., and Zisserman A. Multiple View Geometry (2003), Cambridge University Press, UK
14. E. Imre, S. Knorr, A.A. Alatan, T. Sikora, Prioritized sequential 3D reconstruction in video sequences of dynamic scenes, In: IEEE International Conference on Image Processing (ICIP), 2006, Atlanta, USA
15. M. Irani, S. Peleg, Super resolution from image sequences, in: Proceedings of International Conference on Pattern Recognition (ICPR), Atlantic City, NJ, 1990.
16. ITU, Methodology for the subjective assessment of the quality of television pictures, Recommendation BT.500-10, 2000.
17. Jebara T., Azarbayejani A., and Pentland A. 3D structure from 2D motion. IEEE Signal Process. Mag. 16 3 (1999) 66-84
18. K.T. Kim, M. Siegel, J.Y. Son, Synthesis of a high-resolution 3D stereoscopic image pair from a high-resolution monoscopic image and a low-resolution depth map, in: Proceedings of the SPIE: Stereoscopic Displays and Applications IX, San José, USA, 1998.
19. S. Knorr, T. Sikora, An image-based rendering (IBR) approach for realistic stereo view synthesis of TV broadcast based on structure from motion, In: IEEE International Conference on Image Processing (ICIP), San Antonio, USA, 2007.
20. S. Knorr, E. Imre, B. Özkalayci, A.A. Alatan, T. Sikora, A Modular scheme for 2D/3D conversion of TV broadcast, in: 3rd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT), Chapel Hill, USA.
21. Knorr S., Smolic A., and Sikora T. From 2D- to Stereo- to Multi-View Video, 3DTV-CON (2007), Kos, Greece
22. Knorr S., Kunter M., and Sikora T. Super-Resolution Stereo- and Multi-View Synthesis from Monocular Video Sequences, 3-D Digital Imaging and Modeling (3DIM) (2007), Montréal, Canada
23. Luong Q.-T., and Faugeras O. The fundamental matrix: theory algorithms, and stability analysis. Int. J. Comput. Vision 17 1 (1996) 43-76
24. Luong Q.-T., and Faugeras O. Self calibration of a moving camera from point correspondences and fundamental matrices. Int. J. Comput. Vision 22-3 (1997)
25. L. MacMillan, An Image based approach to three-dimensional computer graphics, Ph.D. Dissertation, 1997, University of North Carolina.
26. Y. Matsumoto, H. Terasaki, K. Sugimoto, T. Arakawa, Conversion system of monocular image sequence to stereo using motion parallax, in: Proceedings of SPIE: Stereoscopic Displays and Virtual Reality Systems IV, vol. 3012, 1997, pp. 108-112.
27. Meesters L.M.J., IJsselsteijn W.A., and Seuntiëns P.J.H. A survey of perceptual evaluations and requirements of three-dimensional TV. IEEE Trans. Circuits Syst. Video Technol. 14 3 (2004) 381-391
28. Mendonca P.R.S., and Cipolla R. A simple technique for self-calibration. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognition (CVPR) (1999)
29. Moustakas K., Tzovaras D., and Strintzis M.G. Stereoscopic video generation based on efficient structure and motion estimation from a monoscopic image sequence. IEEE Trans. Circuits Syst. Video Technol. 15 8 (2005) 1065-1073
30. Nister D. Automatic passive recovery of 3D from images and video. Proceedings of the International Symposium on 3D Processing, Visualization and Transmission (3DPVT) (2004), Thessaloniki, Greece 438-445
31. M. Op de Beeck, A. Redert, Three dimensional video for the home, in: Proceedings of the EUROIMAGE ICAV3D 2001, International Conference on Augmented, Virtual Environments and Three-Dimensional Imaging, 2001, pp. 188-191.
32. Park C., Park M.K., and Kang M.G. Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20 3 (2003) 21-36
33. Pereira F., and Ebrahimi T. The MPEG-4 Book (2002), Prentice-Hall, Englewood Cliffs, NJ
34. M. Pollefeys, Self-calibration and metric 3D reconstruction from uncalibrated image sequences, Ph.D. Thesis, K.U. Leuven, 1999.
35. M. Pollefeys, Tutorial on 3D modeling from images, in: European Conference on Computer Vision (ECCV), 2000.
36. Pollefeys M., Koch R., Vergauwen M., and Van Gool L. Automated reconstruction of 3D scenes from sequences of images. ISPRS J. Photogrammetry Remote Sensing 4 55 (2000) 251-267
37. Ross J. Stereopsis by binocular delay. Nature 248 5446 (1974) 363-364
38. E. Rotem, K. Wolowelsky, D. Pelz, Automatic video to stereoscopic video conversion, in: Proceedings of the SPIE: Stereoscopic Displays and Virtual Reality Systems XII, vol. 5664, 2005, pp. 198-206.
39. In: Schreer O., Kauff P., and Sikora T. (Eds). 3D Videocommunication: Algorithms, Concepts and Real-Time Systems in Human Centered Communication (2005), Wiley, Chichester, England
40. J. Shi, C. Tomasi, Good features to track, in: IEEE International Conference on Computer Vision and Pattern Recognition, Seattle, 1994.
41. P. Sturm, B. Triggs, A factorization based algorithm for multi-image projective structure and motion, in: Proceedings of the European Conference on Computer Vision (ECCV), Cambridge, UK, 1996, pp. 709-720.
42. Szeliski R. Scene reconstruction from multiple cameras. IEEE Signal Processing Society, International Conference on Image Processing (2000), Vancouver, Canada
43. Szeliski R., and Kang S.B. Recovering 3D shape and motion from image streams using non-linear least-squares. J. Visual Commun. Image Representation 5 1 (1994) 10-28
44. C. Tomasi, T. Kanade, Detection and tracking of point features, Technical Report CMU-CS-91-132, Carnegie Mellon University Technical, 1991.
45. Tomasi C., and Kanade T. Shape and motion from image streams: a factorization method. J. Comput. Vision 9 2 (1992) 137-154
46. P.H.S. Torr, A.W. Fitzgibbon, A. Zisserman, Maintaining multiple motion model hypotheses over many views to recover matching and structure, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 1998, pp. 485-491.
47. Torr P.H.S., Fitzgibbon A.W., and Zisserman A. The problem of degeneracy in structure and motion recovery from uncalibrated image sequences. Int. J. Comput. Vision 32 1 (1999) 27-44
48. Triggs B., McLauchlan P., Hartley R., and Fitzgibbon A. Bundle adjustment-a modern synthesis. Vision Algorithms: Theory & Practice, LNCS vol. 1883 (2000), Springer, New York 298-372
49. L. Zhang, J. Tam, D. Wang, Stereoscopic image generation based on depth images, in: IEEE International Conference on Image Processing (ICIP), Singapore, 2004.