A survey on Time-of-Flight stereo fusion

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 8200 LNCS, Issue , 2013, Pages 105-127

  Nair, Rahul ^a,b   Ruhl, Kai ^c   Lenzen, Frank ^a,b   Meister, Stephan ^a,b   Schaf̈er, Henrik ^a,b   Garbe, Christoph S ^a,b   Eisemann, Martin ^c   Magnor, Marcus ^c   Kondermann, Daniel ^a,c  

^a UNIVERSITY OF HEIDELBERG   (Germany)

^b SAARLAND UNIVERSITY   (Germany)

^c TECHNICAL UNIVERSITY OF BRAUNSCHWEIG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; IMAGING TECHNIQUES; PATTERN RECOGNITION; SURVEYS;

DEPTH IMAGING TECHNIQUES; DEPTH MAP; DEPTH SENSORS; EVALUATION CRITERIA; HIGH QUALITY; PASSIVE STEREO; TIME OF FLIGHT;

SENSORS;

EID: 84893753652     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-44964-2_6     Document Type: Conference Paper

References (66)

1. Park, J., Kim, H., Tai, Y., Brown, M., Kweon, I.: High quality depth map upsampling for 3d-tof cameras. In: IEEE Proc. ICCV (2011)
2. Huhle, B., Fleck, S., Schilling, A.: Integrating 3d time-of-flight camera data and high resolution images for 3dtv applications. In: Proc. 3DTV Conf. IEEE (2007)
3. Castaneda, V., Mateus, D., Navab, N.: Stereo time-of-flight. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1684-1691. IEEE (2011)
4. Kuhnert, K., Stommel, M.: Fusion of stereo-camera and pmd-camera data for real-time suited precise 3d environment reconstruction. In: Int. Conf. on Intelligent Robots and Systems, pp. 4780-4785. IEEE (2006)
5. Fischer, J., Arbeiter, G., Verl, A.: Combination of time-of-flight depth and stereo using semiglobal optimization. In: Int. Conf. on Robotics and Automation (ICRA), pp. 3548-3553. IEEE (2011)
6. Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics (Proc. of SIGGRAPH) 23 (2004)
7. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23(3), 664-672 (2004)
8. Chen, J., Paris, S., Wang, J., Matusik, W., Cohen, M., Durand, F.: The video mesh: A data structure for image-based three-dimensional video editing. In: Proc. of the International Conference on Computional Photography (ICCP) (2011)
9. Lo, W.Y., van Baar, J., Knaus, C., Zwicker, M., Gross, M.: Stereoscopic 3d copy & paste. ACM Trans. Graph. 29(6), 147:1-147:10 (2010)
10. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S.A.J., Szeliski, R.: Highquality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600-608 (2004)
11. Devernay, F., Beardsley, P.: Stereoscopic Cinema. In: Ronfard, R., Taubin, G. (eds.) Image and Geometry Processing for 3-D Cinematography. Geometry and Computing, vol. 5, pp. 11-51. Springer, Heidelberg (2010)
12. Fei, Y., Yu, M., Shao, F., Jiang, G.: A color correction algorithm of multi-view video based on depth segmentation. In: International Symposium on Computer Science and Computational Technology (ISCSCT 2008), vol. 2, pp. 206-209 (2008)
13. Wilkes, L.: The role of ocula in stereo post production. The Foundry. Whitepaper (2009)
14. Templin, K., Didyk, P., Ritschel, T., Myszkowski, K., Seidel, H.P.: Highlight microdisparity for improved gloss depiction. ACM Transactions on Graphics (Proc. SIGGRAPH) 31(4) (2012)
15. Mcmillan, L., Gortler, S.: Image-based rendering: A new interface between computer vision and computer graphics. SIGGRAPH Comput. Graph. 33, 61-64 (2000)
16. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47(1), 7-42 (2002)
17. Chiu, W.C., Blanke, U., Fritz, M.: Improving the kinect by cross-modal stereo. In: British Machine Vision Conf. BMVA, pp. 116-1 (2011)
18. Hrkać, T., Kalafatić, Z., Krapac, J.: Infrared-visual image registration based on corners and hausdorff distance. In: Ersbøll, B.K., Pedersen, K.S. (eds.) SCIA 2007. LNCS, vol. 4522, pp. 383-392. Springer, Heidelberg (2007)
19. Bilodeau, G., Torabi, A., Morin, F.: Visible and infrared image registration using trajectories and composite foreground images. Image and Vision Computing 29(1), 41-50 (2011)
20. Toet, A., Van Ruyven, L.J., Valeton, J.M.: Merging thermal and visual images by a contrast pyramid. Optical Engineering 28(7), 287789-287789 (1989)
21. Wu, C.: Visualsfm: A visual structure from motion system (2011)
22. Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: Dtam: Dense tracking and mapping in real-time. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2320-2327. IEEE (2011)
23. Horn, B.K., Brooks, M.J.: Shape from shading. MIT Press (1989)
24. Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 334-341. IEEE (2012)
25. Sturmer, M., Penne, J., Hornegger, J.: Standardization of intensity-values acquired by time-of-flight-cameras. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2008), pp. 1-6. IEEE (2008)
26. Kim, Y., Theobalt, C., Diebel, J., Kosecka, J., Miscusik, B., Thrun, S.: Multi-view image and tof sensor fusion for dense 3d reconstruction. In: ICCV Workshops, pp. 1542-1549. IEEE (2009)
27. Zhu, J., Wang, L., Gao, J., Yang, R.: Spatial-temporal fusion for high accuracy depth maps using dynamic mrfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 899-909 (2010)
28. Ghobadi, S.E., Loepprich, O.E., Lottnera, O., Ahmadov, F., Hartmann, K., Weihs, W., Loffeld, O.: Analysis of the personnel safety in a man-machine-cooperation using 2d/3d images. In: Proceedings of the EURON/IARP International Workshop on Robotics for Risky Interventions and Surveillance of the Environment, Benicassim, Spain (January 2008)
29. Bradski, G.: The OpenCV Library. Dr. Dobb's Journal of Software Tools (2000)
30. Bouguet, J.Y.: Camera calibration toolbox for matlab (2004)
31. Hahne, U., Alexa, M.: Combining time-of-flight depth and stereo images without accurate extrinsic calibration. IJISTA 5(3), 325-333 (2008)
32. Nair, R., Lenzen, F., Meister, S., Schäfer, H., Garbe, C., Kondermann, D.: High accuracy TOF and stereo sensor fusion at interactive rates. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part II. LNCS, vol. 7584, pp. 1-11. Springer, Heidelberg (2012)
33. Dal Mutto, C., Zanuttigh, P., Cortelazzo, G.M.: A probabilistic approach to tof and stereo data fusion. In: 3DPVT, Paris, France, vol. 2 (2010)
34. Schiller, I., Beder, C., Koch, R.: Calibration of a pmd-camera using a planar calibration pattern together with a multi-camera setup. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 37, 297-302 (2008)
35. Zhu, J., Wang, L., Yang, R., Davis, J.: Fusion of time-of-flight depth and stereo for high accuracy depth maps. In: Proc. CVPR, pp. 1-8. IEEE (2008)
36. Kim, Y.M., Chan, D., Theobalt, C., Thrun, S.: Design and calibration of a multiview tof sensor fusion system. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2008), pp. 1-7. IEEE (2008)
37. Horn, B.K.: Closed-form solution of absolute orientation using unit quaternions. JOSA A 4(4), 629-642 (1987)
38. Guan, L., Pollefeys, M., et al.: A unified approach to calibrate a network of camcorders and tof cameras. In: Workshop on Multi-Camera and Multi-Modal Sensor Fusion Algorithms and Applications (M2SFA2 2008) (2008)
39. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2004)
40. Dal Mutto, C., Zanuttigh, P., Mattoccia, S., Cortelazzo, G.: Locally consistent toF and stereo data fusion. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 598-607. Springer, Heidelberg (2012)
41. Zhang, L., Curless, B., Seitz, S.M.: Spacetime stereo: Shape recovery for dynamic scenes. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II-367. IEEE (2003)
42. Yang, Q., Tan, K.H., Culbertson, B., Apostolopoulos, J.: Fusion of active and passive sensors for fast 3d capture. In: MMSP (2010)
43. Reynolds, M., Dobos, J., Peel, L., Weyrich, T., Brostow, G.J.: Capturing time-of-flight data with confidence. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 945-952. IEEE (2011)
44. Zhu, J., Wang, L., Yang, R., Davis, J., et al.: Reliability fusion of time-of-flight depth and stereo for high quality depth maps. TPAMI (99), 1 (2011)
45. Song, Y., Glasbey, C.A., van der Heijden, G.W.A.M., Polder, G., Dieleman, J.A.: Combining stereo and time-of-flight images with application to automatic plant phenotyping. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 467-478. Springer, Heidelberg (2011)
46. Gudmundsson, S., Aanaes, H., Larsen, R.: Fusion of stereo vision and time-of-flight imaging for improved 3d estimation. IJISTA 5(3), 425-433 (2008)
47. Bartczak, B., Koch, R.: Dense depth maps from low resolution time-of-flight depth and high resolution color views. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Kuno, Y., Wang, J., Pajarola, R., Lindstrom, P., Hinkenjann, A., Encarnação, M.L., Silva, C.T., Coming, D. (eds.) ISVC 2009, Part II. LNCS, vol. 5876, pp. 228-239. Springer, Heidelberg (2009)
48. Hahne, U., Alexa, M.: Depth imaging by combining time-of-flight and on-demand stereo. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 70-83. Springer, Heidelberg (2009)
49. Dal Mutto, C., Zanuttigh, P., Cortelazzo, G.M.: A probabilistic approach to tof and stereo data fusion. In: 3DPVT, Paris, France (May 2010)
50. Beder,C.,Bartczak,B.,Koch, R.:Acombined approach for estimating patchlets from PMD depth images and stereo intensity images. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 11-20. Springer, Heidelberg (2007)
51. Ruhl, K., Klose, F., Lipski, C., Magnor, M.: Integrating approximate depth data into dense image correspondence estimation. In: Proc. European Conference on Visual Media Production (CVMP) (August 2012)
52. Gandhi, V., Cech, J., Horaud, R.: High-resolution depth maps based on tof-stereo fusion. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 4742-4749. IEEE (2012)
53. Van Meerbergen, G., Vergauwen, M., Pollefeys, M., Van Gool, L.: A hierarchical symmetric stereo algorithm using dynamic programming. International Journal of Computer Vision 47(1-3), 275-285 (2002)
54. Gallup, D., Frahm, J.M., Mordohai, P., Yang, Q., Pollefeys, M.: Real-time plane-sweeping stereo with multiple sweeping directions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007), pp. 1-8. IEEE (2007)
55. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: Proceedings of the Eighth IEEE International Conference on Computer Vision (ICCV 2001), vol. 2, pp. 508-515. IEEE (2001)
56. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11), 1222-1239 (2001)
57. Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(7), 787-800 (2003)
58. Ihler, A.T., Mcallester, D.A.: Particle belief propagation. In: International Conference on Artificial Intelligence and Statistics, pp. 256-263 (2009)
59. Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(4), 401-406 (1998)
60. Hirschmüller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: Proc. CVPR, pp. 1-8. IEEE (2007)
61. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25-36. Springer, Heidelberg (2004)
62. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. In: Pattern Recognition: 29th DAGM Symposium, vol. 29, pp. 214-223 (2007)
63. Nocedal, J.: Updating quasi-newton matrices with limited storage. Mathematics of Computation 25, 773-782 (1980)
64. Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. TPAMI 30(2), 328-341 (2008)
65. Mattoccia, S.: A locally global approach to stereo correspondence. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCVWorkshops), pp. 1763-1770. IEEE (2009)
66. Cech, J., Sara, R.: Efficient sampling of disparity space for fast and accurate matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007), pp. 1-8. IEEE (2007)