Real-time distance-dependent mapping for a hybrid ToF multi-camera rig

IEEE Journal on Selected Topics in Signal Processing

Volumn 6, Issue 5, 2012, Pages 425-436

  Garcia, Frederic ^a   Aouada, Djamila ^a   Mirbach, Bruno ^b   Ottersten, Böjrn ^a  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

^b IEE S A   (Luxembourg)

Author keywords

3D data fusion; Data matching; mapping; multi sensor systems; multimodal sensors; sensor fusion; time of flight (ToF)

Indexed keywords

3D DATA FUSION; DATA MATCHING; MULTI-SENSOR SYSTEMS; MULTIMODAL SENSOR; SENSOR FUSION; TIME OF FLIGHT;

CAMERAS; MAPPING; SENSORS; TABLE LOOKUP;

SENSOR DATA FUSION;

EID: 84865392601     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2012.2207090     Document Type: Article

References (33)

1. S. Foix, G. Alenyà, and C. Torras, "Exploitation of time-of-flight (ToF) cameras," Institut de Robòtica i Informàtica Industrial, CSIC-UPC, Tech. Rep. IRI-TR-10-07, 2010.
2. A. Kolb, E. Barth, R. Koch, and R. Larsen, "Time-of-flight sensors in computer graphics," Eurographics - State of the Art Reports, pp. 119-134, Mar. 2009.
3. T. Leyvand, C. Meekhof, Y.-C.Wei, J. Sun, and B. Guo, "Kinect identity: Technology and experience," Computer, vol. 44, no. 4, pp. 94-96, Apr. 2011.
4. D. Chan, H. Buisman, C. Theobalt, and S. Thrun, "A noise-aware filter for real-time depth upsampling," in Proc. Workshop on Multi-Camera and Multi-Modal Sensor Fusion Algor. Applicat. (ECCVW), 2008.
5. J. Diebel and S. Thrun, "An application of Markov random fields to range sensing," in NIPS. Cambridge, MA: MIT Press, 2005, pp. 291-298.
6. F. Garcia, B. Mirbach, B. Ottersten, F. Grandidier, and A. Cuesta, "Pixel weighted average strategy for depth sensor data fusion," in Proc. Int. Conf. Image Process. (ICIP), Sep. 2010, pp. 2805-2808.
7. S. Gloud, P. Baumstarck, M. Quigley, Y. N. Andrew, and K. Daphne, "Integrating visual and range data for robotic object detection," in Proc. Workshop Multi-Camera and Multi-Modal Sensor Fusion Algor. Applicat. (ECCVW), 2008.
8. J. Kopf, M. Cohen, D. Lischinski, and M. Uyttendaele, "Joint bilateral upsampling," in SIGGRAPH '07: ACM SIGGRAPH 2007 Papers. New York: ACM, 2007, p. 96.
9. Q. Yang, R. Yang, J. Davis, and D. Nistér, "Spatial-depth super resolution for range images," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (CVPR), 2007, pp. 1-8.
10. J. Zhu, L.Wang, R. Yang, and J. Davis, "Fusion of time-of-flight depth and stereo for high accuracy depth maps," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (CVPR), 2008, pp. 1-8.
11. J. Zhu, L. Wang, R. Yang, J. Davis, and Z. Pan, "Reliability fusion of time-of-flight depth and stereo for high quality depth maps," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1400-1414, Jul. 2011.
12. A. Frick, B. Bartczack, and R. Koch, "3D-TV LDV content generation with a hybrid ToF-multicamera RIG," in Proc. 3DTV-CON, Jun. 2010, pp. 1-4.
13. K. Natroshvili, M. Schmid, M. Stephan, A. Stiegler, and T. Schamm, "Real time pedestrian detection by fusing PMD and CMOS cameras," in Proc. Intell. Veh. Symp., 2008, pp. 925-929.
14. M. Do, Q. Nguyen, H. Nguyen, D. Kubacki, and S. Patel, "Immersive visual communication," IEEE Trans. Signal Process. Mag., vol. 28, no. 1, pp. 58-66, Jan. 2011.
15. E.-K. Lee and Y.-S. Ho, "Generation of multi-viewvideo using a fusion camera system for 3D displays," IEEE Trans. Consumer Electron., vol. 56, no. 4, pp. 2797-2805, Nov. 2010.
16. Ibeo Atutomotive Systems, [Online]. Available: http://www. ibeo-as.com Aug. 2011
17. The Nipon Signal Co., LTD, [Online]. Available: http://www. signal.co.jp, Aug. 2011
18. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comput. Vis., vol. 47, no. 1-3, pp. 7-42, Apr.-Jun. 2002. (Pubitemid 34428419)
19. Y. M. Kim, D. Chan, C. Theobalt, and S. Thrun, "Design and calibration of a multi-viewTOF sensor fusion system," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. Workshops (CVPRW), 2008, pp. 1-7.
20. Y. M. Kim, C. Theobalt, J. Diebel, J. Kosecka, B. Miscusik, and S. Thrun, "Multi-view image and ToF sensor fusion for dense 3D reconstruction," in Proc. IEEE Workshop on 3-D Digital Imaging and Modeling, 3DIM, 2009, pp. 1542-1549.
21. S. Fuchs and G. Hirzinger, "Extrinsic and depth calibration of ToF-cameras," in Proc, IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (CVPR), Jun. 2008, pp. 1-6.
22. T. Kahlmann and H. Ingensand, "Increased accuracy of 3D range imaging camera by means of calibration," in Proc. Opt. 3-D Meas. Tech. VIII, G. Kahmen, Ed., 2007, pp. 101-108.
23. L. Marvin, K. Andreas, and H. Klaus, "Data-fusion of PMD-based distance-information and high-resolution RGB-images," in Proc. Int. Symp. Signals, Circuits, Syst. (ISSCS), 2007, vol. 1, pp. 1-4.
24. J.-Y. Bouguet, Camera Calibration Toolbox for Matlab, [Online]. Available: http://vision.caltech.edu/bouguetj/calib Nov. 2009
25. G. Bradski and A. Kaehler, Learning OpenCV: Computer Vision With the OpenCV Library, 1st ed. Sebastopol, CA: O'Reilly Media, 2008.
26. IEE S.A., 3D MLI Sensor. [Online]. Available: http://www.iee.lu Oct. 2010
27. Point Grey Research, Inc., Flea 2. [Online]. Available: http://www.ptgrey. com/products/flea2/index.asp, Oct. 2010
28. C. A. Rothwell, A. Zisserman, D. A. Forsyth, and J. L. Mundy, "Canonical frames for planar object recognition," in Proc. Eur. Conf. Comput. Vis. (ECCV), 1992, pp. 757-772, Springer Berlin/Heidelberg.
29. F. Garcia, D. Aouada, B. Mirbach, T. Solignac, and B. Ottersten, "Realtime hybrid ToF multi-camera rig fusion system for depth map enhancement," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. Workshops (CVPRW), Jun. 2011, pp. 1-8.
30. F. Garcia, D. Aouada, B. Mirbach, T. Mirbach, and B. Ottersten, "A new multi-lateral filter for real-time depth enhancement," in Proc. Adv. Video and Signal-Based Surveill. (AVSS), 2011.
31. S. Paris and F. Durand, "A fast approximation of the bilateral filter using a signal processing approach," in Int. Journal of Computer Vision. Norwell, MA: Kluwer, 2009, vol. 81, pp. 24-52.
32. F. Porikli, "Constant time o(1) bilateral filtering," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (CVPR), Jun. 2008, pp. 1-8.
33. Q. Yang, K.-H. Tan, and N. Ahuja, "Real-time O(1) bilateral filtering," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (CVPR), Jun. 2009, pp. 557-564.