Unified multi-lateral filter for real-time depth map enhancement

Image and Vision Computing

Volumn 41, Issue , 2015, Pages 26-41

  Garcia, Frederic ^a   Aouada, Djamila ^b   Mirbach, Bruno ^a   Solignac, Thomas ^a   Ottersten, Björn ^b  

^a IEE S A   (Luxembourg)

^b UNIVERSITY OF LUXEMBOURG   (Luxembourg)

Author keywords

Active sensing; Adaptive filters; Data fusion; Depth enhancement; Multi modal sensors; Sensor fusion; Time of Flight

Indexed keywords

ADAPTIVE FILTERS; BANDPASS FILTERS; CAMERAS; DATA FUSION; IMAGE RESOLUTION; SENSOR DATA FUSION;

ACTIVE SENSING; DEPTH ENHANCEMENT; MULTIMODAL SENSOR; SENSOR FUSION; TIME OF FLIGHT;

ADAPTIVE FILTERING;

EID: 84937560442     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2015.06.008     Document Type: Article

References (48)

1. S. Foix, G. Alenyà, and C. Torras Exploitation of time-of-flight (ToF) cameras Tech. Rep. IRI-TR-10-07 2010 Institut de Robòtica i Informàtica Industrial, CSIC-UPC
2. A. Kolb, E. Barth, R. Koch, and R. Larsen Time-of-flight sensors in computer graphics Eurographics - State of the Art Reports 2009 119 134
3. J. Meisner Collaboration, expertise produce enhanced sensing in Xbox One http://www.engadget.com/2013/10/03/xbox-ones-kinect-to-use-time-of-flight-for-more-exact-measure October 2013
4. D. Chan, H. Buisman, C. Theobalt, and S. Thrun A noise-aware filter for real-time depth upsampling Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (M2SFA2) 2008 1 12
5. J. Diebel, and S. Thrun An application of markov random fields to range sensing NIPS 2005 MIT Press 291 298
6. F. Garcia, B. Mirbach, B. Ottersten, F. Grandidier, and A. Cuesta Pixel weighted average strategy for depth sensor data fusion International Conference on Image Processing (ICIP) 2010 2805 2808
7. S. Gould, P. Baumstarck, M. Quigley, A.Y. Ng, and D. Koller Integrating visual and range data for robotic object detection Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (M2SFA2) 2008 1 12
8. J. Kopf, M. F. Cohen, D. Lischinski, M. Uyttendaele, Joint Bilateral Upsampling, ACM Transactions on Graphics (SIGGRAPH) 26 (3).
9. Q. Yang, R. Yang, J. Davis, and D. Nistér Spatial-depth super resolution for range images IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2007 1 8
10. J. Zhu, L. Wang, R. Yang, and J.E. Davis Fusion of time-of-flight depth and stereo for high accuracy depth maps IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2008 1 8
11. J. Zhu, L. Wang, R. Yang, J.E. Davis, and Z. Pan Reliability fusion of time-of-flight depth and stereo for high quality depth maps IEEE Trans. Pattern Anal. Mach. Intell. 33 7 2011 1400 1414
12. A. Frick, B. Bartczack, and R. Koch 3D-TV LDV content generation with a hybrid ToF-multicamera RIG 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON) 2010 1 4
13. F. Garcia, D. Aouada, B. Mirbach, T. Solignac, and B. Ottersten A new multi-lateral filter for real-time depth enhancement IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS) 2011 42 47
14. K. Natroshvili, M. Schmid, M. Stephan, A. Stiegler, and T. Schamm Real time pedestrian detection by fusing PMD and CMOS cameras Intelligent Vehicles Symposium 2008 925 929
15. K. Iyer, K. Maiti, B. Navathe, H. Kannan, and A. Sharma Multiview video coding using depth based 3d warping IEEE International Conference on Multimedia and Expo (ICME) 2010 1108 1113
16. E.-K. Lee, and Y.-S. Ho Generation of multi-view video using a fusion camera system for 3D displays IEEE Trans. Consum. Electron. 56 4 2010 2797 2805
17. F. Garcia, D. Aouada, B. Mirbach, and B. Ottersten Real-time distance-dependent mapping for a hybrid ToF multi-camera rig IEEE J. Sel. Top. Sign. Proces. 6 5 2012 425 436
18. J. Park, H. Kim, Y.-W. Tai, M. Brown, and I. Kweon High quality depth map upsampling for 3D-TOF cameras IEEE International Conference on Computer Vision (ICCV) 2011 1623 1630
19. M. Elad On the origin of the bilateral filter and ways to improve it IEEE Trans. Image Process. 11 10 2002 1141 1151
20. C. Tomasi, and R. Manduchi Bilateral filtering for gray and color images ICCV 1998 839 846
21. S. Paris, and F. Durand A fast approximation of the bilateral filter using a signal processing approach Int. J. Comput. Vis. 81 2009 24 52
22. F. Porikli Constant time O(1) bilateral filtering IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2008 1 8
23. Q. Yang, K.-H. Tan, and N. Ahuja Real-time O(1) bilateral filtering IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2009 557 564
24. R. Crabb, C. Tracey, A. Puranik, and J. Davis Real-time foreground segmentation via range and color imaging IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2008 1 5
25. K. Al Ismaeil, D. Aouada, B. Mirbach, and B. Ottersten Dynamic super resolution of depth sequences with non-rigid motions IEEE International Conference on Image Processing (ICIP) 2013 660 664
26. S.-Y. Kim, J.-H. Cho, A. Koschan, and M. Abidi Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor Int. Conf. Pattern Recog. (ICPR) 2010 2358 2361
27. F. Garcia, D. Aouada, B. Mirbach, and B. Ottersten A new 1-D colour model and its application to image filtering International Symposium on Image and Signal Processing and Analysis (ISPA) 2011 1 4
28. J. Choi, D. Min, and K. Sohn 2D-plus-depth based resolution and frame-rate up-conversion technique for depth video IEEE Trans. Consum. Electron. 56 4 2010 2489 2497
29. J. Choi, D. Min, D. Kim, and K. Sohn 3D JBU based depth video filtering for temporal fluctuation reduction IEEE International Conference on Image Processing (ICIP) 2010 2777 2780
30. F. Garcia, D. Aouada, B. Mirbach, and B. Ottersten Spatio-temporal ToF data enhancement by fusion IEEE International Conference on International Conference on Image Processing (ICIP) 2012 981 984
31. D. Min, J. Lu, and N.D. Minh Depth video enhancement based on weighted mode filtering IEEE Trans. Image Process. 21 3 2012 1176 1190
32. D. Min, J. Lu, V.-A. Nguyen, and M. Do Weighted mode filtering and its applications to depth video enhancement and coding IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2012 5433 5436
33. U. Hahne, and M. Alexa Depth imaging by combining time-of-flight and on-demand stereo Proceedings of Dynamic 3D Imaging (Dyn3D), DAGM Workshop 2009 70 83
34. R. Nair, F. Lenzen, S. Meister, H. Schafer, C. Garbe, and D. Kondermann High accuracy tof and stereo sensor fusion at interactive rates In Proceedings of European Conference in Computer Vision (ECCV) 2012 1 11
35. Q. Yang, K.-H. Tan, B. Culbertson, and J. Apostolopoulos Fusion of active and passive sensors for fast 3D capture IEEE International Workshop on Multimedia Signal Processing (MMSP) 2010 69 74
36. R. Deriche Recursively implementing the gaussian and its derivatives Tech. Rep 1993 Institut National de Recherche en Informatique et en Automatique, Unite de Recherche Inria-Sophia Antipolis
37. IEE S.A. 3D MLI sensor http://www.iee.lu March 2015
38. Dragonfly2 camera http://www.ptgrey.com/dragonfly2-ieee-1394b-firewire-cameras March 2015
39. Middlebury Stereo Datasets http://vision.middlebury.edu/stereo/data/ March 2015
40. W. Zhou, A. Bovik, H. Sheikh, and E. Simoncelli Image quality assessment: from error visibility to structural similarity IEEE Trans. Image Process. 13 4 2004 600 612
41. F. Garcia, D. Aouada, T. Solignac, B. Mirbach, and B. Ottersten Real-time depth enhancement by fusion for RGB-D cameras IET Comput. Vis. 7 5 2013 1 11
42. R. Lange, and P. Seitz Solid-state time-of-flight range camera IEEE J. Quantum Electron. 37 2001 390 397
43. D. Vose Risk Analysis: A Quantitative Guide 3rd ed. 2007 Wiley
44. G. Bradski, and A. Kaehler Learning OpenCV: Computer Vision with the OpenCV Library 1st ed. 2008 O'Reilly Media
45. F. Garcia, D. Aouada, H.K. Abdella, T. Solignac, B. Mirbach, and B. Ottersten Depth enhancement by fusion for passive and active sensing 2.5D Sensing Technologies in Motion: The Quest for 3D (QU3ST) in conjunction with European Conference on Computer Vision (ECCV) 2012 506 515
46. M. Reynolds, J. Dobos, L. Peely, T. Weyrich, and G.J. Brostow Capturing time-of-flight data with confidence IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2011 945 952
47. F. Garcia, D. Aouada, B. Mirbach, T. Solignac, and B. Ottersten Real-time hybrid ToF multi-camera rig fusion system for depth map enhancement IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2011 1 8
48. K. Al Ismaeil, D. Aouada, B. Mirbach, and B. Ottersten Depth Super-Resolution by Enhanced Shift and Add Lecture Notes in Computer Science Computer Analysis of Images and Patterns 8048 2013 100 107