Enhancement of dynamic depth scenes by upsampling for precise super-resolution (UP-SR)

Computer Vision and Image Understanding

Volumn 147, Issue , 2016, Pages 38-49

  Al Ismaeil, Kassem ^a   Aouada, Djamila ^a   Mirbach, Bruno ^b   Ottersten, Björn ^a  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

^b IEE S A   (Luxembourg)

Author keywords

Cumulative motion; Depth video; Moving objects; Non rigid motion; Super resolution; Upsampling

Indexed keywords

IMAGE REGISTRATION; MEAN SQUARE ERROR; MOTION ESTIMATION; SIGNAL SAMPLING;

CUMULATIVE MOTION; DEPTH VIDEOS; MOVING OBJECTS; NON-RIGID MOTION; SUPER RESOLUTION; UP SAMPLING;

OPTICAL RESOLVING POWER;

EID: 84964342343     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2016.04.006     Document Type: Article

References (41)

1. Aodha, O.M., Campbell, N., Nair, A., Brostow, G., Patch based synthesis for single depth image super-resolution. European Conf. on Computer Vision, 2012, 71–84. Part III.
2. Aouada, D., Al Ismaeil, K., Ottersten, B., Patch-based statistical performance analysis of upsampling for precise super resolution. Int. Conf. on Computer Vision Theory and Applications, 2015.
3. Babacan, S.D., Molina, R., Katsaggelos, A.K., Variational bayesian super resolution. IEEE Trans. Image Process 20 (2011), 984–999.
4. Beaulieu, N.C., Jiang, S., ML estimation of signal amplitude in Laplace noise. IEEE Conf. Global Telecommun., 2010, 1–5.
5. Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R., Hierarchical model-based motion estimation. Eur. Conf. Comput. Vis. 588 (1992), 237–252.
6. Bouguet, J. Y., Pyramidal implementation of the lukas kanade feature tracker. Description of the algorithm. http://robots.stanford.edu/cs223b04/algo_tracking, 2000.
7. Chatterjee, P., Milanfar, P., 2009. Bias modeling for image denoising. IEEE Asilomar Conf. on Signals, Systems and Computers, 856–859.
8. Cui, Y., Schuon, S., Thrun, S., Stricker, D., Theobalt, C., Algorithms for 3D shape scanning with a depth camera. IEEE Trans. Pattern Anal. Mach. Intell. 35 (2013), 1039–1050.
9. van Eekeren, A.W.M., Schutte, K., Dijk, J., de Lange, D.-J., van Vliet, L.J., Super-resolution on moving objects and background. IEEE Int. Conf. Image Process, 2006, 2709–2712, 10.1109/ICIP.2006.313074.
10. van Eekeren, A.W.M., Schutte, K., van Vliet, L.J., Multiframe super-resolution reconstruction of small moving objects. IEEE Trans. Image Process 19 (2010), 2901–2912.
11. Elad, M., Feuer, A., Super-resolution reconstruction of continuous image sequence. IEEE Trans. Pattern Anal. Mach. Intell. 21:9 (1999), 817–834.
12. Farsiu, S., Elad, M., Milanfar, P., Video-to-video dynamic superresolution for grayscale and color sequences. J. Adv. Signal Process, 2006, 2006, 232.
13. Farsiu, S., Robinson, D., Elad, M., Milanfar, P., Robust shift and add approach to super-resolution. Int. Symp. Opt. Sci. Technol. 5203 (2003), 121–130.
14. Farsiu, S., Robinson, D., Elad, M., Milanfar, P., Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14 (2004), 47–57.
15. Farsiu, S., Robinson, D., Elad, M., Milanfar, P., Fast and robust multi-frame super-resolution. IEEE Trans. Image Process 13 (2004), 1327–1344.
16. Garcia, F., Aouada, D., Mirbach, B., Solignac, T., Ottersten, B., Real-time hybrid tof multi-camera rig fusion system for depth map enhancement. IEEE Int. Conf. Computer Vision and Pattern Recognition Workshops, 2011, 1–8.
17. Garcia, F., Aouada, D., Mirbach, B., Solignac, T., Ottersten, B., A new multi-lateral filter for real-time depth enhancement. IEEE Int. Conf. on Advanced Video and Signal-Based Surveillance, 2011, 42–47.
18. Garcia, F., Aouada, D., Mirbach, B., Ottersten, B., Real-time distance-dependent mapping for a hybrid ToF multi-camera rig. IEEE Journal of Selected Topics in Signal Processing 6:5 (2012), 1–12.
19. Garcia, F., Aouada, D., Mirbach, B., Solignac, T., Ottersten, B., Spatio-temporal ToF data enhancement by fusion. IEEE Int. Conf. Image Process, 2012, 981–984.
20. Garcia, F., Aouada, D., Mirbach, B., Solignac, T., Ottersten, B., Unified multi-lateral filter for real-time depth map enhancement. Image Vision Comput. 41 (2015), 26–41.
21. Hardie, R., Tuinstra, T., Barnard, K., Bognar, J., Armstrong, E., High resolution image reconstruction from digital video with global and non-global scene motion. IEEE Int. Conf. Image Process 1 (1997), 153–156.
22. Al Ismaeil, K., Aouada, D., Mirbach, B., Ottersten, B., Bilateral filter evaluation based on exponential kernels. Int. Conf. Pattern Recogn., 2012, 258–261.
23. Al Ismaeil, K., Aouada, D., Mirbach, B., Ottersten, B., Depth super-resolution by enhanced shift and add. 15th International Conference in Computer Analysis of Images and Patterns, 2013.
24. Al Ismaeil, K.A., Aouada, D., Mirbach, B., Ottersten, B., Dynamic super-resolution of depth sequences with non-rigid motions. IEEE Int. Conf. Image processing, 2013, 660–664.
25. Al Ismaeil, K.A., Aouada, D., Mirbach, B., Ottersten, B., Multi-frame super-resolution by enhanced shift & add. IEEE Int. Symposium on Image and Signal Processing and Analysis, 2013, 171–176.
26. Al Ismaeil, K.A., Aouada, D., Solignac, T., Mirbach, B., Ottersten, B., Real-time non-rigid multi-frame depth video super-resolution. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’15), 7-12 June, 2015, 8–16.
27. Lindner, M., Kolb, A., Compensation of motion artifacts for timeof-flight cameras. Proc. Dynamic 3D Vision Workshop, 5742, 2009, 16–27 Jena, Sep.
28. Milanfar, P., Super-Resolution Imaging. 2010, CRC Press.
29. MLI, D., 2015. http://www.iee.lu/home-page.
30. pmd CamBoard nano, 2015. http://www.pmdtec.com/products_services/reference_design.php.
31. Rajagopalan, A., Kiran, P., Motion-free superresolution and the role of relative blur. J. Opt. Soc. Amer. 20 (2003), 2022–2032.
32. Richard, A.N., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A., Kinectfusion: Real-time dense surface mapping and tracking. IEEE ISMAR, October, 2011.
33. Robinson, Milanfar, Bias-minimizing filters for motion estimation. IEEE Asilomar Conf. on Signals, Systems and Computers, 2003.
34. Robinson, D., Milanfar, P., Statistical performance analysis of super-resolution. IEEE Trans. Image Process. 15 (2006), 1413–1428.
35. Schuon, S., Theobalt, C., Davis, J., Thrun, S., High-quality scanning using time-of-flight depth superresolution. IEEE Computer Vision and Pattern Recognition Workshops, 2008, 1–7.
36. Schuon, S., Theobalt, C., Davis, J., Thrun, S., Lidarboost: Depth superresolution for tof 3d shape scanning. IEEE Int. Conf. Computer Vision and Pattern Recognition, 2009, 343–350.
37. Takeda, H., Milanfar, P., Protter, M., Elad, M., Superresolution without explicit subpixel motion estimation. IEEE Trans. on Image Process 18 (2009), 1958–1975.
38. Xu, L., Jia, J., Kang, S.B., Improving sub-pixel correspondence through upsampling. Journal on Computer Vision and Image Understanding 116 (2012), 250–261.
39. Yang, Q., Tan, K., Culbertson, B., Apostolopoulos, J., Fusion of active and passive sensors for fast 3D capture. IEEE Int. Workshops Multimedia Signal Processing, 2010, 69–74.
40. Yang, Q., Yang, R., Davis, J., Nister, D., Spatial-depth super resolution for range images. IEEE Int. Conf. Computer Vision and Pattern Recognition, 2007, 1–8.
41. Zhu, J., Wang, L., Yang, R., Davis, J., Fusion of time-of-flight depth and stereo for high accuracy depth maps. IEEE Int. Conf. Computer Vision and Pattern Recognition, 2008, 1–8.