Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 32, Issue 5, 2010, Pages 899-909

  Zhu, Jiejie ^a   Wang, Liang ^b   Gao, Jizhou ^b   Yang, Ruigang ^b  

^a University of Central Florida   (United States)

^b UNIVERSITY OF KENTUCKY   (United States)

Author keywords

Data fusion; Global optimization; MRFs; Stereo; Time of flight sensor

Indexed keywords

DEPTH MAP; DYNAMIC SCENES; LOOPY BELIEF PROPAGATION; MRFS; NEW MODEL; PASSIVE STEREO; POSTERIOR PROBABILITY; RANGE SENSORS; SPATIAL TEMPORALS; TEMPORAL COHERENCE; TEMPORAL RELATIONSHIPS; TIME OF FLIGHT; TIME-OF-FLIGHT SENSORS;

GLOBAL OPTIMIZATION; INFORMATION FUSION; SENSORS; STRUCTURAL FRAMES;

SENSOR DATA FUSION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; METHODOLOGY; PHOTOGRAPHY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; THREE DIMENSIONAL IMAGING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAMMETRY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 77949873218     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2009.68     Document Type: Article

References (51)

1. A. Verri and V. Torre, "Absolute Depth Estimate in Stereopsis," Optical Soc. of Am. A, vol.3, no.3, pp. 297-299, 1986.
2. K. Kanatani, "Detection of Surface Orientation and Motion from Texture by a Stereo-Logical Technique," Artificial Intelligence, vol.23, no.2, pp. 213-237, 1984.
3. J. Michels, A. Saxena, and A.Y. Ng, "High Speed Obstacle Avoidance Using Monocular Vision and Reinforcement Learning," Proc. Int'l Conf. Machine Learning, 2005.
4. D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol.47, no.1, pp. 7-42, 2002.
5. Swissranger, Inc., sr-3, http://www.csem.ch/fs/imaging.htm, 2006.
6. Canesta, Inc., Canestavision Electronic Perception Development Kit, http://www.canesta.com/, 2006.
7. Pmdtec, Inc., Photonix Mixer Device, http://www.pmdtec.com, 2008.
8. 3dv Systems, z-cam, http://www.3dvsystems.com, 2004.
9. Y. Boykov, O. Veksler, and R. Zabih, "Markov Random Fields with Efficient Approximations," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.
10. V. Kolmogorov and R. Zabih, "Computing Visual Correspondence with Occlusions via Graph Cuts," Proc. Int'l Conf. Computer Vision, pp. 508-515, 2001.
11. D. Scharsterin and R. Szeliski, vision.middlebury.edu/stereo/, 2006.
12. J. Sun, N.N. Zeng, and H.Y. Shum, "Stereo Matching Using Belief Propagation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.25, no.7, pp. 787-800, July 2003.
13. K.D. Kuhnert and M. Stommel, "Fusion of Stereo-Camera and pmd-Camera Data for Real-Time Suited Precise 3D Environment Reconstruction," Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, pp. 4780-4785, 2006.
14. S.A. Gümundsson, H. Aanæs, and R. Larsen, "Fusion of Stereo Vision and Time-of-Flight Imaging for Improved 3D Estimation," Proc. Int'l Workshop Dynamic 3D Imaging, 2007.
15. J.J. Zhu, L. Wang, R.G. Yang, and J.E. Davis, "Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
16. J. Diebel and S. Thrun, "An Application of Markov Random Fields to Range Sensing," Proc. Advances in Neural Information Processing Systems 18, 2005.
17. Q.X. Yang, R.G. Yang, J.E. Davis, and D. Nister, "Spatial-Depth Super Resolution for Range Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
18. O. Veksler, Y. Boykov, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.23, no.11, pp. 1222-1239, Nov. 2001.
19. A. Blake, C. Rother, M. Brown, P. Perez, and P. Torr, "Interactive Image Segmentation Using an Adaptive Gmmrf Model," Proc. European Conf. Computer Vision, pp. 428-441, 2004.
20. C. Rother, T. Minka, A. Blake, and V. Kolmogorov, "Cosegmenta-tion of Image Pairs by Histogram Matching: Incorporating a Global Constraint into mrfs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 993-1000, 2006.
21. V. Kolmogorov, A. Criminisi, A. Blake, G. Gross, and C. Rother, "Probabilistic Fusion of Stereo with Color and Contrast for Bilayer Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.28, no.9, pp. 1480-1492, Sept. 2006.
22. S. Roth and M. Black, "Fields of Experts: A Framework for Learning Image Priors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 860-867, 2005.
23. J. Wills, S. Agarwal, and S. Belongie, "What Went Where," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 37-44, 2003.
24. S. Roth and M. Black, "On the Spatial Statistics of Optical Flow," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
25. S. Park, M. Park, and M. Kang, "Super-Resolution Image Reconstruction: A Technical Overview," IEEE Signal Processing Magazine, vol.20, no.3, pp. 21-36, May 2003.
26. W.T. Freeman, E.C. Pasztor, and O.T. Carmichael, "Learning Low Level Vision," Int'l J. Computer Vision, vol.40, no.1, pp. 25-47, 2001.
27. M.F. Tappen and W.T. Freeman, "Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical MRF Parameters," Proc. Int'l Conf. Computer Vision, pp. 900-907, 2003.
28. P. Kohli and P.H. Torr, "Dynamic Graph Cuts for Efficient Inference in Markov Random Fields," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.29, no.12, pp. 2079-2088, Dec. 2007.
29. O. Juan and Y. Boykov, "Active Graph Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
30. M. Isard and J. Maccormick, "Dense Motion and Disparity Estimation via Loopy Belief Propagation," Proc. Asia Conf. Computer Vision, 2006.
31. Z.Z. Yin and R. Collins, "Belief Propagation in a 3D Spatio-Temporal mrf for Moving Object Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
32. J. Chen and C.K. Tang, "Spatio-Temporal Markov Random Field for Video Denoising," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
33. K. Alahari, P. Kohli, and P.H.S. Torr, "Reduce, Reuse and Recycle: Efficiently Solving Multi-Label mrfs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
34. K.J. Yoon and I.S. Kweon, "Locally Adaptive Support-Weight Approach for Visual Correspondence Search," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 924-931, 2005.
35. A.T. Ihler, J.W. Fisher III, and A.S. Willsky, "Loopy Belief Propagation: Convergence and Effects of Message Errors," J. Machine Learning Research, vol.6, pp. 905-936, 2005.
36. D. Scharstein and R. Szeliski, "High-Accuracy Stereo Depth Maps Using Structured Light," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
37. P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Belief Propagation for Early Vision," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 261-268, 2004.
38. Q.X. Yang, L. Wang, R.G. Yang, S.G. Wang, M. Liao, and D. Nister, "Real-Time Global Stereo Matching Using Hierarchical Belief Propagation," Proc. British Machine Vision Conf., 2006.
39. Pointgrey Research, Inc., http://www.pointgrey.com, 2006.
40. A.S. Ogale and Y. Aloimonos, "Shape and the Stereo Correspondence Problem," Int'l J. Computer Vision, vol.65, no.3, pp. 147-162, 2005.
41. Q.X. Yang, L. Wang, R.G. Yang, H. Stewenius, and D. Nister, "Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2347-2354, 2006.
42. S. Birchfield and C. Tomasi, "A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.20, no.4, pp. 401-406, Apr. 1998.
43. I. Schiller, C. Beder, and R. Koch, "Calibration of A PMD-Camera Using a Planar Calibration Pattern Together with a Multi-Camera Setup," Proc. Int'l Soc. for Photogrammetry and Remote Sensing, 2008.
44. T. Kahlmann, F. Remondino, and H. Ingensand, "Calibration of the Fast Range Imaging Camera Swissranger for Use in the Surveillance of the Environment," Proc. SPIE: Electro-Optical Remote Sensing II, 2006.
45. M. Linder, M. Lambers, and A. Kolb, "Sub-Pixel Data Fusion and Edge-Enhanced Distance Refinement for 2D/3D Images," Int'l J. Intelligent Systems Technologies and Applications, vol.5, pp. 344-354, 2008.
46. M. Lindner and A. Kolb, "Calibration of the Intensity-Related Distance Error of the PMD TOF-Camera," Proc. SPIE: Intelligent Robots and Computer Vision XXV, pp. 35-42, 2007.
47. H. Gonzales-Banos and J.E. Davis, "Computing Depth under Ambient Illumination Using Multi-Shuttered Light," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 234-241, 2004.
48. T. Kahlmann and H. Ingensand, "Increased Accuracy of 3D Range Imaging Camera by Means of Calibration," Proc. Conf. Optical 3D Measurement Techniques, July 2007.
49. I. Schiller, C. Beder, and R. Koch, "Calibration of a PMD-Camera Using a Planar Calibration Pattern Together with a Multi-Camera Setup," Proc. XXXVII Int'l Soc. for Photogrammetry and Remote Sensing Congress, 2008.
50. M. Linder, M. Lambers, and A. Kolb, "Sub-Pixel Data Fusion and Edge-Enhanced Distance Refinement for 2D/3D Images," Int'l J. Intelligent Systems Technologies and Applications, vol.5, no.3, pp. 344-354, 2008.
51. L. Guan and M. Pollefeys, "A Unified Approach to Calibrate a Network of Camcorders and ToF Cameras," Proc. IEEE Workshop Multi-Camera and Multi-Model Sensor Fusion Algorithms and Applications, 2008.