Multimodal stereo vision system: 3D data extraction and algorithm evaluation

IEEE Journal on Selected Topics in Signal Processing

Volumn 6, Issue 5, 2012, Pages 437-446

  Barrera Campo, Fernando ^a   Lumbreras Ruiz, Felipe ^a,b   Sappa, Angel Domingo ^a  

^a Barcelona Perception Computing Lab   (Spain)

^b UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

Author keywords

Color and infrared images; multimodal stereo rig; sparse 3D maps

Indexed keywords

3D DATA; ACQUISITION SYSTEMS; ALGORITHM EVALUATION; COLOR CAMERAS; DATA SETS; DEPTH MAP; DISPARITY MAP; GRADIENT INFORMATIONS; GROUND TRUTH; IMAGE PLANE; IMAGE RECTIFICATION; INFRARED IMAGES; MULTI-MODAL; MULTI-SPECTRAL; MULTISPECTRAL IMAGES; MUTUAL INFORMATIONS; RECEIVER OPERATING CHARACTERISTICS; STEREO HEAD; STEREO RIGS; STEREO VISION SYSTEM;

CAMERAS; COLOR; EQUIPMENT TESTING; INFRARED IMAGING; STEREO VISION;

THREE DIMENSIONAL;

EID: 84865371805     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2012.2204036     Document Type: Article

References (33)

1. A. Leykin and R. Hammoud, "Pedestrian tracking by fusion of thermalvisible surveillance videos," Mach. Vis. Applicat., vol. 21, pp. 587-595, 2010.
2. A. Fernández-Caballero, J. C. Castillo, J. Serrano-Cuerda, and S. M. Bascón, "Real-time human segmentation in infrared videos," Expert Syst. With Applicat., vol. 38, no. 3, pp. 2577-2584, 2011.
3. S.-H. Jung, J. Eledath, S. B. Johansson, and V. Mathevon, "Egomotion estimation in monocular infra-red image sequence for night vision applications," in Proc. IEEE Workshop Applicat. Comput. Vis., February 2007.
4. D. Firmenich, M. Brown, and S. Süsstrunk, "Multispectral interest points for RGB-NIR image registration," in Proc. IEEE Int. Conf. Image Process., September 2011, pp. 181-184.
5. G. Hermosillo and O. D. Faugeras, "Variational methods for multimodal image matching," Int. J. Comput. Vis., vol. 50, pp. 329-343, 2002. (Pubitemid 35377419)
6. E. P. Bennett, J. L. Mason, and L. McMillan, "Multispectral bilateral video fusion," IEEE Trans. Image Process., vol. 16, no. 5, pp. 1185-1194, May 2007. (Pubitemid 46629548)
7. Y. Y. Schechner and S. K. Nayar, "Generalized mosaicing: Wide field of viewmultispectral imaging," IEEE Trans. Pattern Anal. Mach. Intell, vol. 24, no. 10, pp. 1334-1348, Oct. 2002. (Pubitemid 35327455)
8. D. A. Socolinsky and L. B. Wolff, "Multispectral image visualization through first-order fusion," IEEE Trans. Image Process., vol. 11, no. 8, pp. 923-931, Aug. 2002. (Pubitemid 34974176)
9. J. Han and B. Bhanu, "Fusion of color and infrared video for moving human detection," Pattern Recognit., vol. 40, pp. 1771-1784, June 2007. (Pubitemid 46399786)
10. S. J. Krotosky and M. M. Trivedi, "Person surveillance using visual and infrared imagery," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 8, pp. 1096-1105, Aug. 2008.
11. R. Yang and Y. Chen, "Design of a 3-d infrared imaging system using structured light," IEEE Trans. Instrum. Meas., vol. 60, no. 2, pp. 608-617, Feb. 2011.
12. S. Prakash, P. Y. Lee, and T. Caelli, "3d mapping of surface temperature using thermal stereo," in Proc. Int. Conf. Control, Autom., Robot., Vis., Dec. 2006, pp. 1-4.
13. S. J. Krotosky and M. M. Trivedi, "On color-, infrared-, and multimodal- stereo approaches to pedestrian detection," IEEE Trans. Intell. Transport. Syst., vol. 8, no. 4, pp. 619-629, Dec. 2007. (Pubitemid 350231639)
14. S. J. Krotosky and M. M. Trivedi, "Mutual information based registration of multimodal stereo videos for person tracking," Comput. Vis. Image Understand., vol. 106, no. 2-3, pp. 270-287, 2007. (Pubitemid 46670743)
15. G. Litos, X. Zabulis, and G. Triantafyllidis, "Synchronous image acquisition based on network synchronization," in Proc. Workshop Comput. Vis. Pattern Recognit., Jun. 2006, p. 167.
16. P. Viola and W. M. W., III, "Alignment by maximization of mutual information," Int. J. Comput. Vis., vol. 24, no. 2, 1997.
17. J. Kostlivá, J. C○ ech, and R. S○ ára, "Feasibility boundary in dense and semi-dense stereo matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 1-8.
18. J.-Y. Bouguet, Jul. 2010, Camera Calibration Toolbox for Matlab, [Online]. Available: http://www.vision.caltech.edu/bouguetj/calib-doc//index. html
19. A. Kuijper, "On detecting all saddle points in 2D images," Pattern Recognit. Lett., vol. 25, no. 15, pp. 1665-1672, Nov. 2004.
20. A. Fusiello, E. Trucco, and A. Verri, "A compact algorithm for rectification of stereo pairs," Mach. Vis. Applicat., vol. 12, no. 1, pp. 16-22, 2000.
21. D. Scribner, P. Warren, and J. Schuler, "Extending color vision methods to bands beyond the visible," Mach. Vis. Applicat., vol. 11, pp. 306-312, 2000.
22. N. Morris, S. Avidan, W. Matusik, and H. Pfister, "Statistics of infrared images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 1-7.
23. H. Hirschmuller and D. Scharstein, "Evaluation of stereo matching costs on images with radiometric differences," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 9, pp. 1582-1599, Sep. 2009.
24. F. Barrera, F. Lumbreras, and A. Sappa, "Multimodal template matching based on gradient and mutual information using scale-space," in Proc. IEEE Int. Conf. Image Process., Sep. 2010, no. 10, pp. 2749-2752.
25. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley-Interscience, 1991.
26. J. Kim, V. Kolmogorov, and R. Zabih, "Visual correspondence using energy minimization and mutual information," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Oct. 2003, pp. 1033-1040.
27. J. P. Pluim, J. B. Maintz, and M. A. Viergever, "Image registration by maximization of combined mutual information and gradient information," IEEE Trans. Med. Imaging, vol. 19, no. 8, pp. 809-814, Aug. 2000. (Pubitemid 32031207)
28. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comput. Vis., vol. 47, pp. 7-42, Apr. 2002. (Pubitemid 34428419)
29. H. Hirschmuller and D. Scharstein, "Evaluation of cost functions for stereo matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 1-8.
30. D. Scharstein and R. Szeliski, "High-accuracy stereo depth maps using structured light," in Proc. IEEE Conf. Comput. Vision Pattern Recognit., Jun. 2003, vol. 1, pp. 195-202.
31. B. J. Davis and S. V., "Background-subtraction using contour-based fusion of thermal and visible imagery," Comput. Vis. Image Understand., vol. 106, no. 2-3, pp. 162-182, 2007.
32. A. Kuijper, "Mutual information aspects of scale space images," Pattern Recognit., vol. 37, no. 12, pp. 2361-2373, 2004. (Pubitemid 40134541)
33. J. W. Davis and V. Sharma, "Background-subtraction using contourbased fusion of thermal and visible imagery," Comput. Vision Image Understand., vol. 106, no. 2-3, pp. 162-182, 2007. (Pubitemid 46670736)