Selective combination of visual and thermal imaging for resilient localization in adverse conditions: Day and night, smoke and fire

Journal of Field Robotics

Volumn 30, Issue 4, 2013, Pages 641-666

  Brunner, Christopher ^a   Peynot, Thierry ^a   Vidal Calleja, Teresa ^b   Underwood, James ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

DIFFERENT FREQUENCY; ELECTROMAGNETIC SPECTRA; EXPERIMENTAL EVALUATION; LOCALIZATION ERRORS; LOCALIZATION METHOD; PERCEPTION SYSTEMS; REALISTIC CONDITIONS; UNMANNED GROUND VEHICLES;

CAMERAS; INFRARED IMAGING; INTELLIGENT VEHICLE HIGHWAY SYSTEMS; ROBOTICS; SMOKE;

QUALITY CONTROL;

EID: 84878851175     PISSN: 15564959     EISSN: 15564967     Source Type: Journal    
DOI: 10.1002/rob.21464     Document Type: Article

References (75)

1. Ardeshir, Goshtasby, A., &, Nikolov, S., (2007). Guest editorial: Image fusion: Advances in the state of the art. Information Fusion, 8 (2), 114-118.
2. Borges, P., Zlot, R., Bosse, M., Nuske, S., &, Tews, A., (2010). Vision-based localization using an edge map extracted from 3d laser range data. In IEEE International Conference on Robotics and Automation, Anchorage, AK.
3. Brooker, G., (2009). Introduction to sensors for ranging and imaging. SciTech Pub.
4. Brunner, C., &, Peynot, T., (2010). Perception quality evaluation with visual and infrared cameras in challenging environmental conditions. In International Symposium on Experimental Robotics, Delhi, India.
5. Brunner, C., Peynot, T., &, Underwood, J., (2009). Towards discrimination of challenging conditions for UGVs with visual and infrared sensors. In ARAA Australasian Conference on Robotics and Automation, Sydney, Australia.
6. Brunner, C., Peynot, T., &, Vidal-Calleja, T., (2011a). Combining multiple sensor modalities for a localisation robust to smoke. In IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA.
7. Brunner, C., Peynot, T., &, Vidal-Calleja, T., (2011b). Visual metrics for the evaluation of sensor data quality in outdoor perception. International Journal of Intelligent Control and Systems, 16 (2), 142-159.
8. Burgard, W., et al. (2009). A comparison of SLAM algorithms based on a graph of relations. In IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO.
9. Carlson, J., &, Murphy, R., (2005). Use of Dempster-Shafer conflict metric to adapt sensor allocation to unknown environments. In American Association for Artificial Intelligence, Pittsburgh, PA.
10. Castro, M., &, Peynot, T., (2012). Laser-to-radar sensing redundancy for resilient perception in adverse environmental conditions. In ARAA Australasian Conference on Robotics and Automation, Sydney, Australia.
11. Davison, A. J., (2003). Real-time simultaneous localisation and mapping with a single camera. In International Conference on Computer Vision, Nice, France.
12. Davison, A. J., (2005). Active search for real-time vision. In International Conference on Computer Vision, Los Alamitos, CA.
13. Dubbelman, G., van der Mark, W., van den Heuvel, J., &, Groen, F., (2007). Obstacle detection during day and night conditions using stereo vision. In IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA.
14. Fay, D., et al. (2000). Fusion of multi-sensor imagery for night vision: Color visualization, target learning and search. In International Conference on Information Fusion, Paris, France.
15. Ferwerda, J., Pattanaik, S., Shirley, P., &, Greenberg, D., (1996). A model of visual adaptation for realistic image synthesis. In Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA.
16. Finlayson, G., Hordley, S., &, Drew, M., (2006). Removing shadows from images. European Conference on Computer Vision.
17. Fischler, M. A., &, Bolles, R. C., (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commununications of the ACM, 24 (6), 381-395.
18. Foyle, D., Ahumada, A., Larimer, J., &, Townsend Sweet, B., (1993). Enhanced/synthetic vision systems: Human factors research and implications for future systems. SAE Transactions, 101.
19. Green, D. M., &, Swets, J. A., (1989). Signal detection theory and psychophysics. Peninsula Pub.
20. Gu, X., Yu, D., &, Zhang, L., (2005). Image shadow removal using pulse coupled neural network. IEEE Transactions on Neural Networks, 16 (3), 692-698.
21. Hartley, R. I., &, Zisserman, A., (2004). Multiple view geometry in computer vision, 2nd ed. Cambridge University Press.
22. Hines, G., Rahman, Z., Jobson, D., Woodell, G., &, Harrah, S., (2005). Real-time enhanced vision system. In SPIE Enhanced and Synthetic Vision, Orlando, FL.
23. I. T. U. (1999). Subjective video quality assessment methods for multimedia applications.
24. Jung, I., &, Lacroix, S., (2003). High resolution terrain mapping using low altitude aerial stereo imagery. In International Conference on Computer Vision, Nice, France.
25. Kelly, A., et al. (2006). Toward reliable off road autonomous vehicles operating in challenging environments. The International Journal of Robotics Research, 25 (5-6), 449-483.
26. Kobayashi, M., &, Kameyama, K., (2010). Partial image retrieval using sift based on illumination invariant features. In International Conference on Multimedia Computing and Information Technology, Sharjah, UAE.
27. Lanir, J., Maltz, M., Yatskaer, I., &, Rotman, S., (2006). Comparing multispectral image fusion methods for a target detection task. In IEEE International Conference on Information Fusion.
28. Leonard, J., et al. (2008). A perception driven autonomous urban vehicle. Journal of Field Robotics, 25 (10), 727-774.
29. Lewis, J., O'Callaghan, R., Nikolov, S., Bull, D., &, Canagarajah, N., (2007). Pixel- and region-based image fusion with complex wavelets. Information Fusion, 8 (2), 119-130.
30. Lowe, D., (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60 (2), 91-110.
31. Luo, R., &, Su, K., (2003). A multiagent multisensor based real-time sensory control system for intelligent security robot. In IEEE International Conference on Robotics and Automation, Taipei, Taiwan.
32. Mackay, D., (2007). Information Theory, Inference & Learning Algorithms. Cambridge University Press.
33. Maddern, W., &, Vidas, S., (2012). Towards robust night and day place recognition using visible and thermal imaging. In Beyond Laser and Vision: Alternative Sensing Techniques for Robotics Perception, Workshop, Robotics: Science and Systems, Sydney, Australia.
34. Martinsen, G., Hosket, J., &, Pinkus, A., (2008). Correlating military operatorsâ visual demands with multi-spectral image fusion. In Proceedings of SPIE.
35. Mikolajczyk, K., &, Schmid, C., (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (10), 1615-1630.
36. Montiel, J. M. M., Civera, J., &, Davison, A. J., (2006). Unified inverse depth parametrization for monocular SLAM. In Robotics: Science and Systems, Philadelphia, PA.
37. Moreno-Noguer, F., (2011). Deformation and illumination invariant feature point descriptor. In IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO.
38. Nandhakumar, N., &, Aggarwal, J., (1988). Integrated analysis of thermal and visual images for scene interpretation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10 (4), 469-481.
39. Narasimhan, S., &, Nayar, S., (2003). Shedding light on the weather. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI.
40. Nayar, S., &, Narasimhan, S., (1999). Vision in bad weather. In International Conference on Computer Vision, Corfu, Greece.
41. Nuske, S., Roberts, J., &, Wyeth, G., (2009). Robust outdoor visual localization using a three-dimensional-edge map. Journal of Field Robotics, 26 (9), 728-756.
42. Owens, K., &, Matthies, L., (1999). Passive night vision sensor comparison for unmanned ground vehicle stereo vision navigation. In IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, Fort Collins, CO.
43. Paz, L. M., Piniés, P., Tardõs, J. D., &, Neira, J., (2008). Large scale 6DOF slam with stereo-in-hand. IEEE Transactions on Robotics, 24 (5), 946-957.
44. Pearsall, J., editor (1999). The concise Oxford dictionary, 10th ed. Oxford, UK: Oxford Univ. Press.
45. Perbet, J., Baron, L., Parus, R., &, Quancard, S., (1993). Enhanced vision systems. In International Symposium on Head Up Display, Enhanced Vision and Virtual Reality, Amsterdam, The Netherlands.
46. Peynot, T., &, Kassir, A., (2010). Laser-camera data discrepancies and reliable perception in outdoor robotics. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.
47. Peynot, T., Scheding, S., &, Terho, S., (2010). The Marulan data sets: multi-sensor perception in natural environment with challenging conditions. International Journal of Robotics Research, 29 (13), 1602-1607.
48. Peynot, T., Underwood, J., &, Scheding, S., (2009). Towards reliable perception for unmanned ground vehicles in challenging conditions. In IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO.
49. Roberts, J., Tews, A., &, Nuske, S., (2008). Redundant sensing for localisation in outdoor industrial environments. In 6th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human Environments.
50. Sadjadi, F., (2005). Comparative image fusion analysis. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.
51. Scandaliaris, J., &, Sanfeliu, A., (2010). Discriminant and invariant color model for tracking under abrupt illumination changes. In International Conference on Pattern Recognition, Istanbul, Turkey.
52. Scanlan, J., Chabries, D., &, Christiansen, R., (1990). A shadow detection and removal algorithm for 2-d images. In International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM.
53. Solà, J., Monin, A., Devy, M., &, Vidal-Calleja, T., (2008). Fusing monocular information in multi-camera SLAM. IEEE Transactions on Robotics, 24 (5), 958-968.
54. Soleimanpour, S., Ghidary, S., &, Meshgi, K., (2008). Sensor fusion in robot localization using ds-evidence theory with conflict detection using mahalanobis distance. In IEEE International Conference on Cybernetic Intelligent Systems, Chengdu, China.
55. Tan, R., (2008). Visibility in bad weather from a single image. In IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK.
56. Thrun, S., (2006). Winning the DARPA grand challenge: A robot race through the Mojave Desert. In IEEE/ACM International Conference on Automated Software Engineering, Tokyo, Japan.
57. Thrun, S., et al. (2007). Stanley: The robot that won the DARPA grand challenge. Springer.
58. Toet, A., (2003). Natural colour mapping for multiband nightvision imagery. Information Fusion, 4 (3), 155-166.
59. Toet, A., Valeton, J., &, van Ruyven, L., (1989). Merging thermal and visual images by a contrast pyramid. Optical Engineering, 28 (7), 789-792.
60. Torr, P. H., &, Murray, D. W., (1997). The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, 24 (3), 271-300.
61. Torr, P. H. S., (2002). Bayesian model estimation and selection for epipolar geometry and generic manifold fitting. International Journal of Computer Vision, 50 (1), 35-61.
62. Toth, D., Aach, T., &, Metzler, V., (2000). Illumination-invariant change detection. In IEEE Southwest Symposium on Image Analysis and Interpretation.
63. Tumblin, J., &, Rushmeier, H., (1993). Tone reproduction for realistic images. IEEE Computer Graphics and Applications, 13 (6), 42-48.
64. Underwood, J., (2009). Reliable and safe autonomy for ground vehicles in unstructured environments. Ph.D. thesis, University of Sydney, Sydney, Australia.
65. Urmson, C., et al. (2008). Autonomous driving in urban environments: Boss and the urban challenge. Journal of Field Robotics, 25 (8), 425-466.
66. Vedaldi, A., &, Fulkerson, B., (2008). VLFeat: An open and portable library of computer vision algorithms. http://www.vlfeat.org/.
67. Vidal-Calleja, T. A., Berger, C., Sola, J., &, Lacroix, S., (2011). Large scale multiple robot visual mapping with heterogeneous landmarks in semi-structured terrain. Robotics and Autonomous Systems, 59 (9), 654-674.
68. Wang, Z., &, Bovik, A., (2006). Modern image quality assessment. Morgan & Claypool.
69. Ward, G., (1994). A contrast-based scale factor for luminance display. San Diego, CA: Academic Press Professional, Inc.
70. Waxman, A., Aguilar, M., Fay, D., Ireland, D., &, Racamato, J., (1998). Solid-state color night vision: Fusion of low-light visible and thermal infrared imagery. Lincoln Laboratory Journal, 11 (1), 41-60.
71. Waxman, A., Gove, A., Fay, D., Racamato, J., Carrick, J., Seibert, M., &, Savoye, E., (1997). Color night vision: Opponent processing in the fusion of visible and ir imagery. Neural Networks, 10 (1), 1-6.
72. Winkler, S., (2005). Digital video quality: Vision models and metrics. New Delhi: John Wiley & Sons, Ltd.
73. Yu, Y., Huang, K., Chen, W., &, Tan, T., (2012). A novel algorithm for view and illumination invariant image matching. IEEE Transactions on Image Processing, 21 (1), 229-240.
74. Zabih, R., &, Woodfill, J., (1996). A non-parametric approach to visual correspondence. In IEEE Transactions on Pattern Analysis and Machine Intelligence.
75. Zitova, B., &, Flusser, J., (2003). Image registration methods: A survey. Image and Vision Computing, 21, 977-1000.