Real-time visual SLAM for autonomous underwater hull inspection using visual saliency

IEEE Transactions on Robotics

Volumn 29, Issue 3, 2013, Pages 719-733

  Kim, Ayoung ^a   Eustice, Ryan M ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Computer vision; information gain; marine robotics; simultaneous localization and mapping (SLAM); visual saliency

Indexed keywords

INFORMATION GAIN; KEY FRAME SELECTION; MARINE ROBOTICS; NOVELTY DETECTION; SHIP-HULL INSPECTIONS; SIMULTANEOUS LOCALIZATION AND MAPPING; VISUAL SALIENCY; VISUAL SIMULTANEOUS LOCALIZATION AND MAPPINGS;

ALGORITHMS; AUTONOMOUS UNDERWATER VEHICLES; COMPUTER VISION; HULLS (SHIP); INSPECTION; ROBOTS; VISUALIZATION;

ROBOTICS;

EID: 84879041473     PISSN: 15523098     EISSN: None     Source Type: Journal    
DOI: 10.1109/TRO.2012.2235699     Document Type: Article

References (76)

1. A. Kim and R. M. Eustice, "Pose-graph visual SLAM with geometric model selection for autonomous underwater ship hull inspection," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., St. Louis, MO, Oct. 2009, pp. 1559-1565.
2. A. Kim and R. M. Eustice, "Combined visually and geometrically informative link hypothesis for pose-graph visual SLAM using bag-of-words," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., San Francisco, CA, Sep. 2011, pp. 1647-1654.
3. J. Mittleman and D. Wyman, "Underwater ship hull inspection," Naval Eng. J., vol. 92, no. 2, pp. 122-128, Apr. 1980.
4. J. Mittleman and L. Swan, "Underwater inspection for welding and overhaul," Naval Eng. J., vol. 105, no. 5, pp. 37-42, Sep. 1993.
5. R. B. Olds, "Marine mammals systems in support of force protection," in Proc. SSC San Diego Biennial Rev., San Diego, CA, 2003, pp. 131-135.
6. D. Lynn and G. Bohlander, "Performing ship hull inspections using a remotely operated vehicle," in Proc. IEEE/MTS OCEANS Conf. Exhib., Aug. 1999, vol. 2, pp. 555-562. (Pubitemid 30532815)
7. A. Carvalho, L. Sagrilo, I. Silva, J. Rebello, and R. Carneval, "On the reliability of an automated ultrasonic system for hull inspection in ship-based oil production units," Appl. Ocean Res., vol. 25, no. 5, pp. 235-241, Oct. 2003. (Pubitemid 38531629)
8. S. Negahdaripour and P. Firoozfam, "An ROV stereovision system for ship hull inspection," IEEE J. Ocean. Eng., vol. 31, no. 3, pp. 551-546, Jul. 2006.
9. G. S. Bohlander, G. Hageman, F. S. Halliwell, R. H. Juers, and D. C. Lynn, "Automated underwater hull maintenance vehicle," Automated underwater hull maintenance vehicle, Bethesda, MD, Tech. Rep. ADA261504, May 1992.
10. S. Harris and E. Slate, "Lamp ray: Ship hull assessment for value, safety and readiness," in Proc. IEEE/MTS OCEANS Conf. Exhib., vol. 1, Seattle, WA, 1999, pp. 493-500. (Pubitemid 30532973)
11. G. Trimble and E. Belcher, "Ship berthing and hull inspection using the cetus II AUV and MIRIS high-resolution sonar," in Proc. IEEE/MTS OCEANS Conf. Exhib., 2002, vol. 2, pp. 1172-1175.
12. J. Vaganay, M. Elkins, D. Esposito, W. O'Halloran, F. Hover, and M. Kokko, "Ship hull inspection with the HAUV: U.S. Navy and NATO demonstrations results," in Proc. IEEE/MTS OCEANS Conf. Exhib., Boston, MA, Sep. 2006, pp. 1-6.
13. L. Menegaldo, M. Santos, G. Ferreira, R. Siqueira, and L. Moscato, "SIRUS: A mobile robot for floating production storage and offloading (FPSO) ship hull inspection," in Proc. 10th IEEE Int. Workshop Adv. Motion Control, Mar. 2008, pp. 27-32.
14. L. Menegaldo, G. Ferreira, M. Santos, and R. Guerato, "Development and navigation of a mobile robot for floating production storage and offloading ship hull inspection," IEEE Trans. Ind. Electron., vol. 56, no. 9, pp. 3717-3722, Sep. 2009.
15. "Ship hull inspections with AquaMap," Desert Star Syst., Marina, CA, Aug. 2002.
16. H. Durrant-Whyte and T. Bailey, "Simultaneous localization and mapping: Part I," IEEE Robot. Autom. Mag., vol. 13, no. 2, pp. 99-110, Jun. 2006.
17. T. Bailey and H. Durrant-Whyte, "Simultaneous localization and mapping (SLAM): Part II," IEEE Robot. Autom. Mag., vol. 13, no. 3, pp. 108-117, Sep. 2006. (Pubitemid 46530408)
18. P. Ridao, M. Carreras, D. Ribas, and R. Garcia, "Visual inspection of hydroelectric dams using an autonomous underwater vehicle," J. Field Robot., vol. 27, no. 6, pp. 759-778, Nov. 2010.
19. M. Walter, F. Hover, and J. Leonard, "SLAM for ship hull inspection using exactly sparse extended information filters," in Proc. IEEE Int. Conf. Robot. Autom., Pasadena, CA, May 2008, pp. 1463-1470.
20. H. Johannsson, M. Kaess, B. Englot, F. Hover, and J. J. Leonard, "Imaging sonar-aided navigation for autonomous underwater harbor surveillance," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Taipei, Taiwan, Oct. 2010, pp. 4396-4403.
21. J. Vaganay, L. Gurfinkel, M. Elkins, D. Jankins, and K. Shurn, "Hovering autonomous underwater vehicle-system design improvements and performance evaluation results," in Proc. Int. Symp. Unmanned Untethered Submersible Technol., Durham, NH, 2009, pp. 1-14.
22. F. S. Hover, R. M. Eustice, A. Kim, B. Englot, H. Johannsson, M. Kaess, and J. J. Leonard, "Advanced perception, navigation and planning for autonomous in-water ship hull inspection," Int. J. Robot. Res., vol. 31, no. 12, pp. 1445-1464, Oct. 2012.
23. L. G. Weiss, "Autonomous robots in the fog of war," IEEE Spectr., vol. 48, no. 8, pp. 30-36, Aug. 2011.
24. F. Lu and E. Milios, "Globally consistent range scan alignment for environment mapping," Auton. Robot., vol. 4, pp. 333-349, Apr. 1997. (Pubitemid 127508781)
25. K. Konolige, "Large-scale map-making," in Proc. AAAI Nat. Conf. Artif. Intell., San Jose, CA, Jul. 2004, pp. 457-463.
26. R. M. Eustice, H. Singh, and J. J. Leonard, "Exactly sparse delayed-state filters for view-based SLAM," IEEE Trans. Robot., vol. 22, no. 6, pp. 1100-1114, Dec. 2006. (Pubitemid 44921741)
27. E. Olson, J. Leonard, and S. Teller, "Spatially-adaptive learning rates for online incremental SLAM," presented at the Robot.: Sci. Syst. Conf., Atlanta, GA, Jun. 2007.
28. K. Konolige and M. Agrawal, "FrameSLAM: From bundle adjustment to real-time visual mapping," IEEE Trans. Robot., vol. 24, no. 5, pp. 1066-1077, Oct. 2008.
29. G. Grisetti, D. Lodi Rizzini, C. Stachniss, E. Olson, and W. Burgard, "Online constraint network optimization for efficient maximum likelihood map learning," inProc. IEEE Int. Conf. Robot. Autom., Pasadena, CA, May 2008, pp. 1880-1885.
30. M. Kaess, V. Ila, R. Roberts, and F. Dellaert, "The Bayes tree: An algorithmic foundation for probabilistic robot mapping," in Proc. Int. Workshop Algorithmic Found. Robot., Singapore, Dec. 2010, pp. 157-173.
31. M. Kaess, A. Ranganathan, and F. Dellaert, "iSAM: Incremental smoothing and mapping," IEEE Trans. Robot., vol. 24, no. 6, pp. 1365-1378, Dec. 2008.
32. M. Kaess, H. Johannsson, and J. Leonard. (2010). Open source implementation of iSAM. [Online]. Available:http://people.csail.mit.edu/kaess/ isam
33. M. Kaess and F. Dellaert, "Covariance recovery from a square root information matrix for data association," Robot. Auton. Syst., vol. 57, no. 12, pp. 1198-1210, Dec. 2009.
34. R. M. Eustice, O. Pizarro, and H. Singh, "Visually augmented navigation for autonomous underwater vehicles," IEEE J. Ocean. Eng., vol. 33, no. 2, pp. 103-122, Apr. 2008.
35. R. M. Eustice, O. Pizarro, H. Singh, and J. Howland, "UWIT: Underwater image toolbox for optical image processing and mosaicking in MATLAB," in Proc. Int. Symp. Underwater Technol., Tokyo, Japan, Apr. 2002, pp. 141-145.
36. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
37. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-up robust features (SURF)," Comput. Vis. Imag. Understand., vol. 110, no. 3, pp. 346-359, Jun. 2008.
38. C. Wu. (2007). SiftGPU: A GPU implementation of scale invariant feature transform (SIFT) [Online]. Available: http://cs.unc.edu/ccwu/siftgpu
39. R. Haralick, "Propagating covariance in computer vision," in Proc. Int. Conf. Pattern Recognit., vol. 1, Jerusalem, Israel, Oct. 1994, pp. 493-498.
40. A. S. Huang, E. Olson, and D. C. Moore, "LCM: Lightweight communications and marshalling," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., Taipei, Taiwan, Oct. 2010, pp. 4057-4062.
41. L. Itti and C. Koch, "Computational modeling of visual attention," Nature Rev. Neurosci., vol. 2, no. 3, pp. 194-203, 2001.
42. T. Kadir and M. Brady, "Saliency, scale and image description," Int. J. Comput. Vis., vol. 45, no. 2, pp. 83-105, 2001. (Pubitemid 33125858)
43. Y.-J. Lee and J.-B. Song, "Autonomous salient feature detection through salient cues in an HSV color space for visual indoor simultaneous localization and mapping," Adv. Robot., vol. 24, no. 11, pp. 1595-1613, Jan. 2010.
44. M. Johnson-Roberson, "Large-scale multi-sensor 3D reconstructions and visualizations of unstructured underwater environments," Ph.D. dissertation, School Aerospace, Mech. Mechatronic Eng., Univ. Sydney, Sydney, Australia, 2010.
45. E. Nowak, F. Jurie, and B. Triggs, "Sampling strategies for bag-of-features image classification," in Proc. Eur. Conf. Comput. Vis., 2006, vol. 3954, pp. 490-503. (Pubitemid 44029653)
46. L. Marchesotti, C. Cifarelli, and G. Csurka, "A framework for visual saliency detection with applications to image thumbnailing," in Proc. IEEE Int. Conf. Comput. Vis., Sep.-Oct. 2009, pp. 2232-2239.
47. G. Csurka, C. R. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," in Proc. Eur. Conf. Comput. Vis., 2004, pp. 1-22.
48. R. Toldo, U. Castellani, and A. Fusiello, "A bag of words approach for 3D object categorization," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 116-127.
49. T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three-dimensional textons," Int. J. Comput. Vis., vol. 43, no. 1, pp. 29-44, 2001. (Pubitemid 32720091)
50. J. Sivic and A. Zisserman, "Video google: A text retrieval approach to object matching in videos," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2003, vol. 2, pp. 1470-1477.
51. L. Wu, S. Hoi, and N. Yu, "Semantics-preserving bag-of-words models and applications," IEEE Trans. Image Process., vol. 19, no. 7, pp. 1908-1920, Jul. 2010.
52. N. Lazic and P. Aarabi, "Importance of feature locations in bag-of-words image classification," in Proc. IEEE Conf. Acoust. Speech, Signal Process., Apr. 2007, vol. 1, pp. 641-644.
53. M.-J. Hu, C.-H. Li, Y.-Y. Qu, and J.-X. Huang, "Foreground objects recog-nitioninvideo based onbag-of-words model," inProc. Chin. Conf. Pattern Recognit., 2009, pp. 1-5.
54. D. Larlus, J. Verbeek, and F. Jurie, "Category level object segmentation by combining bag-of-words models with Dirichlet processes and random fields," Int. J. Comput. Vis., vol. 88, no. 2, pp. 238-253, Jun. 2010.
55. M. Cummins and P. Newman, "FAB-MAP: Probabilistic localization and mapping in the space of appearance," Int. J. Robot. Res., vol. 27, no. 6, pp. 647-665, Jun. 2008.
56. M. Cummins and P. Newman, "Highly scalable appearance-only SLAM- FAB-MAP 2.0," presented at the Robot.: Sci. Syst. Conf., Seattle, WA, Jun. 2009.
57. A. Angeli, D. Filliat, S. Doncieux, and J.-A. Meyer, "Fast and incremental method for loop-closure detection using bags of visual words," IEEE Trans. Robot., vol. 24, no. 5, pp. 1027-1037, Oct. 2008.
58. A. Kawewong, N. Tongprasit, S. Tangruamsub, and O. Hasegawa, "Online and incremental appearance-based SLAM in highly dynamic environments," Int. J. Robot. Res., vol. 30, no. 1, pp. 33-55, Jan. 2011.
59. H. Shahbazi and H. Zhang, "Application of locality sensitive hashing to realtime loop closure detection," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., Sep. 2011, pp. 1228-1233.
60. T. Nicosevici and R. Garcia, "On-line visual vocabularies for robot navigation and mapping," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., Oct. 2009, pp. 205-212.
61. G. Bradski, "The openCV library," Dr. Dobb's J. Softw. Tools, vol. 120, pp. 122-125, Nov. 2000.
62. B. Julesz, "Textons, the elements of texture perception, and their interactions," Nature, vol. 290, no. 5802, pp. 91-97, Mar. 1981. (Pubitemid 11149681)
63. M. Varma and A. Zisserman, "A statistical approach to material classification using image patch exemplars," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 2032-2047, Nov. 2009.
64. K. S. Jones, "A statistical interpretation of term specificity and its application in retrieval," J. Document., vol. 28, pp. 11-21, 1972.
65. G. Salton andC. S. Yang, "On the specification of term valuesin automatic indexing," J. Document., vol. 29, no. 4, pp. 351-372, 1973.
66. S. Robertson, "Understanding inverse document frequency: On theoretical arguments for idf," J. Document., vol. 60, no. 5, pp. 503-520, 2004. (Pubitemid 39538229)
67. H. Jegou, M. Douze, and C. Schmid, "On the burstiness of visual elements," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1169-1176.
68. O. Chum, J. Philbin, and A. Zisserman, "Near duplicate image detection: Min-Hash and tf-idf weighting," in Proc. Brit. Mach. Vis. Conf., 2008, pp. 493-502.
69. O. Chum and J. Matas, "Unsupervised discovery of co-occurrence in sparse high dimensional data," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3416-3423.
70. C. D. Manning, P. Raghavan, and H. Schtze, Introduction to Information Retrieval. New York: Cambridge Univ. Press, 2008.
71. A. Kim and R. M. Eustice, "Next-best-view visual SLAM for bounded-error area coverage," presented at the IROS Workshop Active Semantic Percept., Vilamoura, Portugal, Oct. 2012.
72. A. Kim, "Active visual SLAM with exploration for autonomous underwater navigation," Ph.D. dissertation, Dept. Mech. Eng., Univ. Michigan, Ann Arbor, Aug. 2012.
73. V. Ila, J. M. Porta, and J. Andrade-Cetto, "Information-based compact pose SLAM," IEEE Trans. Robot., vol. 26, no. 1, pp. 78-93, Feb. 2010.
74. R. Sim and N. Roy, "Global A-optimal robot exploration in SLAM," in Proc. IEEE Int. Conf. Robot. Autom., Barcelona, Spain, Apr. 2005, pp. 661-666. (Pubitemid 46083872)
75. T. Vidal-Calleja, A. Davison, J. Andrade-Cetto, and D. Murray, "Active control for single camera SLAM," in Proc. IEEE Int. Conf. Robot. Autom., vol. FL, Orlando, May 2006, pp. 1930-1936. (Pubitemid 44940351)
76. M. Bryson and S. Sukkarieh, "An information-theoretic approach to autonomous navigation and guidance of an uninhabited aerial vehicle in unknown environments," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., Aug. 2005, pp. 3770-3775.