Evaluating maps produced by urban search and rescue robots: Lessons learned from RoboCup

Autonomous Robots

Volumn 27, Issue 4, 2009, Pages 449-464

  Balaguer, Benjamin ^a   Balakirsky, Stephen ^b   Carpin, Stefano ^a   Visser, Arnoud ^c  

^a University of California   (United States)

^b NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Multi robot systems; RoboCup; Robot benchmarking; Simultaneous localization and mapping; Urban search and rescue

Indexed keywords

MULTI-ROBOT SYSTEMS; ROBOCUP; ROBOCUP ROBOT; SIMULTANEOUS LOCALIZATION AND MAPPING; URBAN SEARCH AND RESCUE;

BENCHMARKING; INDUSTRIAL ROBOTS; QUALITY CONTROL; ROBOT LEARNING;

MULTIPURPOSE ROBOTS;

EID: 72349093079     PISSN: 09295593     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10514-009-9141-z     Document Type: Article

References (43)

1. S. M. Abdallah D. C. Smar J. S. Zelek 2007 A benchmark for outdoor vision SLAM systems Journal of Field Robotics 24 1/2 145 165 10.1002/rob.20180 (Pubitemid 46464285)
2. Amigoni, F. (2008). Experimental evaluation of some exploration strategies for mobile robots. In IEEE international conference on robotics and automation (pp. 2818-2823).
3. Amigoni, F., & Gallo, A. (2005). A multi-objective exploration strategy for mobile robots. In IEEE international conference on robotics and automation (pp. 3850-3855).
4. Balaguer, B., & Carpin, S. (2008a). Uc mercenary team description paper: Robocup 2008 virtual robot rescue simulation league. In Team description paper for RoboCup 2008.
5. Balaguer, B., Carpin, S. (2008b). Where Am I? A Simulated GPS Sensor for Outdoor Robotic Applications. In S. Carpin et al. (Eds.), Proceedings of the first international conference on simulation, modeling and programming for autonomous robots (pp. 222-233). Berlin: Springer.
6. Balaguer, B., Carpin, S., & Balakirsky, S. (2007). Towards quantitative comparisons of robot algorithms: Experiences with slam in simulation and real world systems. In Workshop on performance evaluation and benchmarking for intelligent robots and systems at IEEE/RSJ IROS.
7. S. Balakirsky S. Carpin A. Kleiner M. Lewis A. Visser J. Wang V. A. Ziparo 2007 Towards heterogeneous robot teams for disaster mitigation: Results and performance metrics from robocup rescue Journal of Field Robotics 24 11-12 943 967 10.1002/rob.20212 (Pubitemid 350302673)
8. Baltes, J. (2000). A benchmark suite for mobile robots. In Proceedings of the IEEE/RSJ conference on intelligent robots and systems (pp. 1101-1106).
9. R. J. Baron 1981 Mechanisms of human facial recognition International Journal of Man Machine Studies 15 137 178 10.1016/S0020-7373(81)80001-6
10. Basilico, N., & Amigoni, F. (2008). On evaluating performance of exploration strategies for autonomous mobile robots. In Proceedings of the performance evaluation and benchmarking for intelligent robots and systems workshop at IEEE/RSJ international conference on intelligent robots and systems.
11. Birk, A. (1996). Learning geometric concepts with an evolutionary algorithm. In The fifth annual conference on evolutionary programming.
12. Birk, A., Poppinga, J., Stoyanov, T., & Nevatia, Y. (2009). Planetary exploration in USARsim: a case study including real world data from Mars. In L. Iocchi, H. Matsubara, A. Weitzenfeld, & C. Zhou (Eds.), Lecture notes in artificial intelligence (LNAI). RoboCup 2008: Robot WorldCup XII. Berlin: Springer.
13. Calisi, D., Iocchi, L., & Nardi, D. (2008). A unified benchmark framework for autonomous mobile robots and vehicles motion algorithms (movema benchmarks). In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008).
14. Carlson, J., Murphy, R., Christopher, S., & Casper, J. (2005). Conflict metric as a measure of sensing quality. In IEEE international conference on robotics and automation.
15. Carpin, S., Stoyanov, T., Nevatia, Y., Lewis, M., & Wang, J. (2006a). Quantitative assessments of usarsim accuracy. In Proceedings of the performance metrics for intelligent systems workshop.
16. Carpin, S., Wang, J., Lewis, M., Birk, A., & Jacoff, A. (2006b). High fidelity tools for rescue robotics: results and perspectives. In LNCS. Robocup 2005: Robot Soccer World Cup IX (pp. 301-311). Berlin: Springer.
17. Carpin, S., Lewis, M., Wang, J., Balakirsky, S., & Scrapper, C. (2007a). Bridging the gap between simulation and reality in urban search and rescue. In LNCS : Vol. 4434. Robocup 2006: Robot Soccer World Cup X (pp. 1-12). Berlin: Springer.
18. Carpin, S., Lewis, M., Wang, J., Balakirsky, S., & Scrapper, C. (2007b). USARSim: a robot simulator for research and education. In Proceedings of the IEEE international conference on robotics and automation (pp. 1400-1405).
19. Collins, J. J., & O'Sullivan, S. (2004). Developing a benchmarking framework for map building paradigms. In Ninth international symposium on artificial life and robots.
20. Collins, T., Collins, J. J., Mansfield, M., & O'Sullivan, S. (2005). Evaluating techniques for resolving redundant information and specularity in occupancy grids. In S. Zhang, & R. Jarvis (Eds.), Proceedings of AI 2006: advances in artificial intelligence (pp. 235-244). Berlin: Springer.
21. Collins, T., Collins, J. J., & Ryan, C. (2007). Occupancy grid mapping: and empirical evaluation. In Proceedings of the Mediterranean conference on control and automation (pp. 1-6).
22. Egerton, S. J., & Callaghan, V. (2004). A benchmark for measuring mobile robot environment modelling performacne. In Proceedings of the IEEE conference on robotics, automation and mechatronics (pp. 407-412).
23. Fontana, G., Matteucci, M., & Sorrenti, D. G. (2008). The RAWSEED proposal for representation-independent benchmarking of SLAM. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008).
24. Frontoni, E., Ascani, A., Mancini, A., & Zingaretti, P. (2008). Performance metric for vision based robot localization. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008).
25. The GeoTIFF website (2009). http://trac.osgeo.org/geotiff.
26. G. Grisetti C. Stachniss W. Burgard 2007 Improved techniques for grid mapping iwht Rao-Blackwellized particle filters IEEE Transactions on Robotics 23 1 36 46 10.1109/TRO.2006.889486 (Pubitemid 46443340)
27. Guyon, I., Gunn, S., Nikravesh, M., & Zadeh, L. (2006). Feature extraction, foundations and applications. Series studies in fuzziness and soft computing. Berlin: Springer.
28. Jacoff, A., Messina, E., Weiss, B., Tadokoro, S., & Nakagawa, Y. (2003). Test arenas and performance metrics for urban search and rescue robots. In International conference on intelligent robots and systems (pp. 3396-3403).
29. B. Kuipers 1978 Modeling spatial knowledge Cognitive Science 2 129 153 10.1016/S0364-0213(78)80003-2
30. Martin, M. C., & Moravec, H. P. (1996). Robot evidence grids. Technical Report CMU-RI-TR-96-06, Robotics Institute-Carnegie Mellon University, March 1996.
31. Michel, O., Bourquin, Y., & Baillie, J. C. (2008). Robotstadium: Online humanoid robot soccer simulation competition. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008).
32. MOAST project (2009). http://sourceforge.net/projects/moast.
33. H. P. Moravec 1988 Sensor fusion in certainty grids for mobile robots AI Magazine 9 3 61 74
34. OpenSlam (2009). http://www.openslam.org.
35. O'Sullivan, S. (2003). An empirical evaluation of map building methodologies in mobile robotics using the feature prediction sonar noise filter and metric grip map benchmarking suite. Master's thesis, University of Limerick.
36. O'Sullivan, S., Collins, J. J., Mansfield, M., Haskett, D., & Eaton, M. (2004a). Linear feature prediction for confidence estimation of sonar readings in map building. In Ninth international symposium on artificial life and robots.
37. O'Sullivan, S., Collins, J. J., Mansfield, M., Haskett, D., & Eaton, M. (2004b). A quantitative evaluation of sonar models and mathematical update methods for map building with mobile robots. In Ninth international symposium on artificial life and robots.
38. Radish-the robotics data set repository (2009). http://radish. sourceforge.net.
39. Sato, K., Yotsukura, S., & Takahashi, T. (2008). To a rescue simulation applicable to anywhere. Team Description Hinomiyagura, July 2008. Proceedings CD.
40. Schmits, T., & Visser, A. (2008). An omnidirectional camera simulation for the USARSim World. In Proceedings of the 12th RoboCup international symposium, July 2008. Proceedings CD. To be published in the Lecture Notes on Artificial Intelligence series.
41. Sim, R., Dudek, G., & Roy, N. (2004). Online control policy optimization for minimizing map uncertainty during exploration. In IEEE international conference on robotics and automation.
42. Stachniss, C., & Burgard, W. (2003). Exploring unknown environments with mobile robots using coverage maps. In 18th joint conference on artificial intelligence.
43. Varsadan, I., Birk, A., Pingsthorn, M., Schwertfeger, S., & Pathak, K. (2008). The Jacobs map analysis toolkit. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008).