On measuring the accuracy of SLAM algorithms

Autonomous Robots

Volumn 27, Issue 4, 2009, Pages 387-407

  Kümmerle, Rainer ^a   Steder, Bastian ^a   Dornhege, Christian ^a   Ruhnke, Michael ^a   Grisetti, Giorgio ^a   Stachniss, Cyrill ^a   Kleiner, Alexander ^a  

^a UNIVERSITY OF FREIBURG   (Germany)

Author keywords

Benchmarking; Mapping accuracy; SLAM

Indexed keywords

DATA SETS; ESTIMATION TECHNIQUES; MAPPING ACCURACY; MATCHING ALGORITHM; REFERENCE FRAME; SENSOR MODALITY; SLAM ALGORITHM; SLAM APPROACH;

BENCHMARKING;

EID: 72349089505     PISSN: 09295593     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10514-009-9155-6     Document Type: Article

References (58)

1. Amigoni, F., Gasparini, S., & Gini, M. (2007). Good experimental methodologies for robotic mapping: A proposal. In Proc. of the IEEE int. conf. on robotics & automation (ICRA).
2. Balaguer, B., Carpin, S., & Balakirsky, S. (2007). Towards quantitative comparisons of robot algorithms: Experiences with SLAM in simulation and real world systems. In IROS 2007 workshop.
3. Bar-Shalom, Y., Li, X. R., & Kirubarajan, T. (2001). Estimation with application to tracking and navigation. New York: Wiley.
4. Bonarini, A., Burgard, W., Fontana, G., Matteucci, M., Sorrenti, D. G., & Tardos, J. D. (2006). Rawseeds a project on SLAM benchmarking. In Proceedings of the IROS'06 workshop on benchmarks in robotics research. Available online at http://www.robot.uji.es/EURON/pdfs/LectureNotesIROS06.pdf.
5. Bosse, M., Newman, P. M., Leonard, J. J., & Teller, S. (2003). An ALTAS framework for scalable mapping. In Proc. of the IEEE int. conf. on robotics & automation (ICRA) (pp. 1899-1906). Taipei, Taiwan.
6. Burgard, W., Stachniss, C., Grisetti, G., Steder, B., Kümmerle, R., Dornhege, C., Ruhnke, M., Kleiner, A., & Tardós, J. D. (2009, to appear). A comparison of slam algorithms based on a graph of relations. In Proc. of the int. conf. on intelligent robots and systems (IROS).
7. J. Canny 1986 A computational approach to edge detection IEEE Transactions on Pattern Analysis and Machine Intelligence 8 6 679 698 10.1109/TPAMI.1986.4767851
8. Censi, A. (2006). Scan matching in a probabilistic framework. In Proc. of the IEEE int. conf. on robotics & automation (ICRA) (pp. 2291-2296).
9. Darpa (2007). Darpa Urban Challenge. http://www.darpa.mil/grandchallenge/ .
10. Dellaert, F. (2005). Square Root SAM. In Proc. of robotics: science and systems (RSS) (pp. 177-184). Cambridge, MA, USA.
11. Dellaert, F., Fox, D., Burgard, W., & Thrun, S. (1998). Monte Carlo localization for mobile robots. In Proc. of the IEEE int. conf. on robotics & automation (ICRA). Leuven, Belgium.
12. Dissanayake, G., Durrant-Whyte, H., & Bailey, T. (2000). A computationally efficient solution to the simultaneous localisation and map building (SLAM) problem. In Proc. of the IEEE int. conf. on robotics & automation (ICRA) (pp. 1009-1014). (Pubitemid 30894937)
13. Doucet, A., de Freitas, N., & Gordan, N. (Eds.) (2001). Sequential Monte-Carlo methods in practice. Berlin: Springer.
14. T. Duckett S. Marsland J. Shapiro 2002 Fast, on-line learning of globally consistent maps Autonomous Robots 12 3 287 300 1012.68644 10.1023/A: 1015269615729 (Pubitemid 34566391)
15. EPFL and IROS (2002). Cleaning Robot Contest. http://robotika.cz/ competitions/cleaning2002/en.
16. ESA (2008). Lunar robotics challenge. http://www.esa.int/esaCP/ SEM4GKRTKMF-index-0.html.
17. C. Estrada J. Neira J. D. Tardós 2005 Hierarchical SLAM: Real-time accurate mapping of large environments IEEE Transactions on Robotics 21 4 588 596 10.1109/TRO.2005.844673 (Pubitemid 41226889)
18. Eustice, R., Singh, H., & Leonard, J. J. (2005a). Exactly sparse delayed-state filters. In Proc. of the IEEE int. conf. on robotics & automation (ICRA) (pp. 2428-2435).
19. Eustice, R., Walter, M., & Leonard, J. J. (2005b). Sparse extended information filters: Insights into sparsification. In Proc. of the int. conf. on intelligent robots and systems (IROS) (pp. 641-648). Edmonton, Canada.
20. U. Frese 2006 Treemap: An o(log∈n) algorithm for indoor simultaneous localization and mapping Autonomous Robots 21 2 103 122 10.1007/s10514-006- 9043-2 (Pubitemid 44359124)
21. Frese, U. (2008). Dlr spatial cognition data set. http://www.informatik. uni-bremen.de/agebv/en/DlrSpatialCognitionDataSet.
22. U. Frese P. Larsson T. Duckett 2005 A multilevel relaxation algorithm for simultaneous localisation and mapping IEEE Transactions on Robotics 21 2 1 12 10.1109/TRO.2004.839220 (Pubitemid 40608450)
23. C. Früh A. Zakhor 2004 An automated method for large-scale, ground-based city model acquisition International Journal of Computer Vision 60 5 24 10.1023/B:VISI.0000027787.82851.b6
24. Grisetti, G., Grzonka, S., Stachniss, C., Pfaff, P., & Burgard, W. (2007a). Efficient estimation of accurate maximum likelihood maps in 3D. In Proc. of the int. conf. on intelligent robots and systems (IROS). San Diego, CA, USA.
25. G. Grisetti C. Stachniss W. Burgard 2007 Improved techniques for grid mapping with Rao-Blackwellized particle filters IEEE Transactions on Robotics 23 34 46 10.1109/TRO.2006.889486 (Pubitemid 46443340)
26. Grisetti, G., Stachniss, C., Grzonka, S., & Burgard, W. (2007c). A tree parameterization for efficiently computing maximum likelihood maps using gradient descent. In Proc. of robotics: science and systems (RSS).
27. Gutmann, J.-S., & Konolige, K. (1999). Incremental mapping of large cyclic environments. In Proc. of the IEEE int. symposium on computational intelligence in robotics and automation (CIRA).
28. Hähnel, D., Burgard, W., Fox, D., & Thrun, S. (2003). An efficient FastSLAM algorithm for generating maps of large-scale cyclic environments from raw laser range measurements. In Proc. of the int. conf. on intelligent robots and systems (IROS) (pp. 206-211).
29. Hermosillo, J., Pradalier, C., Sekhavat, S., Laugier, C., & Baille, G. (2003). Towards motion autonomy of a bi-steerable car: Experimental issues from map-building to trajectory execution. In Proc. of the IEEE int. conf. on robotics & automation (ICRA).
30. A. Hoover G. Jean-Baptiste X. Jiang P. J. Flynn H. Bunke D. B. Goldgof K. K. Bowyer D. W. Eggert A. W. Fitzgibbon R. B. Fisher 1996 An experimental comparison of range image segmentation algorithms IEEE Transactions on Pattern Analysis and Machine Intelligence 18 7 673 689 10.1109/34.506791
31. Howard, A., & Roy, N. (2003). Radish: The robotics data set repository, standard data sets for the robotics community. http://radish. sourceforge.net/.
32. Julier, S., Uhlmann, J., & Durrant-Whyte, H. (1995). A new approach for filtering nonlinear systems. In Proc. of the American control conference (pp. 1628-1632).
33. Kaess, M., Ranganathan, A., & Dellaert, F. (2007). iSAM: Fast incremental smoothing and mapping with efficient data association. In Proc. of the IEEE int. conf. on robotics & automation (ICRA).
34. Kümmerle, R., Steder, B., Dornhege, C., Kleiner, A., Grisetti, G., & Burgard, W. (2009). Large scale graph-based SLAM using aerial images as prior information. In Proc. of robotics: science and systems (RSS).
35. J. J. Leonard H. F. Durrant-Whyte 1991 Mobile robot localization by tracking geometric beacons IEEE Transactions on Robotics and Automation 7 4 376 382 10.1109/70.88147 (Pubitemid 21660980)
36. Lu, F., & Milios, E. (1994). Robot pose estimation in unknown environments by matching 2d range scans. In IEEE computer vision and pattern recognition conference (CVPR) (pp. 935-938).
37. F. Lu E. Milios 1997 Globally consistent range scan alignment for environment mapping Autonomous Robots 4 333 349 10.1023/A:1008854305733
38. Montemerlo, M., Thrun, S., Koller, D., & Wegbreit, B. (2003). FastSLAM 2.0: An improved particle filtering algorithm for simultaneous localization and mapping that provably converges. In Proc. of the int. conf. on artificial intelligence (IJCAI) (pp. 1151-1156).
39. Nüchter, A., Lingemann, K., Hertzberg, J., & Surmann, H. (2005). 6d SLAM with approximate data association. In Proc. of the 12th int. conference on advanced robotics (ICAR) (pp. 242-249).
40. Olson, E. (2008). Robust and efficient robotic mapping. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA.
41. Olson, E., Leonard, J., & Teller, S. (2006). Fast iterative optimization of pose graphs with poor initial estimates. In Proc. of the IEEE int. conf. on robotics & automation (ICRA) (pp. 2262-2269).
42. Pfaff, P., Burgard, W., & Fox, D. (2006). Robust Monte-Carlo localization using adaptive likelihood models. In H.I. Christiensen (Ed.), STAR Springer tracts in advanced robotics : Vol. 22. European robotics symposium 2006 (pp. 181-194). Berlin: Springer. (Pubitemid 44895809)
43. Ranganathan, A., Kaess, M., & Dellaert, F. (2007). Loopy sam. In Proc. of the int. conf. on artificial intelligence (IJCAI).
44. RoboCup Federation (2009). RoboCup Competitions. http://www.robocup.org.
45. Scharstein, D., & Szeliski, R. (2002). Middlebury stereo vision page. http://www.middlebury.edu/stereo.
46. R. C. Smith P. Cheeseman 1986 On the representation and estimation of spatial uncertainty International Journal of Robotics Research 5 4 56 68 10.1177/027836498600500404
47. Smith, R., Self, M., & Cheeseman, P. (1990). Estimating uncertain spatial relationships in robotics. In I. Cox & G. Wilfong (Eds.), Autonomous robot vehicles (pp. 167-193). Berlin: Springer.
48. Stachniss, C., Frese, U., & Grisetti, G. (2007a). OpenSLAM. org-give your algorithm to the community. http://www.openslam.org.
49. Stachniss, C., Grisetti, G., Roy, N., & Burgard, W. (2007b). Evaluation of Gaussian proposal distributions for mapping with Rao-Blackwellized particle filters. In Proc. of the int. conf. on intelligent robots and systems (IROS).
50. Symeo GmbH (2008). http://www.symeo.de.
51. S. Thrun 2001 An online mapping algorithm for teams of mobile robots International Journal of Robotics Research 20 5 335 363 10.1177/ 02783640122067435
52. S. Thrun Y. Liu D. Koller A. Y. Ng Z. Ghahramani H. Durrant-Whyte 2004 Simultaneous localization and mapping with sparse extended information filters International Journal of Robotics Research 23 7/8 693 716
53. Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic robotics. Cambridge: MIT Press.
54. Thrun, S. et al. (2006). Winning the Darpa grand challenge. Journal on Field Robotics.
55. Torralba, A., Murphy, K. P., & Freeman, W. T. (2007). Labelme: the open annotation tool. http://labelme.csail.mit.edu/.
56. Uhlmann, J. (1995). Dynamic map building and localization: New theoretical foundations. PhD thesis, University of Oxford.
57. O. Wulf A. Nüchter J. Hertzberg B. Wagner 2008 Benchmarking urban six-degree-of-freedom simultaneous localization and mapping Journal of Field Robotics 25 3 148 163 10.1002/rob.20234
58. Yguel, M., Keat, C. T. M., Braillon, C., Laugier, C., & Aycard, O. (2007). Dense mapping for range sensors: Efficient algorithms and sparse representations. In Proc. of robotics: science and systems (RSS).