Trajectory optimization using reinforcement learning for map exploration

International Journal of Robotics Research

Volumn 27, Issue 2, 2008, Pages 175-196

  Kollar, Thomas ^a   Roy, Nicholas ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Exploration; Reinforcement learning; Trajectory optimization

Indexed keywords

MAP EXPLORATION; TRAJECTORY OPTIMIZATION;

ALGORITHMS; COMPUTER SIMULATION; CONFORMAL MAPPING; MOBILE ROBOTS; OPTIMIZATION; REINFORCEMENT LEARNING;

TRAJECTORIES;

EID: 38649114040     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364907087426     Document Type: Conference Paper

References (60)

1. Acar, E.U., Choset, H., Zhang, Y. and Schervish, M. (2003). Path planning for robotic demining: robust sensor-based coverage of unstructured environments and probabilistic methods. International Journal of Robotics Research, 22 (7-8). 441 - 466.
2. Bagnell, J.D., Kakade, S., Ng, A. and Schneider, J. (2003). Policy search by dynamic programming. Neural Information Processing Systems, Vol. 16. Cambridge, MA, MIT Press.
3. Barraquand, J., Langlois, B. and Latombe, J. (1992). Numerical potential field techniques for robot path planning. IEEE Transactions on Systems, Man, and Cybernetics 22 (2). 224 - 241.
4. Bosse, M., Newman, P.M., Leonard, J.J., and Teller, S. (2005). SLAM in large-scale cyclic environments using the ATLAS framework. International Journal of Robotics Research, 23 (12). 1113 - 1139.
5. Bourgault, F., Makarenko, A., Williams, S., Grocholsky, B. and Durrant-Whyte, H. (2002). Information based adaptive robotic exploration. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Lausanne, EPFL.
6. Bourque, E. and Dudek, G. (2000). On-line construction of iconic maps. Proceedings of the IEEE International Conference in Robotics and Automation, San Francisco, CA.
7. Burges, C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2 (2). 121 - 167.
8. Choset, H. (2000). Coverage of known spaces: The Boustrophedon cellular decomposition. Autonomous Robots, 9: 247 - 253.
9. Choset, H. (2001). Coverage for robotics-a survey of recent results. Annals of Mathematics and Artificial Intelligence, 31 (1). 113 - 126.
10. Cohn, D.A., Ghahramani, Z. and Jordan, M.I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4: 129 - 145.
11. Cover, T.M. and Thomas, J.A. (1991). Elements of Information Theory. New York, Wiley.
12. Davison, A.J., Reid, I.D., Molton, N.D. and Stasse, O. (2007). MonoSLAM: Real-time single camera SLAM. Transactions on Pattern Analysis and Machine Intelligence, 29 (6). 1052 - 1067.
13. Dellaert, F. and Kaess, M. (2006). Square root SAM: simultaneous localization and mapping via square root information smoothing. International Journal of Robotics Research, 25 (12). 1181 - 1203.
14. Duan, K.-B. and Keerthi, S.S. (2005). Which is the best multi-class SVM method? An empirical study. Multiple Classifier Systems, 3541: 278 - 285.
15. Eliazar, A. and Parr, R. (2004). Learning probabilistic motion models for mobile robots. Proceedings of the International Conference on Machine Learning, Vol. 69, p. 32.
16. Eustice, R., Walter, M. and Leonard, J. (2005). Sparse extended information filters: Insights into sparsification. Proceedings of the International Conference on Intelligent Robots and Systems, 3281-3288.
17. Feder, H.J.S., Leonard, J.J. and Smith, C.M. (1999). Adaptive mobile robot navigation and mapping. International Journal of Robotics Research, 18 (7). 650 - 668.
18. Fedorov, V. (1972). Theory of Optimal Experiments. New York, Academic Press.
19. Fox, D., Burgard, W., and Thrun, S. (1998). Active Markov localization for mobile robots. Robotics and Autonomous Systems, 25 (3-4). 195 - 207.
20. Freund, Y., Seung, H., Shamir, E. and Tishby, N. (1997). Selective sampling using the Query By Committee algorithm. Machine Learning, 28: 133 - 168.
21. Ganguli, A., Cortes, J. and Bullo, F. (2006). Distributed deployment of asynchronous guards in art galleries. Proceedings of the American Control Conference, Minneapolis, MN, pp. 1416-1421.
22. Haehnel, D., Burgard, W. and Thrun, S. (2003). Learning compact 3D models of indoor and outdoor environments with a mobile robot. Robotics and Autonomous Systems, 44 (1). 15 - 27.
23. Hhnel, D., Burgard, W., Fox, D. and Thrun, S. (2003). A highly efficient FastSLAM algorithm for generating cyclic maps of large-scale environments from raw laser range measurements. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
24. Haykin, S. (1994). Neural Networks: A Comprehensive Foundation. Toronto, ON, MacMillan.
25. Howard, A. and Roy, N. (2003). The robotics data set repository (Radish). http://radish.sourceforge.net/.
26. Hsu, C. and Lin, C. (2001). A comparison of methods for multi-class support vector machines. Technical Report, Taipei, Taiwan, National Taiwan University.
27. Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering D, 82: 35 - 45.
28. Leonard, J. and Durrant-Whyte, H.F. (1991). Simultaneous map building and localization for an autonomous mobile robot. Proceedings of the IEEE International Workshop on Intelligent Robots and Systems, Osaka, Japan, pp. 1442- 1447.
29. Lu, F. and Milios, E. (1997). Globally consistent range scan alignment for environment mapping. Autonomous Robots, 4: 333 - 349.
30. Makarenko, A., Williams, S., Bourgault, F. and Durrant-Whyte, H. (2002). An experiment in integrated exploration. Proceeding of the IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 534-539. Lousanne, Switzerland.
31. Martinelli, A., Nguyen, V., Tomatis, N. and Siegwart, R. (2007). A relative map approach to SLAM based on shift and rotation invariants. Robotics and Autonomous Systems, 55 (1). 50 - 61.
32. Martinez-Cantin, R. and Castellanos, J. (2005). Unscented SLAM for large-scale outdoor environments. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 328-333.
33. Martinez-Cantin, R., de Freitas, N., Doucet, A. and Castellanos, J. (2007). Active policy learning for robot planning and exploration under uncertainty. Proceedings of Robotics: Science and Systems (RSS), Atlanta, GA.
34. Meliou, A., Krause, A., Guestrin, C. and Hellerstein, J. (2007). Nonmyopic informative path planning in spatio-temporal models. Proceedings of the 22nd Conference on Artificial Intelligence (AAAI), pp. 602-607.
35. Moravec, H. and Martin, M. (1994). Robot navigation by 3D spatial evidence grids. Technical Report, Pittsburgh, PA, Carnegie Mellon University, Robotics Institute, Mobile Robot Laboratory.
36. Moutarlier, P. and Chatila, R. (1989). An experimental system for incremental environment modeling by an autonomous mobile robot. Proceedings of the 5th International Symposium on Robotics Research, Tokyo.
37. Newman, P.M. and Leonard, J.J. (2003). Consistent convergent constant time slam. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-03), Acapulco, Mexico, pp. 1143-1150.
38. Ng.,A.Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E. and Liang, E. (2004). Inverted autonomous helicopter flight via reinforcement learning. Proceedings of the International Symposium on Experimental Robotics.
39. Ollero, A. and Heredia, G. (1995). Stability analysis of mobile robot path tracking. Proceedings of the 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 3, pp. 461-466.
40. Olson, E., Leonard, J. and Teller, S. (2006). Fast iterative optimization of pose graphs with poor initial estimates. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2262-2269.
41. Pfaff, P., Triebel, R., Stachniss, C., Lamon, P., Burgard, W. and Siegwart, R. (2007). Towards mapping of cities. Proceedings of the of the IEEE International Conference on Robotics and Automation (ICRA), Rome, Italy.
42. Porta, J., Spaan, M., Vlassis, N. and Poupart, P. (2006). Point-based value iteration for continuous POMDPs. Journal of Machine Learning Research, 7: 2329 - 2367.
43. Roy, N. and McCallum, A. (2001). Toward optimal active learning through Monte Carlo estimation of error reduction. Proceedings of the International Conference on Machine Learning (ICML 2001). Williamstown, MA.
44. Roy, N. and Thrun, S. (1999). Coastal navigation with mobile robots. Advances in Neural Processing Systems, 12: 1043 - 1049.
45. Sim, R. (2006). On visual maps and their automatic construction. Ph.D. Thesis, McGill University.
46. Sim, R., Dudek, G. and Roy, N. (2004). Online control policy optimization for minimizing map uncertainty during exploration. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2004). New Orleans, LA, pp. 1758-1763.
47. Singh, A., Krause, A., Guestrin, C., Kaiser, W. and Batalin, M. (2007). Efficient planning of informative paths for multiple robots. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI).
48. Smith, R., Self, M. and Cheeseman, P. (1990). Estimating uncertain spatial relationships in robotics. Autonomous Robot Vehicles. Orlando, FL, Springer, pp. 167 - 193.
49. Sondik, E. (1971). The optimal control of partially observable Markov decision processes. Ph.D. Thesis, Stanford, CA, Stanford University.
50. Stachniss, C. (2006). Exploration and mapping with mobile robots. Ph.D. Thesis, University of Freiburg, Department of Computer Science.
51. Stachniss, C. and Burgard, W. (2003). Exploring unknown environments with mobile robots using coverage maps. Proceedings of the International Conference on Artificial Intelligence (IJCAI), Acapulco, Mexico.
52. Stachniss, C., Grisetti, G. and Burgard, W. (2005). Information gain-based exploration using Rao-Blackwellized particle filters. Proceedings of Robotics: Science and Systems (RSS), Cambridge, MA, pp. 65-72.
53. Thrun, S. (1998). Bayesian landmark learning for mobile robot localization. Machine Learning, 33: 41 - 76.
54. Thrun, S. (2000). Monte Carlo POMDPs. Advances in Neural Information Processing Systems, 12: 1064 - 1070.
55. Thrun, S., Burgard, W. and Fox, D. (2005). Probabilistic Robotics. Cambridge, MA, MIT Press.
56. Thrun, S., Liu, Y., Koller, D., Ng, A., Ghahramani, Z. and Durrant-Whyte, H. (2004). Simultaneous localization and mapping with sparse extended information filters. International Journal of Robotics Research, 23 (7-8). 693 - 716.
57. Thrun, S. and Montemerlo, M. (2005). The GraphSLAM algorithm with applications to large-scale mapping of urban structures. International Journal on Robotics Research, 25 (5-6). 403 - 430.
58. Walter, M., Eustice, R. and Leonard, J. (2005). A provably consistent method for imposing exact sparsity in feature-based slam information filters. Proceedings of the 12th International Symposium of Robotics Research (ISRR), San Francisco, CA.
59. Walter, M. and Leonard, J. (2004). An experimental investigation of cooperative SLAM. Proceedings of the 5th IFAC Symposium on Intelligent Autonomous Vehicles, Lisbon, Portugal.
60. Whaite, P. and Ferrie, F. (1997). Autonomous exploration: driven by uncertainty. IEEE Transactions on Patten Analysis and Machine Intelligence, 19 (3). 193 - 205.