A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot

Autonomous Robots

Volumn 27, Issue 2, 2009, Pages 93-103

  Martinez Cantin, Ruben ^a   De Freitas, Nando ^b   Brochu, Eric ^b   Castellanos, José ^c   Doucet, Arnaud ^b  

^a INSTITUTO SUPERIOR TÉCNICO   (Portugal)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c UNIVERSITY OF ZARAGOZA   (Spain)

Author keywords

Active learning; Active SLAM; Active vision; Attention and gaze planning; Bayesian optimization; Dynamic sensor networks; Model predictive control; Online path planning; Policy search; Reinforcement learning; Sequential experimental design

Indexed keywords

ACTIVE LEARNING; ACTIVE SLAM; ACTIVE VISION; ATTENTION AND GAZE PLANNING; BAYESIAN OPTIMIZATION; DYNAMIC SENSOR NETWORKS; ONLINE PATH PLANNING; POLICY SEARCH; SEQUENTIAL EXPERIMENTAL DESIGN;

BAYESIAN NETWORKS; EDUCATION; MOBILE ROBOTS; MODEL PREDICTIVE CONTROL; OPTIMIZATION; PREDICTIVE CONTROL SYSTEMS; REINFORCEMENT; REINFORCEMENT LEARNING; SENSOR NETWORKS; STATISTICS; WIRELESS NETWORKS;

ROBOT PROGRAMMING;

EID: 70349325516     PISSN: 09295593     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10514-009-9130-2     Document Type: Article

References (55)

1. Bailey, T., Nieto, J., Guivant, J., Stevens, M., & Nebot, E. (2006). Consistency of the EKF-SLAM algorithm. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems, 2006.
2. J. Baxter P. L. Bartlett 2001 Infinite-horizon policy-gradient estimation Journal of Artificial Intelligence Research 15 4 319 350 0994.68119 10.1016/S0954-1810(01)00028-0 1884081 (Pubitemid 33076651)
3. Bergman, N. (1999). Recursive Bayesian estimation: navigation and tracking applications. PhD thesis, Linköping University.
4. Bertsekas, D. (1995). Dynamic programming and optimal control. Nashua: Athena Scientific.
5. Brochu, E., de Freitas, N., & Ghosh, A. (2007). Active preference learning with discrete choice data. In Advances in neural information processing systems, 2007.
6. M. Bryson S. Sukkarieh 2008 Observability analysis and active control for airborne SLAM IEEE Transaction on Aerospace Electronic Systems 44 1 261 280 10.1109/TAES.2008.4517003 (Pubitemid 351772586)
7. K. Chaloner I. Verdinelli 1995 Bayesian experimental design: a review Journal of Statistical Science 10 273 304 0955.62617 10.1214/ss/1177009939 1390519
8. H. Durrant-Whyte T. Bailey 2006 Simultaneous localisation and mapping (SLAM): part I the essential algorithms Robotics and Automation Magazine 13 99 110 10.1109/MRA.2006.1638022 (Pubitemid 43898863)
9. Finkel, D. (2003). DIRECT optimization algorithm user guide. Center for Research in Scientific Computation, North Carolina State University.
10. Gablonsky, J. (2001). Modification of the DIRECT algorithm. PhD thesis, Department of Mathematics, North Carolina State University, Raleigh, North Carolina.
11. Hernandez, M. (2004). Optimal sensor trajectories in bearings-only tracking. In P. Svensson & J. Schubert (Eds.), Proc. of the seventh int. conf. on information fusion, international society of information fusion, Mountain View, CA (Vol. II, pp. 893-900).
12. M. Hernandez T. Kirubarajan Y. Bar-Shalom 2004 Multisensor resource deployment using posterior Cramèr-Rao bounds IEEE Transactions on Aerospace Electronic Systems 40 2 399 416 10.1109/TAES.2004.1309993
13. Howard, M., Klanke, S., Gienger, M., Goerick, C., & Vijayakumar, S. (2009). A novel method for learning policies from variable constraint data. Autonomous Robots, 27 (Special issue on Robot Learning, Part B) (this issue).
14. D. Jones 2001 A taxonomy of global optimization methods based on response surfaces Journal of Global Optimization 21 345 383 01845262 10.1023/A:1012771025575 (Pubitemid 33672374)
15. D. Jones C. Perttunen B. Stuckman 1993 Lipschitzian optimization without the Lipschitz constant Journal of Optimization Theory and Applications 79 1 157 181 0796.49032 10.1007/BF00941892 1246501
16. D. Jones M. Schonlau W. Welch 1998 Efficient global optimization of expensive black-box functions Journal of Global Optimization 13 4 455 492 0917.90270 10.1023/A:1008306431147 1673460
17. Kato, H., & Billinghurst, M. (1999). Marker tracking and hmd calibration for a video-based augmentedreality conferencing system. In Proc. of the 2nd IEEE and ACM int. work. on augmented reality (pp. 85-94) 1999.
18. T. Kollar N. Roy 2008 Trajectory optimization using reinforcement learning for map exploration International Journal of Robotics Research 27 2 175 197 10.1177/0278364907087426 (Pubitemid 351169713)
19. V. Konda J. Tsitsiklis 2003 On actor-critic algorithms SIAM Journal on Control and Optimization 42 4 1143 1166 1049.93095 10.1137/S0363012901385691 2044789
20. Kueck, H., de Freitas, N., & Doucet, A. (2006). SMC samplers for Bayesian optimal nonlinear design. In Nonlinear statistical signal processing workshop (NSSPW), 2006.
21. H. Kushner 1964 A new method of locating the maximum of an arbitrary multipeak curve in the presence of noise Journal of Basic Engineering 86 97 106
22. Leung, C., Huang, S., Dissanayake, G., & Forukawa, T. (2005). Trajectory planning for multiple robots in bearing-only target localisation. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems, 2005.
23. Lizotte, D. (2008). Practical Bayesian optimization. PhD thesis, Dept. of Computer Science, University of Alberta.
24. Lizotte, D., Wang, T., Bowling, M., & Schuurmans, D. (2007). Automatic gait optimization with Gaussian process regression. In International joint conference on artificial intelligence, 2007.
25. M. Locatelli 1997 Bayesian algorithms for one-dimensional global optimization Journal of Global Optimization 10 57 76 0871.90087 10.1023/A:1008294716304 1432103
26. Maciejowski, J. (2002). Predictive control: with constraints. New York: Prentice-Hall.
27. Martinez-Cantin, R. (2008). Active map learning for robots: insights into statistical consistency. PhD thesis, University of Zaragoza.
28. Martinez-Cantin, R., de Freitas, N., & Castellanos, J. (2006). Analysis of particle methods for simultaneous robot localization and mapping and a new algorithm: Marginal-SLAM. In Proc. of the IEEE int. conf. on robotics & automation, 2006.
29. Martinez-Cantin, R., de Freitas, N., & Castellanos, J. (2007a). Active policy learning for robot planning and exploration under uncertainty. In Proc. of robotics: science and systems, 2007.
30. Martinez-Cantin, R., de Freitas, N., Doucet, A., & Castellanos, J. (2007b). Active policy learning for robot planning and exploration under uncertainty. In Robotics: science and systems (RSS), 2007.
31. Meger, D., Marinakis, D., Rekleitis, I., & Dudek, G. (2009). Inferring a probability distribution function for the pose of a sensor network using a mobile robot. In: ICRA, 2009.
32. G. Metta P. Fitzpatrick L. Natale 2006 Yarp: yet another robot platform International Journal on Advanced Robotics Systems 3 1 140 151
33. Mockus, J., Tiesis, V., & Zilinskas, A. (1978). The application of Bayesian methods for seeking the extremum. In L. Dixon & G. Szego (Eds.), Towards global optimisation (Vol. 2, pp. 117-129). Amsterdam: Elsevier.
34. Ng, A., & Jordan, M. (2000). PEGASUS: a policy search method for large MDPs and POMDPs. In Proc. of the sixteenth conf. on uncertainty in artificial intelligence, 2000.
35. Paris, S., & Le Cadre, J. (2002). Planification for terrain-aided navigation. In Fusion 2002, Annapolis, Maryland (pp. 1007-1014).
36. Peters, J., & Schaal, S. (2006). Policy gradient methods for robotics. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems, 2006.
37. J. Peters S. Schaal 2008 Natural actor critic Neurocomputing 71 7-9 1180 1190 10.1016/j.neucom.2007.11.026
38. J. Peters S. Schaal 2008 Reinforcement learning of motor skills with policy gradients Neural Networks 21 4 682 697 10.1016/j.neunet.2008.02.003
39. Rasmussen, C., & Williams, C. (2006). Gaussian processes for machine learning. Cambridge: The MIT Press.
40. M. Riedmiller T. Gabel R. Hafner S. Lange 2009 Reinforcement learning for robot soccer Autonomous Robots 27 1 55 73 10.1007/s10514-009-9120-4 (Special issue on Robot Learning, Part A)
41. Sasena, M. (2002). Flexibility and efficiency enhancement for constrained global design optimization with Kriging approximations. PhD thesis, University of Michigan.
42. Schonlau, M., Welch, W., & Jones, D. (1998). Global versus local search in constrained optimization of computer models. In N. Flournoy, W. Rosenberger, W. Wong (Eds.) New developments and applications in experimental design (Vol. 34, pp. 11-25). Institute of Mathematical Statistics.
43. Sim, R., & Roy, N. (2005). Global A-optimal robot exploration in SLAM. In Proc. of the IEEE int. conf. on robotics & automation, 2005.
44. Singh, A., Krause, A., Guestrin, C., Kaiser, W., & Batalin, M. (2007). Efficient planning of informative paths for multiple robots. In Proc. of the int. joint conf. on artificial intelligence, 2007.
45. A. Singh A. Krause C. Guestrin W. Kaiser 2009 Efficient informative sensing using multiple robots Journal of Artificial Intelligence Research (JAIR) 34 707 755
46. Singh, S., Kantas, N., Doucet, A., Vo, B., & Evans, R. (2005). Simulation-based optimal sensor scheduling with application to observer trajectory planning. In Proc. of the IEEE conf. on decision and control and eur. control conference (pp. 7296-7301) 2005.
47. R. Smallwood E. Sondik 1973 The optimal control of partially observable Markov processes over a finite horizon Operations Research 21 1071 1088 0275.93059 10.1287/opre.21.5.1071
48. Stachniss, C., Grisetti, G., & Burgard, W. (2005). Information gain-based exploration using Rao-Blackwellized particle filters. In Proc. of robotics: science and systems, Cambridge, USA, 2005.
49. Stolle, M., & Atkeson, C. (2009). Finding and transferring policies using stored behaviors. Autonomous Robots, 27 (Special issue on Robot Learning, Part B) (this issue).
50. O. Tremois J. Le Cadre 1999 Optimal observer trajectory in bearings-only tracking for manoeuvering sources IEE Proceeding Radar, Sonar Navigation 146 1 31 39 10.1049/ip-rsn:19990262
51. Vazquez, E., & Bect, J. (2008). On the convergence of the expected improvement algorithm. arXivorg arXiv:0712.3744v2 [stat.CO], http://arxiv.org/abs/0712.3744v2.
52. Vidal-Calleja, T., Davison, A., Andrade-Cetto, J., & Murray, D. (2006). Active control for single camera SLAM. In Proc. of the IEEE int. conf. on robotics & automation (pp. 1930-1936) 2006.
53. Vlassis, N., Toussaint, G. K. M., & Piperidis, S. (2009). Learning model-free robot control using a Monte Carlo em algorithm. Autonomous Robots, 27 (Special issue on Robot Learning, Part B) (this issue).
54. R. Williams 1992 Simple statistical gradient-following algorithms for connectionist reinforcement learning Machine Learning 8 3 229 256 0772.68076
55. A. Zilinskas J. Zilinskas 2002 Global optimization based on a statistical model and simplicial partitioning Computers and Mathematics with Applications 44 957 967 1047.90036 10.1016/S0898-1221(02)00206-7 1927740 (Pubitemid 34979858)