Towards direct policy search reinforcement learning for robot control

IEEE International Conference on Intelligent Robots and Systems

Volumn , Issue , 2006, Pages 3178-3183

  El Fakdi, Andres ^a   Carreras, Marc ^a   Ridao, Pere ^a  

^a UNIVERSITY OF GIRONA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CONTROL SYSTEM ANALYSIS; INTELLIGENT CONTROL; OPTIMIZATION; ROBOTS; STATE ESTIMATION;

ROBOT CONTROL; UNDERWATER ROBOTS;

REINFORCEMENT LEARNING;

EID: 34250644253     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IROS.2006.282342     Document Type: Conference Paper

References (30)

1. R. Sutton and A. Barto, Reinforcement Learning, an introduction. MIT Press, 1998.
2. W. D. Smart and L. P. Kaelbling, "Practical reinforcement learning in continuous spaces," in International Conference on Machine Learning, 2000.
3. N. Hernandez and S. Mahadevan, "Hierarchical memory-based reinforcement learning," in Fifteenth International Conference on Neural Information Processing Systems, Denver, USA, 2000.
4. M. Carreras, P. Ridao, R. Garcia, and T. Nicosevici, "Vision-based localization of an underwater robot in a structured environment," in IEEE International Conference on Robotics and Automation, Taipei, Taiwan, 2003.
5. D. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
6. R. Sutton, D. McAllester, S. Singh, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation," Advances in Neural Information Processing Systems, vol. 12, pp. 1057-1063, 2000.
7. C. Anderson, "Approximating a policy can be easier than approximating a value function," University of Colorado State," Computer Science Technical Report, 2000.
8. D. A. Aberdeen, "Policy-gradient algorithms for partially observable markov decision processes," Ph.D. dissertation, Australian National University, April 2003.
9. E. A. Hansen, "Solving POMDPs by searching in policy space," in 8th Conference on Uncertainty in Artificial Intelligence, Madison, WI, 1998, pp. 211-219.
10. N. Meuleau, K. E. Kim, L. P. Kaelbling, and A. R. Cassandra, "Solving POMDPs by searching the space of finite policies," in 15th Conference on Uncertainty in Artificial Intelligence, M. Kaufmann, Ed., Computer science Dep., Brown University, July 1999, pp. 127-136.
11. S. Singh, T. Jaakkola, and M. Jordan, "Learning without state-estimation in partially observable markovian decision processes," in Proceedings of the Eleventh International Conference on Machine Learning, New Jersey, USA, 1994.
12. J. Baxter and P. Bartlett, "Direct gradient-based reinforcement learning," in International Symposium on Circuits and Systems, Geneva, Switzerland, May 2000.
13. C. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, pp. 279-292, 1992.
14. P. Marbach and J. N. Tsitsiklis, "Gradient-based optimization of Markov reward processes: Practical variants," Center for Communications Systems Research, University of Cambridge, Tech. Rep., March 2000.
15. V. Konda and J. Tsitsiklis, "On actor-critic algorithms," SIAM Journal on Control and Optimization, vol. 42, number 4, pp. 1143-1166, 2003.
16. N. Meuleau, L. Peshkin, and K. Kim, "Exploration in gradient based reinforcement learning," Massachusetts Institute of Technology, AI Memo 2001-003, Tech. Rep., April 2001.
17. R. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning," Machine Learning, vol. 8, pp. 229-256, 1992.
18. H. Kimura, K. Miyazaki, and S. Kobayashi, "Reinforcement learning in pomdps with function approximation," in Fourteenth International Conference on Machine Learning (ICML'97), D. H. Fisher, Ed., 1997, pp. 152-160.
19. T. Jaakkola, S. Singh, and M. Jordan, Reinforcement Learning algorithms for partially observable Markov decision problems. Morgan Kaufman, 1995, vol. 7, pp. 345-352.
20. J. Baxter and P. Bartlett, "Direct gradient-based reinforcement learning: I. gradient estimation algorithms," Australian National University, Tech. Rep., 1999.
21. P. Marbach and J. N. Tsitsiklis, "Simulation-based optimization of markov reward processes," Technical report LIDS-P-2411, Massachussets Institute of Technology, 1998.
22. P. Marbach, "Simulation-based methods for markov decision processes," PhD Thesis, Laboratory for Information and Decision Systems, MIT, 1998.
23. J. Bagnell and J. Schneider, "Autonomous helicopter control using reinforcement learning policy search methods," in Proceedings of the IEEE International Conference on Robotics and Automation, Korea, 2001.
24. M. Rosenstein and A. Barto, "Robot weightlifting by direct policy search," in Proceedings of the International Joint Conference on Artificial Intelligence, 2001.
25. N. Kohl and P. Stone, "Policy gradient reinforcement learning for fast quadrupedal locomotion," in IEEE International Conference on Robotics and Automation (ICRA), 2004.
26. R. Tedrake, T. W. Zhang, and H. S. Seung, "Stochastic policy gradient reinforcement learning on a simple 3D biped," in IEEE/RSJ International Conference on Intelligent Robots and Systems IROS'04, Sendai, Japan, September 28 - October 2 2004.
27. T. Matsubara, J. Morimoto, J. Nakanishi, M. Sato, and K. Doya, "Learning sensory feedback to CPG with policy gradient for biped locomotion," in Proceedings of the International Conference on Robotics and Automation ICRA, Barcelona, Spain, April 2005.
28. S. Haykin, Neural Networks, a comprehensive foundation, 2nd ed. Prentice Hall, 1999.
29. A. Savitzky and M. Golay, Analytical Chemistry, 1964, vol. 36, pp. 1627-1639.
30. P. Ridao, M. Carreras, and J. Batlle, "Model identification of a low-speed UUV," in Control Applications in Marine Systems CAMS'01, Scotland (UK), 2001.