Parameter-exploring policy gradients

Neural Networks

Volumn 23, Issue 4, 2010, Pages 551-559

  Sehnke, Frank ^a   Osendorfer, Christian ^a   Ruckstiess Thomas T ^a   Graves, Alex ^a   Peters, Jan ^c   Schmidhuber, Jürgen ^a,b  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

^c MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

Control; Policy gradients; Reinforcement learning; Robotics; Stochastic optimisation

Indexed keywords

COMPLEX CONTROL; CONTROL POLICY; GRADIENT ESTIMATES; HUMANOID ROBOT; INDIVIDUAL COMPONENTS; MODEL FREE; OTHER ALGORITHMS; PARAMETER SPACES; PARTIALLY OBSERVABLE MARKOV DECISION PROBLEMS; POLICY GRADIENT; POLICY GRADIENT METHODS; POLICY GRADIENTS; REINFORCEMENT LEARNING METHOD; STOCHASTIC OPTIMISATION;

ANTHROPOMORPHIC ROBOTS; GRADIENT METHODS; REINFORCEMENT; ROBOTICS; STOCHASTIC SYSTEMS;

REINFORCEMENT LEARNING;

ALGORITHM; ARTICLE; MATHEMATICAL ANALYSIS; POLICY; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; ROBOTICS; SAMPLING; SIMULATION; STANDARD; STATISTICAL PARAMETERS; TASK PERFORMANCE;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; MARKOV CHAINS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); REINFORCEMENT (PSYCHOLOGY); ROBOTICS;

EID: 77950297907     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.12.004     Document Type: Article

References (23)

1. Aberdeen, D. (2003). Policy-gradient algorithms for partially observable markov decision processes. Ph.D. thesis, Australian National University.
2. Baxter J., Bartlett P.L. Reinforcement learning in POMDPs via direct gradient ascent. Proc. 17th international conf. on machine learning 2000, 41-48. Morgan Kaufmann, San Francisco, CA.
3. Benbrahim H., Franklin J. Biped dynamic walking using reinforcement learning. Robotics and Autonomous Systems 1997, 22:283-302.
4. Buss, M., & Hirche, S. (2008). Institute of Automatic Control Engineering, TU München, Germany. http://www.lsr.ei.tum.de/.
5. Hansen N., Ostermeier A. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 2001, 9(2):159-195.
6. Jordan, M. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. In Proc. of the eighth annual conference of the cognitive science society (pp. 531-546) Vol. 8.
7. Müller, H., Lauer, M., Hafner, R., Lange, S., Merke, A., & Riedmiller, M. (2007). Making a robot learn to play soccer. In Proceedings of the 30th annual German conference on artificial intelligence (KI-2007).
8. Munos R., Littman M. Policy gradient in continuous time. Journal of Machine Learning Research 2006, 7:771-791.
9. Peters, J., & Schaal, S. (2006). Policy gradient methods for robotics. In IROS-2006(pp. 2219-2225).
10. Peters J., Schaal S. Natural actor-critic. Neurocomputing 2008, 71:1180-1190.
11. Peters J., Schaal S. Reinforcement learning of motor skills with policy gradients. Neural Networks 2008, 682-697.
12. Peters, J., Vijayakumar, S., & Schaal, S. (2005). Natural actor-critic. In ECML-2005(pp. 280-291).
13. Riedmiller, M., Peters, J., & Schaal, S. (2007). Evaluation of policy gradient methods and variants on the cart-pole benchmark. In ADPRL-2007.
14. Rückstieß T., Felder M., Schmidhuber J. State-dependent exploration for policy gradient methods. LNAI 2008, Vol. 5212:234-249. W.D. (Ed.).
15. Schraudolph N., Yu J., Aberdeen D. Fast online policy gradient learning with smd gain vector adaptation. Advances in neural information processing systems 2006, Vol. 18. MIT Press, Cambridge, MA. Y. Weiss, B. Schölkopf, J. Platt (Eds.).
16. Schwefel H. Evolution and optimum seeking 1995, Wiley, New York.
17. Sehnke, F. (2009). PGPE-Policy Gradients with Parameter-based exploration-demonstration video: Learning in robot simulatons. http://www.pybrain.org/videos/jnn10/.
18. Spall J. An overview of the simultaneous perturbation method for efficient optimization. Johns Hopkins APL Technical Digest 1998, 19(4):482-492.
19. Spall J. Implementation of the simultaneous perturbation algorithm for stochastic optimization. IEEE Transactions on Aerospace and Electronic Systems 1998, 34(3):817-823.
20. Sutton, R., McAllester, D., Singh, S., & Mansour, Y. (2000). Policy gradient methods for reinforcement learning with function approximation. In NIPS-1999(pp. 1057-1063).
21. Ulbrich, H. (2008). Institute of Applied Mechanics, TU München, Germany. http://www.amm.mw.tum.de/.
22. Wierstra, D., Schaul, T., Peters, J., & Schmidhuber, J. (2008). Fitness expectation maximization. In Proceedings of the 10th international conference on parallel problem solving from nature.
23. Williams R. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 1992, 8:229-256.