Policy gradient methods for reinforcement learning with function approximation

Advances in Neural Information Processing Systems

Volumn , Issue , 2000, Pages 1057-1063

  Sutton, Richard S ^a   McAllester, David ^a   Singh, Satinder ^a   Mansour, Yishay ^a  

^a AT AND T LABS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

GRADIENT METHODS;

ACTOR-CRITIC METHODS; DIFFERENTIABLE FUNCTIONS; FUNCTION APPROXIMATION; FUNCTION APPROXIMATORS; OPTIMAL POLICIES; POLICY GRADIENT METHODS; POLICY ITERATION; REINFORCEMENT LEARNING WITH FUNCTION APPROXIMATIONS;

REINFORCEMENT LEARNING;

EID: 84898939480     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Baird' L. C. (1993). Advantage Updating. Wright Lab. Technical Report WL-TR-93-1146.
2. Baird' L. C. (1995). Residual algorithms: Reinforcement learning with function approximation. Proc. of the Twelfth Int. Conf. on Machine Learning' pp. 30-37. Morgan Kaufmann
3. Morgan Kaufmann. Baird' L. C' Moore' A. W. (1999). Gradient descent for general reinforcement learning. NIPS 11. MIT Press.
4. Barto' A. G.' Sutton' R. S.' Anderson' C. W. (1983). Neuronlike elements that can solve difficult learning control problems. IEEE Trans' on Systems' Man' and Cybernetics 13.835.
5. Baxter' J.' Bartlett' P. (in prep.) Direct gradient-based reinforcement learning: I. Gradient estimation algorithms.
6. Bertsekas' D. P.' Tsitsiklis' J. N. (1996). Neuro-Dynamic. Programming. Athena Scientific.
7. Cao' X.-R.' Chen' H.-F. (1997). Perturbation realization' potentials' and sensitivity analysis of Markov Processes' IEEE Trans' on Automatic Control 42(10)=1382-1393.
8. Dayan' P. (1991). Reinforcement comparison. In D. S. Touretzky' J. L. Elman' T. J. Sejnowski' and G. E. Hinton (eds.)' Connectionist Models: Proceedings of the 1990 Summer School' pp. 45-51. Morgan Kaufmann.
9. Gordon' G. J. (1995). Stable function approximation in dynamic programming. Proceedings of the Twelfth Int. Conf. on Machine Learning' pp. 261-268. Morgan Kaufmann
10. Morgan Kaufmann. Gordon' G. J. (1996). Chattering in SARSA(λ). CMU Learning Lab Technical Report.
11. Jaakkola' T.' Singh' S. P.' Jordan' M. I. (1995) Reinforcement learning algorithms for partially observable Markov decision problems' NIPS 7' pp. 345-352. Morgan Kaufman.
12. Kimura' H.' Kobayashi' S. (1998). An analysis of actor/critic algorithms using eligibility traces: Reinforcement learning with imperfect value functions. Proc. ICML-98' pp. 278-286.
13. Konda' V. R.' Tsitsiklis' J. N. (in prep.) Actor-critic algorithms
14. Mar bach' P.' Tsitsiklis' J. N. (1998) Simulation-based optimization of Markov reward processes' technical report LIDS-P-2411' Massachusetts Institute of Technology.
15. Singh' S. P.' Jaakkola' T.' Jordan' M. I. (1994). Learning without state-estimation in partially observable Markovian decision problems. Proc. I CML-94' pp. 284-292.
16. Sutton' R. S. (1984). Temporal Credit Assignment in Reinforcement Learning. Ph.D. thesis' University of Massachusetts' Amherst.
17. Sutton' R. S.' Barto' A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
18. Tsitsiklis' J. N. Van Roy' B. (1996). Feature-based methods for large scale dynamic programming. Machine Learning 22:59-94.
19. Williams' R. J. (1988). Toward a theory of reinforcement-learning connectionist systems. Technical Report NU-CCS-88-3' Northeastern University' College of Computer Science.
20. Williams' R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 5:229-256.