Basis Function Adaptation Methods for Cost Approximation in MDP

2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2009 - Proceedings

Volumn , Issue , 2009, Pages 74-81

  Yu, Huizhen ^a   Bertsekas, Dimitri P ^b  

^a UNIVERSITY OF HELSINKI   (Finland)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTATION FRAMEWORK; ADAPTATION METHODS; ADAPTATION SCHEME; BASIS FUNCTIONS; FUNCTION APPROXIMATION; LOW ORDER; MARKOV DECISION PROCESS; NONLINEAR OPTIMAL; OBJECTIVE FUNCTIONS; POLICY GRADIENT METHODS; TD METHOD; TEMPORAL DIFFERENCES;

COSTS; DYNAMIC PROGRAMMING; GRADIENT METHODS; LEARNING ALGORITHMS; MARKOV PROCESSES; REINFORCEMENT LEARNING; SYSTEMS ENGINEERING;

REINFORCEMENT;

EID: 67650458822     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2009.4927528     Document Type: Conference Paper

References (17)

1. I. Menache, S. Mannor, and N. Shimkin, "Basis function adaptation in temporal difference reinforcement learning," Ann. Oper. Res., Vol. 134, no. 1, pp. 215-238, 2005. (Pubitemid 40550047)
2. R. S. Sutton, "Learning to predict by the methods of temporal differences," Machine Learning, Vol. 3, pp. 9-44, 1988.
3. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
4. R. S. Sutton and A. G. Barto, Reinforcement Learning. Cambridge, MA: MIT Press, 1998.
5. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Belmont, MA: Athena Scientific, 2007, Vol. II.
6. V. R. Konda and J. N. Tsitsiklis, "Actor-critic algorithms," SIAM J. Control Optim., Vol. 42, no. 4, pp. 1143-1166, 2003.
7. J. A. Boyan, "Least-squares temporal difference learning," in Proc. The 16th Int. Conf. Machine Learning, 1999.
8. D. P. Bertsekas, V. S. Borkar, and A. Nedić, "Improved temporal difference methods with linear function approximation," in Learning and Approximate Dynamic Programming. IEEE Press, 2004.
9. V. S. Borkar, "Stochastic approximation with 'controlled Markov' noise," Systems Control Lett., Vol. 55, pp. 139-145, 2006.
10. V. S. Borkar, Stochastic Approximation: A Dynamic Viewpoint. New Delhi: Hindustan Book Agency, 2008.
11. D. P. Bertsekas and H. Yu, "Projected equation methods for approximate solution of large linear systems," J. Comput. Sci. Appl. Math., 2008, to appear.
12. J. N. Tsitsiklis and B. Van Roy, "Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing financial derivatives," IEEE Trans. Automat. Contr., Vol. 44, pp. 1840-1851, 1999. (Pubitemid 30546876)
13. D. S. Choi and B. Van Roy, "A generalized Kalman filter for fixed point approximation and efficient temporal-difference learning," Discrete Event Dyn. Syst., Vol. 16, no. 2, pp. 207-239, 2006.
14. H. Yu and D. P. Bertsekas, "A least squares Q-learning algorithm for optimal stopping problems," MIT, LIDS Tech. Report 2731, 2006.
15. R. T. Rockafellar and R. J.-B. Wets, Variational Analysis. Berlin: Springer-Verlag, 1998.
16. S. M. Robinson, "An implicit-function theorem for a class of nonsmooth functions," Math. Oper. Res., Vol. 16, no. 2, pp. 292-309, 1991.
17. A. L. Dontchev and R. T. Rockafellar, "Robinson's implicit function theorem and its extensions," Math. Program. Ser. B, Vol. 117, no. 1, pp. 129-147, 2008.