Least squares policy evaluation algorithms with linear function approximation

Discrete Event Dynamic Systems: Theory and Applications

Volumn 13, Issue 1-2, 2003, Pages 79-110

  Nedic A ^a   Bertsekas, D P ^a  

^a USA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; DYNAMIC PROGRAMMING; FUNCTION EVALUATION; GRADIENT METHODS; LEAST SQUARES APPROXIMATIONS; LINEAR PROGRAMMING; PROBABILITY DISTRIBUTIONS; PROBLEM SOLVING;

LEAST SQUARES SUBPROBLEMS; LINEAR FUNCTION APPROXIMATION; POLICY EVALUATION ALGORITHMS;

ALGORITHMS;

EID: 0037288398     PISSN: 09246703     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022192903948     Document Type: Article

References (21)

1. Ash, R. B. 1972. Real Analysis and Probability. New York: Academic Press Inc.
2. Bertsekas, D. P. 1995. A counterexample to temporal differences learning. Neural Computation 7: 270-279.
3. Bertsekas, D. P., and Ioffe, S. 1996. Temporal differences-based policy iteration and application in neuro-dynamic programming. Lab. for Info. and Decision Systems Report LIDS-P-2349. Cambridge, MA: MIT.
4. Bertsekas, D. P. 1999. Nonlinear Programming, 2nd edition. Belmont, MA: Athena Scientific.
5. Bertsekas, D. P. 2001. Dynamic Programming and Optimal Control, 2nd edition. Belmont, MA: Athena Scientific.
6. Bertsekas, D. P., and Tsitsiklis, J. N. 1996. Neuro-Dynamic Programming. Belmont, MA: Athena Scientific.
7. Bertsekas, D. P., and Tsitsiklis, J. N. 2000. Gradient convergence in gradient methods with errors. SIAM J. Optim. 10: 627-642.
8. Boyan, J. A. 2002. Technical update: least-squares temporal difference learning. To appear in Machine Learning, 49.
9. Bradtke, S. J., and Barto, A. G. 1996. Linear least-squares algorithms for temporal difference learning. Machine Learning 22: 33-57.
10. Dayan, P., and Sejnowski, T. J. 1994. TD(λ) converges with probability 1. Machine Learning 14: 295-301.
11. Gallager, R. G. 1995. Discrete Stochastic Processes. Boston, MA: Kluwer Academic Publishers.
12. Gurvits, L., Lin, L., and Hanson, S. J. 1994. Incremental Learning of Evaluation Functions for Absorbing Markov Chains: New Methods and Theorems. Working paper, Princeton, NJ: Siemens Corporate Research.
13. Golub, G. H., and Van Loan, C. F. 1996. Matrix Computations, 3rd edition. Baltimore, MD: Johns Hopkins University Press.
14. Jaakkola, T., Jordan, M. I., and Singh S. P. 1994. On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation 6: 1185-1201.
15. Kemeny, J. G., and Snell, J. L. 1967. Finite Markov Chains. New York: Van Nostrand Company.
16. Neveu, J. 1975. Discrete Parameter Martingales. Amsterdam: North-Holland.
17. Parzen, E. 1962. Modern Probability Theory and Its Applications. New York: John Wiley Inc.
18. Puterman, M. L. 1994. Markovian Decision Problems. New York: John Wiley Inc.
19. Sutton, R. S. 1988. Learning to predict by the methods of temporal differences. Machine Learning 3: 9-44.
20. Tadić, V. 2001. On the convergence of temporal-difference learning with linear function approximation. Machine Learning 42: 241-267.
21. Tsitsiklis, J. N., and Van Roy, B. 1997. An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control 42: 674-690.