On the existence of fixed points for approximate value iteration and temporal-difference learning

Journal of Optimization Theory and Applications

Volumn 105, Issue 3, 2000, Pages 589-608

  De Farias, D P ^a   Van Roy, B ^a  

^a Stanford University   (United States)

Author keywords

Dynamic programming; Neurodynamic programming; Reinforcement learning; Temporal difference learning; Value iteration

Indexed keywords

DYNAMIC PROGRAMMING; ITERATIVE METHODS; ORDINARY DIFFERENTIAL EQUATIONS;

CURSE OF DIMENSIONALITY; DYNAMIC PROGRAMS; FIXED POINTS; NEURO-DYNAMIC PROGRAMMING; REINFORCEMENT LEARNINGS; SIMPLE ALGORITHM; SIMPLER ALGORITHMS; TEMPORAL DIFFERENCE LEARNING; VALUE ITERATION; VALUE ITERATION ALGORITHM;

REINFORCEMENT LEARNING;

EID: 0034342516     PISSN: 00223239     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1004641123405     Document Type: Article

References (11)

1. BELLMAN, R., and DREYFUS, S., Functional Approximations and Dynamic Programming, Mathematical Tables and Other Aids to Computation, Vol. 13, pp. 247-251, 1959.
2. SUTTON, R. S., Learning to Predict by the Method of Temporal Differences, Machine Learning, Vol. 3, pp. 9-44, 1988.
3. GURVITS, L., LIN, L. J., and HANSON, S. J., Incremental Learning of Evaluation Functions for Absorbing Markov Chains: New Methods and Theorems, Preprint, 1994.
4. PINEDA, F., Mean-Field Analysis for Batched TD(λ), Neural Computation, Vol. 9, pp. 1403-1419, 1997.
5. TSITSIKLIS, J. N., and VAN ROY, B., An Analysis of Temporal-Difference Learning with Function Approximation. IEEE Transactions on Automatic Control, Vol. 42, pp. 674-690, 1997.
6. DAYAN, P. D., The Convergence of TD(λ) for General λ, Machine Learning, Vol. 8, pp. 341-362, 1992.
7. BERTSEKAS, D. P., and TSITSIKLIS, J. N., Neurodynamic Programming, Athena Scientific, Belmont, Massachusetts, 1995.
8. VAN ROY, B., Learning and Value Function Approximation in Complex Decision Processes, PhD Dissertation, MIT, 1998.
9. GALLAGER, R. G., Discrete Stochastic Processes, Kluwer Academic Publishers, Boston, Massachusetts, 1996.
10. VAN ROY, B., BERTSEKAS, D. P., LEE, Y., and TSITSIKLIS, J. N., A Neurodynamic Programming Approach to Retailer Inventory Management, Proceedings of the IEEE Conference on Decision and Control, pp. 4052-4057, 1997.
11. BENVENISTE, A., MÉTIVIER, M., and PRIOURET, P., Adaptive Algorithms and Stochastic Approximation, Springer Verlag, Berlin, Germany, 1990.