Linear least-squares algorithms for temporal difference learning

Machine Learning

Volumn 22, Issue 1-3, 1996, Pages 33-57

  Bradtke, Steven J ^a  

^a GTE Data Services   (United States)

Author keywords

Least squares; Markov decision problems; Reinforcement learning; Temporal difference methods

Indexed keywords

EID: 0001771345     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1007/BF00114723     Document Type: Article

References (24)

1. Anderson, C. W. (1988). Strategy learning with multilayer connectionist representations Technical Report 87-509.3. GTE Laboratories Incorporated. Computer and Intelligent Systems Laboratory, 40 Sylvan Road Waltham, MA 02254.
2. Barto, A. G., Sutton, R. S. & Anderson, C. W. (1983). Netuonlike elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13:835-846.
3. Bradtke, S. J., (1994). Incremental Dynamic Programming for On-Line Adaptive Optimal Control. PhD thesis, University of Massachusetts, Computer Science Dept. Technical Report 94-62.
4. Darken, C. Chang, I. & Moody, J., (1992) Learning rate schedules for faster stochastic gradient search. In Neural Networks for Signal Processing 2 - Proceedings of the 1992 IEEE Workshop. IEEE Press.
5. Dayan, P., (1992) The convergence of TP(λ) for general λ. Machine Learning, 8:341-362.
6. Dayan, P. & Sejnowski, T.J., (1994). TD(λ): Convergence with probability I. Machine Learning.
7. Goodwin, G.C. & Sin, K.S., (1984). Adaptive Filtering Prediction and Control. Prentice-Hall, Englewood Cliffs, N.J.
8. Jaakkola, T. Jordan, M.I & Singh, S. P., (1994). On the convergence of stochastic iterative dynamic programming algorithms Neural Computation, 6(6).
9. Kemeny, J.G. & Snell, J.L., (1976). Finite Markov Chains. Springer-Verlag, New York.
10. Ljung, L. & Söderström T., (1983). Theory and Practice of Recursive Identification MIT Press, Cambridge. MA.
11. Lukes, G., Thompson, B. & Werbos, P., (1990) Expectation driven learning with an associative memory. In Proceedings of the International Joint Conference on Neural Networks, pages 1:521-524.
12. Robbins, H & Monro, S., (1951). A stochastic approximation method. Annals of Mathematical Statistics, 22:400-407.
13. Söderström, T. & Stoica, P.G., (1983). Instrumental Variable Methods for System Identification. Springer Verlag. Berlin.
14. Sutton A.S., (1984). Temporal Credit Assignment in Reinforcement Learning. PhD thesis, Department of Computer and Information Science, University of Massachusetts at Amherst, Amherst, MA 01003.
15. Sutton, R.S., (1988) Learning to predict by the method of temporal differences. Machine Learning. 3:9-44.
16. Tesauro, G.J., (1992). Practical issues in temporal difference learning Machine Learning 8(3/4):257-277.
17. Tsitsiklis, J. N. (1993). Asynchronous stochastic approximation and Q-learning. Technical Report LIDS-P-2172, Laboratory for Information and Decision Systems. MIT. Cambridge, MA.
18. Watkins, C. J. C. H. (1989) Learning from Delayed Rewards. PhD thesis, Cambridge University Cambridge, England.
19. Watkins, C. J. C. H. & Dayan, P., (1992). Q-learning. Machine Learning, 8(3/4):257-277, May 1992.
20. Werbos, P.J., (1987). Building and understanding adaptive systems: A statistical/numerical approach to factory automation and brain research. IEEE Transactions on Systems, Man. and Cybernetics, 17(1):7-20.
21. Werbos, P.J. (1988). Generalization of backpropagation with application to a recurrent gas market model. Neural Networks, 1(4):339-356, 1988.
22. Werbos, P.J. (1990) Consistency of HDP applied to a simple reinforcement learning problem. Neural Networks, 3(2):179-190.
23. Werbos, P.J. (1992) Approximate dynamic programming for real time control and neural modeling. In D. A. White and D. A. Sofge, editors, Handbook of Intelligent Control Neural, Fuzzy, and Adaptive Approaches. pages 493-525. Van Nostrand Reinhold. New York.
24. Young, P., (1984). Recursive Estimation and Time-series Analysis. Springer-Verlag.