Convergence analysis on temporal difference learning

International Journal of Innovative Computing, Information and Control

Volumn 5, Issue 4, 2009, Pages 913-922

  Leng, Jinsong ^a   Jain, Lakhmi ^a   Fyfe, Colin ^b  

^a UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

^b UNIVERSITY OF THE WEST OF SCOTLAND   (United Kingdom)

Author keywords

Agent; Convergence analysis; Temporal difference learning

Indexed keywords

EID: 64349090183     PISSN: 13494198     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (27)

1. R. Bellman, A markovian decision process, Journal of Mathematics and Mechanics, vol.6, no.3, pp.679-693, 1957.
2. M. Wooldridge and N. Jennings, Intelligent agents: Theory and practice, Knowledge Engineering Review, vol.10, no.2, pp.115-152, 1995.
3. R. Bellman, Dynamic Programming, Princeton University Press, Princeton, NJ, 1957.
4. S. Russell and P. Norvig, Heuristic: Intelligent Search Strategies for Computer Problem Solving, A Modern Approach, Prentice-Hall, Englewood, NJ, 1995.
5. R. S. Sutton, Learning to predict by the method of temporal differences, Machine Learning, vol.3, no.9, pp.9-44, 1988.
6. A. A. Sherstov and P. Stone, Function approximation via tile coding: Automating parameter choice, Proc. of the SARA 2005, Berlin, LNCS, no.3607, pp.194-205, 2005.
7. A. Potapov and M. K. Ali, Convergence of reinforcement learning algorithms and acceleration of learning, Physical Review E, vol.67, no.2, 2003.
8. Teambots, http://www.es.emu.edu/^trb/Teambots/Domains/SoccerBots, 2000.
9. J. S. Albus, A theory of cerebellar function, Mathematical Biosciences, vol.10, pp.25-61, 1971.
10. C. J. C. H. Watkins, C J. C H. Watkins, Ph.D. Thesis, Cambridge University, Cambridge, England, 1989.
11. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998.
12. P. Dayan and T. J. Sejnowski, TD(λ) converges with probability 1, Machine Learning, vol.14, no.l, pp.295-301, 1994.
13. J. N. Tsitsiklis, Asynchronous stochastic approximation and Q-learning, Machine Learning, vol.16, no.l, pp.185-202, 1994.
14. L. Gurvits, L. J. Lin and S. J. Hanson, Incremental Learning of Evaluation Functions for Absorbing Markov Chains: New Methods and Theorems, Preprint, 1994.
15. J. N. Tsitsiklis and B. Van Roy, An analysis of temporal-difference learning with function approximation, IEEE Transactions on Automatic Control, vol.42, no.5, pp.674-690, 1997.
16. J. Leng, C. Fyfe and L. Jain, Reinforcement learning of competitive skills with soccer agents, Proc. of the 11th International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, LNAI 4692, pp.572-579, 2007.
17. R. S. Sutton and S. P. Singh, On step-size and bias in temporal-difference learning, Proc. of the Eighth Yale Workshop on adaptive and Learning Systems, New Haven, CT, pp.91-96, 1994.
18. A. Gosavi, Simulation-based Optimization: Parametric Optimization Techniques and Reinforcement Learning, Kluwer Academic Publishers, 2003.
19. X. Cai, Z. Cui, J. Zeng and Y. Tan, Performance-dependent adaptive particle swam optimization, International Journal of Innovative Computing, Information and Control, vol.3, no.6(B), pp.1697-1706, 2007.
20. S.-C. Chu and P.-W. Tsai, Computational intelligence based on the behavior of cats, International Journal of Innovative Computing, Information and Control, vol.3, no.l, pp.163-173, 2007.
21. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, M. N. Teller and E. Teller, Equations of state calculations by fast computing machines, Journal of Chemical Physics, vol.21, pp.1087-1091, 1953.
22. L. Ingber, Very fast simulated re-annealing, Mathematical Computer Modelling, vol.12, no.8, pp.967-973, 1989.
23. A. F. Atiya, A. G. Parlos and L. Ingber, A reinforcement learning method based on adaptive simulated annealing, Proc. of the 46th IEEE International Midwest Symposium on, vol.1, pp.121-124, 2003.
24. P. Stefan, Combined Use of Reinforcement Learning and Simulated Annealing: Algorithms and Applications, Ph.D. Thesis, University of Miskolc, Hungary, 2003.
25. G. E. P. Box and K. B. Wilson, On the experimental attainment of optimum conditions (with discussion), Journal of the Royal Statistical Society Series B, vol.13, no.l, pp.1-45, 1951.
26. G. A. F. Seber, Linear Regression Analysis, John Wiley & Sons: New York, 1977.
27. The MathWorks. http://www.mathworks.com.