Feature-based methods for large scale dynamic programming

Machine Learning

Volumn 22, Issue 1-3, 1996, Pages 59-94

  Tsitsiklis, John N ^a   Van Roy, Benjamin ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Compact representation; Curse of dimensionality; Dynamic programming; Features; Function approximation; Neuro dynamic programming; Reinforcement learning

Indexed keywords

APPROXIMATION THEORY; CONVERGENCE OF NUMERICAL METHODS; DYNAMIC PROGRAMMING; ERRORS; FEATURE EXTRACTION; LARGE SCALE SYSTEMS; LEARNING ALGORITHMS; PROBLEM SOLVING; STOCHASTIC CONTROL SYSTEMS;

COMPACT REPRESENTATION; CURSE OF DIMENSIONALITY; FEATURE BASED METHODS; FUNCTION APPROXIMATION; REINFORCEMENT LEARNING;

LEARNING SYSTEMS;

EID: 0029752470     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1007/BF00114724     Document Type: Article

References (22)

1. Bakshi, B. R. & Stephanopoulos G., (1993). "Wave-Net: A Multiresolution, Hierarchical Neural Network with Localized Learning," AIChE Journal, vol. 39, no 1, pp. 57-81.
2. Barto, A. G., Bradtke, S. J., & Singh, S. P., (1995). "Real time Learning and Control Using Asynchronous Dynamic Programming," Artificial Intelligence, vol. 72, pp. 81-138.
3. Bellman, R. E. & Dreyfus, S. E., (1959) "Functional Approximation and Dynamic Programming," Math. Tables and Other Aids Comp., Vol. 13, pp. 247-251.
4. Bertsekas, D. P., (1995). Dynamic Programming and Optimal Control, Athena Scientific, Bellmont, MA.
5. Bertsekas, D. P. (1994) "A Counter-Example to Temporal Differences Learning," Neural Computation, vol 7, pp. 270-279.
6. Bertsekas D. P. & Castañon, D. A., (1989). 'Adaptive Aggregation for Infinite Horizon Dynamic Programming," IEEE Transactions on Automatic Control, Vol. 34, No. 6, pp. 589-598.
7. Bertsekas, D. P. & Tsitsiklis, J. N., (1989). Parallel and Distributed Computation: Numerical Methods. Prentice Hall, Eaglewood Cliffs, NJ.
8. Dayan, P. D., (1992) "The Convergence of TD(λ) for General λ," Machine Learning, vol. 8, pp. 341-362.
9. Gordon, G. J., (1995). "Stable Function Approximation in Dynamic Programming," Technical Report CMU-CS-9-103, Carnegie Mellon University.
10. Jaakola, T., Jordan M. I., &. Singh, S. P., (1994). "On the Convergence of Stochastic Iterative Dynamic Programming Algorithms," Neural Computation, Vol. 6. No. 6.
11. Jaakola T., Singh S. P., & Jordan, M. I., (1995). "Reinforcement Algorithms for Partially Observable Markovian Decision Processes," in Advances in Neural information Processing Systems, 7, J. D. Cowan G. Tesauro, and D. Touretzky, editors, Morgan Kaufmann.
12. Korf, R. E. (1987). "Planning as Search. A Quantitative Approach," Artificial Intelligence, vol 33, pp. 65-88.
13. Lippman, R. E. Kukolich, L. & Singer, E., (1993). LNKner: Neural Network, Machine Learning, and Statistical Software for Pattern Classification," The Lincoln Laboratory Journal, vol 6. no 2, pp 249-268
14. Morin, T. J., (1987) "Computational Advances in Dynamic Programming," in Dynamic Programming and Its Applications, edited by Puterman, M.L., pp. 53-90.
15. Poggio, T. & Girosi, F., (1990) "Networks for Approximation and learning," Proceedings of the IEEE, vol. 78, no. 9, pp. 1481-1497.
16. Reetz, D., (1977) "Approximate Solutions of a Discounted Markovian Decision Process." Bonner Mathematische Schriften vol. 98: Dynamische Optimierung, pp. 77-92.
17. Schweitzer, P. J., & Seidmann. A., (1985). 'Generalized Polynomial Approximation.', in Markovian Decision Processes.'' Journal of Mathematical Analysis and Applications vol. 110, pp 568-582.
18. Sutton, R. S., (1988) "Learning to Predict by the Method of Temporal Differences,"Machine Learning, vol. 3, pp. 9-44.
19. Tesauro, G., (1992) "Practical Issues in Temporal Difference Learning," Machine Learning, vol. 8, pp 257-277.
20. Tsitsiklis, J. N., (1994) "Asynchronous Stochastic Approximation and Q-Learning," Machine Learning, vol. 16, pp 185-202.
21. Watkins, C. J. C. H., Dayan, P., (1992). "Q-learning," Machine Learning, vol. 8, pp. 279-292.
22. Whitt, W., (1978). Approximations of Dynamic Programs I Mathematics of Operations Research, vol. 3, pp. 231-243