Experiments with reinforcement learning in problems with continuous state and action spaces

Adaptive Behavior

Volumn 6, Issue 2, 1997, Pages 163-217

  Santamaría, Juan Carlos ^a   Ram, Ashwin ^a   Sutton, Richard S ^b  

^a Georgia Institute of Technology   (United States)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

Continuous domains; Function approximation; Memory based methods; Optimal control; Reinforcement learning; Resource preallocation

Indexed keywords

EID: 0031231885     PISSN: 10597123     EISSN: None     Source Type: Journal    
DOI: 10.1177/105971239700600201     Document Type: Article

References (26)

1. Albus, J. S. (1975). A new approach to manipulator control: The cerebellar model articulation controller (cmac). Journal of Dynamic Systems, Measurement, and Control, 97(3), 220-227.
2. Atkeson, C. G. (1991). Memory-based learning control. In Proceedings of the 1991 American Control Conference (Vol. 3). New York: American Automatic Control Council.
3. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983). Neuronlike elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13, 835-846.
4. Bellman, R. (1957). Dynamic programming. Princeton, NJ: Princeton University Press.
5. Bertsekas, D. P. (1995). Dynamic programming and optimal control (Vol. 1). Belmont, MA: Athena Scientific.
6. Cishosz, P. (1996). Truncating temporal differences: on the efficient implementation of TD(λ) for reinforcement learning. Journal of Artificial Intelligence Research, Athena Scientific, 287-318.
7. Kanerva, P. (1993). Sparse distributed memory and related models. In M. H. Hassoun (ed.), Associative neural memories: Theory and implementation. New York: Oxford University Press.
8. Kibler, D., & Aha, D. W. (1989). Instance-based prediction of real-valued attributes. Computational Intelligence, 5(2), 51-57.
9. Lin, L. J. (1992). Self-improving reactive agents based on reinforcement learning. Machine Learning, 8(3-4), 293-321.
10. Mahadevan, S., & Connell, J. (1991). Scaling reinforcement learning to robotics by exploiting the subsumption architecture. In Proceedings of the Eighth International Workshop on Machine Learning (Vol. 1). San Mateo: Morgan Kaufmann.
11. McCallum, R. A., Tesauro, G., Touretzky, D., & Leen, T. (1995). Instance-based state identification for reinforcement learning. Advances in Neural Information Processing Systems, 7, 377-384.
12. Moore, A. W., & Atkeson, C. G. (1995). The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. Machine Learning, 21(3), 199-233.
13. Narendra, K. S., & Annaswamy, A. M. (1989). Stable adaptive systems. Englewood Cliffs, NJ: Prentice-Hall.
14. Peng, J. (1993). Efficient dynamic prograinming-based learning for control. Unpublished doctoral thesis, Department of Computer Science, Northeastern University.
15. Peng, J., & Williams, R. J. (1994). Incremental multi-step Q-learning. In W. W. Cohen & H. Hirsh (Eds.), Machine Learning: Proceedings of the Eleventh International Conference. San Mateo: Morgan Kaufmann.
16. Ram, A., & Santamaría, J. C. (1997). Continuous case-based reasoning. Artificial Intelligence, 90(1-2), 25-77.
17. Richards, R. J. (1979). An introduction to dynamics and control. New York: Longman.
18. Rummery, G. A., & Niranjan, M. (1994). On-line q-learning using connectionist systems (Tech. Rep. No. CUED/F-INFEG/TR66). Cambridge, Engl.: Cambridge University Department.
19. Shanmugam, K. S. (1979). Digital and analog communication systems. New York: Longman.
20. Singh, S. P., & Sutton, R. S. (1996). Reinforcement learning with replacing eligibility traces. Machine Learning, 22, 123-158.
21. Stengel, R. F. (1994). Optimal control and estimation. Mineola, NY: Dover Publications.
22. Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3, 9-44.
23. Sutton, R. S. (1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. Advances in Neural Information Processing Systems, 8, 1038-1044.
24. Tham, C. L. (1995). Reinforcement learning of multiple tasks using a hierarchical cmac architecture. Robotics and Autonomous Systems, 15(4), 247-274.
25. Tsitsiklis, J. N., & Van Roy, B. (1997). An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control, H2(5), 674-690.
26. Watkins, C. J. C. H. (1989). Learning from delayed rewards. Unpublished doctoral thesis, University of Cambridge, Cambridge, England.