Supervised actor-critic reinforcement learning

Handbook of Learning and Approximate Dynamic Programming

Volumn , Issue , 2004, Pages 359-380

  Rosenstein, Michael T ^a   Barto, Andrew G ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

Data structures; Learning; Optimization; Robots; Supervised learning; Training

Indexed keywords

DATA STRUCTURES; OPTIMIZATION; PERSONNEL TRAINING; ROBOTS; STOCHASTIC SYSTEMS; SUPERVISED LEARNING; SUPERVISORY PERSONNEL;

ACTOR-CRITIC REINFORCEMENT LEARNING; ALGORITHM DESCRIPTION; FEEDBACK CONTROLLER; INTERMITTENT CONTROLS; LEARNING; LEARNING PROBLEM; STOCHASTIC SEARCH; STRUCTURE PARTS;

REINFORCEMENT LEARNING;

EID: 84979715630     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1109/9780470544785.ch14     Document Type: Chapter

References (33)

1. J. Albus, Brains, Behavior, and Robotics, Byte Books, Peterborough, NH, 1981.
2. A. G. Barto, Reinforcement learning in motor control, in M. A. Arbib, (ed.), The Handbook of Brain Theory and Neural Networks, Second Edition, pp. 968-972, MIT Press, Cambridge, MA, 2003.
3. A. G. Barto, R. S. Sutton, and C. W. Anderson, Neuronlike elements that can solve difficult learning control problems, IEEE Trans. Systems, Man, and Cybernetics, vol. 13, pp. 835-846,1983.
4. H. Benbrahim and J. A. Franklin, Biped dynamic walking using reinforcement learning, Robotics and Autonomous Systems, vol. 22, pp. 283-302,1997.
5. N. A. Bernstein, The Co-ordination and Regulation of Movements, Pergamon Press, Oxford, 1967.
6. A. E. Bryson and Y.-C. Ho, Applied Optimal Control, Hemisphere Publishing Corp., New York, 1975.
7. J. A. Clouse, On Integrating Apprentice Learning and Reinforcement Learning, Ph.D. Thesis, University of Massachusetts, Amherst, 1996.
8. J. A. Clouse and P. E. Utgoff, A teaching method for reinforcement learning, Proc. Ninth International Conference on Machine Learning, pp. 92-101, Morgan Kaufmann, San Francisco, CA, 1992.
9. J. J. Craig, Introduction To Robotics: Mechanics and Control, Addison-Wesley, Reading, MA, 1989.
10. M. Dorigo and M. Colombetti, Robot shaping: developing autonomous agents through learning, Artificial Intelligence, vol. 71, no. 2, pp. 321-370,1994.
11. V. Gullapalli, A stochastic reinforcement learning algorithm for learning realvalued functions, Neural Networks, vol. 3, no. 6, pp. 671-692, 1990.
12. M. Huber and R. A. Grupen, A feedback control structure for on-line learning tasks, Robotics and Autonomous Systems, vol. 22, no. 3-4, pp. 303-315, 1997.
13. M. I. Jordan and D. E. Rumelhart, Forward models: Supervised learning with a distal teacher, Cognitive Science, vol. 16, no. 3, pp. 307-354,1992.
14. M. Kaiser and R. Dillmann, Building elementary robot skills from human demonstration, Proc. IEEE International Conference on Robotics and Automation, pp. 2700-2705, IEEE, Piscataway, NJ, 1996.
15. R. M. Kretchmar, P. M. Young, C. W. Anderson, D. C. Hittle, M. L. Anderson, C. C. Delnero, and J. Tu, Robust reinforcement learning control with static and dynamic stability, International Journal of Robust and Nonlinear Control, vol. 11, pp. 1469-1500,2001.
16. L.-J. Lin, Self-improving reactive agents based on reinforcement learning, planning and teaching, Machine Learning, vol. 8, no. 3-4, pp. 293-321,1992.
17. R. Maclin and J. W. Shavlik, Creating advice-taking reinforcement learners, Machine Learning, vol. 22, no. 1-3, pp. 251-281,1996.
18. M. J. Mataric, Reward functions for accelerated learning, Proc. Eleventh International Conference on Machine Learning, pp. 181-189, San Francisco, CA, 1994.
19. M. J. Mataric, Sensory-motor primitives as a basis for imitation: linking perception to action and biology to robotics, in C. Nehaniv and K. Dautenhahn, (eds.), Imitation in Animals and Artifacts, MIT Press, Cambridge, MA, 2000.
20. A. Y. Ng, D. Harada, and S. Russell, Policy invariance under reward transformations: Theory and applications to reward shaping, Proc. Sixteenth International Conference on Machine Learning, pp. 278-287, Morgan Kaufmann, San Francisco, CA, 1999.
21. T. J. Perkins and A. G. Barto, Lyapunov-constrained action sets for reinforcement learning, in C. Brodley and A. Danyluk, (eds.), Proc. 18th International Conference on Machine Learning, pp. 409-416, Morgan Kaufmann, San Francisco, CA, 2001.
22. T. J. Perkins and A. G. Barto, Lyapunov design for safe reinforcement learning, Journal of Machine Learning Research, vol. 3, pp. 803-832, 2002.
23. B. Price and C. Boutilier, Implicit imitation in multiagent reinforcement learning, in I. Bratko and S. Dzeroski, (eds.), Proc. 16th International Conference on Machine Learning, pp. 325-334, Morgan Kaufmann, San Francisco, CA, 1999.
24. J. C. Santamaria, R. S. Sutton, and A. Ram, Experiments with reinforcement learning in problems with continuous state and action spaces, Adaptive Behavior, vol. 6, pp. 163-217, 1997.
25. S. Schaal, Learning from demonstration, in M. C. Mozer, M. I. Jordan, and T. Petsche, (eds.), Advances In Neural Information Processing Systems 9, pp. 1040-1046, MIT Press, Cambridge, MA, 1997.
26. S. Schaal, Is imitation learning the route to humanoid robots? Trends in Cognitive Science, vol. 3, pp. 233-242,1999.
27. J. S. Shamma, Linearization and gain-scheduling, in W. S. Levine, (ed.), The Control Handbook, pp. 388-396, CRC Press, Boca Raton, FL, 1996.
28. S. P. Singh and R. S. Sutton, Reinforcement learning with replacing eligibility traces^ Machine Learning, vol. 22, no. 1-3, pp. 123-158, 1996.
29. W. D. Smart and L. P. Kaelbling, Effective reinforcement learning for mobile robots, Proc. IEEE International Conference on Robotics and Automation, pp. 3404-3410, IEEE, Piscataway, NJ, 2002.
30. R. S. Sutton, Learning to predict by the method of temporal differences, Machine Learning, vol. 3, pp. 9-44,1988.
31. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.
32. C. J. C. H. Watkins and P. Dayan, Q-leaming, Machine Learning, vol. 8, no. 3-4, pp 279-292,1992.
33. R. J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning, Machine Learning, vol. 8, pp. 229-256,1992.