An introduction to stochastic control theory, path integrals and reinforcement learning

AIP Conference Proceedings

Volumn 887, Issue , 2007, Pages 149-181

  Kappen, Hilbert J ^a  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

Path integral control; Reinforcement learning; Stochastic optimal control

Indexed keywords

EID: 33947410345     PISSN: 0094243X     EISSN: 15517616     Source Type: Conference Proceeding    
DOI: 10.1063/1.2709596     Document Type: Conference Paper

References (38)

1. L. Abbott, J. Varela, K. Sen, and S. Nelson, Science pp. 220-224 (1997).
2. D. Blitz, K. Foster, and W. Regehr, Nature Rev. Neurosci. 5, 630-640 (2004).
3. J. Gray, and P. Thompson, Nature Rev. Neurosci. 5 (2004).
4. W. T. Dickens, and J. R. Flynn, Psychol. Rev. 108, 346-369 (2001).
5. D. Hebb, The organization of behaviour, Wiley, New York, 1949.
6. S. Kelso, A. Ganong, and T. Brouwn, Proc. Natl. Acad. Sci. USA 83, 5326-5330 (1986).
7. P. Dayan, and L. Abbott, Theoretical Neuroscience. Computational and Mathematical Modeling of Neural Systems, MIT Press, New York, 2001.
8. R. Stengel, Optimal control and estimation, Dover publications, New York, 1993.
9. W. Fleming, and H. Soner, Controlled Markov Processes and Viscosity solutions, Springer Verlag, 1992.
10. R. Sutton, Machine Learn. 3, 9-44 (1988).
11. C. Watkins, Learning from delayed rewards, Ph.D. thesis, University of Cambridge, England (1989).
12. A. G. Barto, "," in Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser, MIT Press, Cambridge, Massachusetts, 1995, pp. 215-232.
13. R. Crites, and A. Barto, "Improving elevator performance using reinforcement learning," in Advances in Neural Information Processing Systems 8: Proceedings of the 1995 Conference, MIT Press, Cambridge MA, 1996.
14. S. Schaal, and C. Atkeson, Control Syst. Magaz. 14 (1994).
15. G. Tesauro, Communications of the ACM 38, 58-68 (1995).
16. L. Sugrue, G. Corrado, and W. Newsome, Nature Rev. Neurosci. 6, 365-375 (2005).
17. D. J. Barraclough, M. L. Conroy, and D. Lee, Nature Neurosci. 7, 404-410 (2004).
18. W. Schultz, Annu. Rev. Psychol. 57, 87-115 (2006).
19. R. Sutton, and A. Barto, Reinforcement learning: an introduction, MIT Press, 1998.
20. L. Pontryagin, V. Boltyanskii, R. Gamkrelidze, and E. Mishchenko, The mathematical theory of optimal processes. Interscience, 1962.
21. H. Kappen, Phys. Rev. Lett. 95, 200201 (2005).
22. H. Kappen, J. Stat. Mech.: Theor. & Exp. p. P11011 (2005).
23. B. Broek, W. W., and H. Kappen, J. Artif. Intell. Resear. (2006), in preparation.
24. W. Wiegerinck, B. v. d. Broek, and H. Kappen, "Stochastic optimal control in continuous space-time multi-agent systems," in Proceedings UAI, Association for Uncertainty in Artificial Intelligence, 2006, in press.
25. R. Bellman, and R. Kalaba, Selected papers on mathematical trends in control theory, Dover, 1964.
26. J. Yong, and X. Zhou, Stochastic controls. Hamiltonian Systems and HJB Equations, Springer, 1999.
27. W. Fleming, Applied Math. Optim. 4, 329-346 (1978).
28. U. Jönsson, C. Trygger, and P. Ögren, Lectures on optimal control (2002).
29. E. Nelson, Dynamical Theories of Brownian Motion, Princeton University Press, Princeton, 1967.
30. F. Guerra, "Introduction to Nelson Stochastic mechanics as a Model for Quantum Mechanics," in The Foundation of Quantum Mechanics, Kluwer, Amsterdam, 1995.
31. L. Kaelbling, M. Littman, and A. Moore, J. Artif. Intell. Resear. 4, 237-285 (1996).
32. W. Schultz, P. Dayan, and P. Montague, Science 275, 1593-1598 (1997).
33. P. R. Montague, and G. S. Berns, Neuron 36, 265-284 (2002).
34. S. M. McClure, N. D. Daw, and P. R. Montague, Trends Neurosci. 26, 423-428 (2003).
35. P. R. Montague, S. E. Hyman, and J. D. Cohen, Nature 431, 760-767 (2004).
36. R. White, Psychol. Rev. 66, 297-333 (1959).
37. D. E. Berlyne, Conflict, Arousal, and Curiosity, McGraw-Hill, New York, 1960.
38. S. Singh, A. Barto, and N. Chentanez, "Intrinsically motivated reinforcement learning," in Advances in Neural Information Processing Systems 17: Proceedings of the 2004 Conference, MIT Press, Cambridge MA, 2005.