Concurrent Q-learning: Reinforcement learning for dynamic goals and environments

International Journal of Intelligent Systems

Volumn 20, Issue 10, 2005, Pages 1037-1052

  Ollington, Robert B ^a   Vamplew, Peter W ^a  

^a UNIVERSITY OF TASMANIA   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; INFORMATION ANALYSIS; PROBLEM SOLVING;

CONCURRENT Q-LEARNING; REINFORCEMENT LEARNING;

LEARNING SYSTEMS;

EID: 27844586782     PISSN: 08848173     EISSN: None     Source Type: Journal    
DOI: 10.1002/int.20105     Document Type: Article

References (27)

1. Sutton RS. Learning to predict by the methods of temporal differences. Mach Learn 1988;3:9-44.
2. Foster DJ, Morris RGM, Dayan P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 2000;10(1):1-16.
3. Sutton RS, Barto AG. Reinforcement learning: An introduction. Cambridge, MA: MIT Press; 1998.
4. Morris RGM. Spatial localization does not require the presence of local cues. Learn Motiv 1981;12:239-260.
5. Steele RJ, Morris RGM. Delay-dependent impairment of a matching-to-place task with chronic and intrahippocampal infusion of the NMDA-Antagonist D-AP5. Hippocampus 1999;9(2):118-136.
6. O'Keefe I Place units in the hippocampus of the freely moving rat. Exp Neural 1976;51: 78-109.
7. O'Keefe J, Dostrovsky J. The hippocampus as a spatial map. Preliminary evidence from the unit activity in the freely-moving rat. Brain Res 1971;34:171-175.
8. Muller RU, Kubie JL. The firing of hippocampal place cells predicts the future position of freely moving rats. J Neurosci 1989;9(12):4101-4110.
9. Joel D, Niv Y, Ruppin E. Actor-critic models of the basal ganglia: New anatomical and computational perspectives. Neural Netw 2002;15:535-547.
10. Schultz W, Dickinson A. Neuronal coding of prediction errors. Ann Rev Neurosci 2000;23:473-500.
11. Schultz W. Tremblay L, Hollerman JR. Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb Cortex 2000;10:272-283.
12. Barto AG, Sutton RS, Anderson CW. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans Syst Man Cyber 1983;13(5):834-846.
13. Kaelbling LP. Learning to achieve goals. Chambéry, France: Morgan Kaufmann; 1993.
14. Dayan P, Hinton GE. Feudal reinforcement learning. In: Giles CL, Hanson SJ, Cowan JD, editors. Advances in neural information processing systems 5. San Mateo, CA: Morgan Kaufmann; 1993.
15. Dietterich TG. The MAXQ method for hierarchical reinforcement learning. In: Proc Fifteenth Int Conf on Machine Learning. San Francisco, CA: Morgan Kaufmann; 1998. pp 118-126.
16. Digney BL. Emergent hierarchical control structures: Learning reactive/hierarchical relationships in reinforcement environments. In: Proc Fourth Conf on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press; 1996.
17. Drummond C. Accelerating reinforcement learning by composing solutions of automatically identified subtasks. J Artif Intell Res 2002;16:59-104.
18. Parr R, Russell S. Reinforcement learning with hierarchies of machines. In: Jordan MI, Kearns MJ, Solla SA, editors. Advances in neural information processing systems. Cambridge, MA: The MIT Press; 1997.
19. Singh SP. Reinforcement learning with a hierarchy of abstract models. In: Proc Tenth National Conf on Artificial Intelligence, San Jose, CA; 1992. pp 202-207.
20. Ollington RB, Vamplew PW. Concurrent Q-learning for autonomous mapping and navigation. In: Vadakkepat P, Tan WW, Tan WC, Loh AP, editors. Proc 2nd Int Conf on Computational Intelligence, Robotics and Autonomous Systems, Singapore. National University of Singapore; 2003.
21. Rummery GA, Niranjan M. On-line Q-learning using connectionist systems. Report CUED/F-INFENG/TR 166, Cambridge University Engineering Department; 1994.
22. Singh SP, Sutton RS. Reinforcement learning with replacing eligibility traces. Mach Learn 1996;22:123-158.
23. Sutton RS. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming; San Mateo, CA: Morgan Kaufmann; 1990. pp 216-224.
24. Watkins CJCH, Dayan P. Q-learning. Mach Learn 1992;8(3-4):279-292.
25. Peng J, Williams RJ. Incremental multi-step Q-learning. Mach Learn 1996;22(1-3): 283-290.
26. Wierling M, Schmidhuber J. Fast online Q(λ). Mach Learn 1998;33:105-115.
27. Kaelbling LP. Hierarchical learning in stochastic domains: Preliminary results. In: Proc Tenth Int Conf on Machine Learning. San Francisco, CA: Morgan Kaufmann; 1993. pp 167-173.