Isotropic-sequence-order learning in a closed-loop behavioural system

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences

Volumn 361, Issue 1811, 2003, Pages 2225-2244

  Porr, Bernd ^a   Wörgötter, Florentin ^a  

^a UNIVERSITY OF STIRLING   (United Kingdom)

Author keywords

Autonomous behaviour; Control theory; Inverse controller; Reinforcement learning; Temporal sequence learning

Indexed keywords

EID: 0742301619     PISSN: 1364503X     EISSN: None     Source Type: Journal    
DOI: 10.1098/rsta.2003.1273     Document Type: Article

References (30)

1. Blinchikoff, H. J. 1976 Filtering in the time and frequency domain. Wiley.
2. Brooks, R. A. 1989 How to build complete creatures rather than isolated cognitive simulators. In Architectures for intelligence (ed. K. VanLehn), pp. 225-239. Hillsdale, NJ: Erlbaum.
3. Der, R. & Liebscher, R. 2002 True autonomy from self-organised adaptivity. In WGW'02, Proc. EPSRC/BBSRC Int. Biologically Inspired Robotics: the legacy of W. Grey Walter, HP Bristol Labs, Bristol, UK, 14-16 August 2002 (ed. R. Damper & D. Cliff).
4. Doetsch, G. 1961 Guide to the applications of the Laplace and z-transforms. London: Van Nostrand Reinhold.
5. Doya, K. 2000 Reinforcement learning in continuous time and space. Neural Netw. 12, 219-245.
6. Ekdahl, B. 2000 Agents as anticipatory systems. In Proc. 4th World Multiconference on Systemic, Cybernetics and Informatics (SCI 2000) and 6th Int. Conf. on Information Systems Analysis and Synthesis (ISAS 2000), Orlando, FL 23-26 July 2000, pp. 133-137. Skokie, IL: IIIS.
7. Haken, H. 1995 Entstehung von Biologischer Information und Ordnung. Darmstadt: Wissenschaftliche Buchgesellschaft.
8. Haruno, M., Wolpert, D. M. & Kawato, M. 2001 Mosaic model for sensorimotor learning and control. Neural Comput. 13, 2201-2220.
9. Klopf, A. H. 1986 A drive-reinforcement model of single neuron function. In Neural networks for computing: AIP Conf. Proc. (ed. J. S. Denker), vol. 151, pp. 265-270. New York: American Institute of Physics.
10. McFarland, D. J. 1971 Feedback mechanisms in animal behaviour. Academic.
11. McFarland, D. J. 1989 Problems of animal behaviour. Harlow: Longman.
12. Millán, J. D. R., Posentano, D. & Dedieu, E. 2002 Continuous-action Q-learning. Mach. Learn. 49, 247-265.
13. Palm, W. J. 2000 Modelling, analysis and control of dynamic systems. Wiley.
14. Pfeifer, R. & Scheier, C. 1998 Representation in natural and artificial agents: a perspective from embodied cognitive science. Z. Naturforsch. C 53, 480-503.
15. Pfeifer, R. & Scheier, C. 1999 Understanding intelligence. Cambridge, MA: MIT Press.
16. Porr, B. & Wörgötter, F. 2002 Isotropic sequence order learning using a novel linear algorithm in a closed loop behavioural system. Biosystems 67, 195-202.
17. Rescorla, R. & Wagner, A. 1972 A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In Classical conditioning 2. Current theory and research (ed. A. Black & W. Prokasy), pp. 64-99. New York: Appleton-Century-Crofts.
18. Roberts, P. D. 1999 Temporally asymmetric learning rules. I. Differential Hebbian learning. J. Comput. Neurosci. 7, 235-246.
19. Santos, J. M. & Touzet, C. 1999 Exploration-tuned reinforcement function. Neurocomputing 28, 93-105.
20. Scheier, C. & Lambrosios, D. 1996 Categorization in a real-world agent using haptic exploration and active perception. In From animals to animals, 4th Int. Conf. on Simulation of Adaptive Behaviour, Cape Cod, MA. Cambridge, MA: MIT Press.
21. Shepherd, G. M. (ed.) 1990 The synaptic organisation of the brain. Oxford University Press.
22. Stewart, J. L. 1960 Fundamentals of signal theory. New York: McGraw-Hill.
23. Sutton, R. 1988 Learning to predict by method of temporal differences. Mach. Learn. 3, 9-44.
24. Sutton, R. & Barto, A. 1981 Towards a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88, 135-170.
25. Sutton, R. S. & Barto, A. G. 1982 Simulation of anticipatory responses in classical conditioning by a neuron-like adaptive element. Behav. Brain Sci. 4, 221-235.
26. Sutton, R. S. & Barto, A. G. 1987 A temporal-difference model of classical conditioning. In Proc. 9th Ann. Conf. Cognitive Sci. Soc., pp. 355-378. Mahwah, NJ: Erlbaum.
27. Sutton, R. S. & Barto, A. G. 1990 Time-derivative models of Pavlovian reinforcement. In Learning and computational neuroscience. Foundations of adaptive networks (ed. M. Gabriel & J. Moore), pp. 497-537. Cambridge, MA: MIT Press.
28. Verschure, P. & Voegtlin, T. 1998 A bottom-up approach towards the acquisition, retention, and expression of sequential representations: distributed adaptive control. III. Neural Netw. 11, 1531-1549.
29. von Glasersfeld, E. 1996 Learning and adaptation in constructivism. In Critical readings on Piaget (ed. L. Smith), pp. 22-27. London: Routledge.
30. Wolpert, D. M. & Ghahramani, Z. 2000 Computational principles of movement neuroscience. Nature Neurosci. Suppl. 3, 1212-1217.