Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios

Artificial Life XI: Proceedings of the 11th International Conference on the Simulation and Synthesis of Living Systems, ALIFE 2008

Volumn , Issue , 2008, Pages 569-576

  Soltoggio, Andrea ^a   Bullinaria, John A ^a   Mattiussi, Claudio ^b   Dürr, Peter ^b   Floreano, Dario ^b  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b EPFL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

BIOLOGICAL NEURAL NETWORKS; KEY FACTORS; LEARNING AND MEMORY; LEARNING PROBLEM; LEARNING TASKS; NEURAL CONTROL; NEURAL PLASTICITY; NEUROMODULATION; SIMULATED EVOLUTIONS;

BIOLOGICAL SYSTEMS; NEURONS;

NEURAL NETWORKS;

EID: 84874614281     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Abbott, L. F. and Regehr, W. G. (2004). Synaptic computation. Nature, 431:796-803.
2. Bäck, T., Fogel, D. B., and Michalevicz, Z., editors (1997). Handbook of Evolutionary Computation. Oxford University Press, Oxford.
3. Bailey, C. H., Giustetto, M., Huang, Y.-Y., Hawkins, R. D., and Kandel, E. R. (2000a). Is heterosynaptic modulation essential for stabilizing hebbian plasticity and memory? Nature Reviews Neuroscience, 1(1):11-20.
4. Bailey, C. H., Giustetto, M., Zhu, H., Chen, M., and Kandel, E. R. (2000b). A novel function for serotonin-mediated short-term facilitation in aplysia: Conversion of a transient, cell-wide homosynaptic hebbian plasticity into a persistent, protein synthesis-independent synapse-specific enhancement. PNAS, 97(21):11581-11586.
5. Birmingham, J. T. and Tauck, D. L. (2003). Neuromodulation in invertebrate sensory systems: from biophysics to behavior. The Journal of Experimental Biology, 20:3541-3546.
6. Blynel, J. and Floreano, D. (2002). Levels of dynamics and adaptive behavior in evolutionary neural controllers. In Proceedings of the seventh international conference on simulation of adaptive behavior on From animals to animats, pages 272- 281. MIT Press Cambridge, MA, USA.
7. Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., and Byrne, J. H. (2002). Operant Reward Learning in Aplysia: Neuronal Correlates and Mechanisms. Science, 296(5573):1706-1709.
8. Britannica (2007). Animal learning. Encyclopedia Britannica 2007 Ultimate Reference Suite.
9. Burrell, B. D. and Sahley, C. L. (2001). Learning in simple systems. Current Opinion in Neurobiology, 11:757-764.
10. Carew, T. J., Walters, E. T., and Kandel, E. R. (1981). Classical conditioning in a simple withdrawal reflex in aplysia californica. The Journal of Neuroscience, 1(12):1426-1437.
11. Cooper, S. J. (2005). Donald O. Hebb's synapse and learning rule: a history and commentary. Neuroscience and Biobehavioral Reviews, 28(8):851-874.
12. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4-6):495-506.
13. Fellous, J.-M. and Linster, C. (1998). Computational models of neuromodulation. Neural Computation, 10:771-805.
14. Jay, M. T. (2003). Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Progress in Neurobiology, 69(6):375-390.
15. Kondo, T. (2007). Evolutionary design and behaviour analysis of neuromodulatory neural networks for mobile robots control. Applied Soft Computing, 7(1):189-202.
16. Marcus, E. A. and Carew, T. J. (1998). Developmental emergence of different forms of neuromodulation in Aplysia sensory neurons. PNAS, Neurobiology, 95:4726-4731.
17. Matlab (2007). Box plots. The Mathworks Documentation.
18. Niv, Y., Joel, D., Meilijson, I., and Ruppin, E. (2002). Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviours. Adaptive Behaviour, 10(1):5-24.
19. Roberts, A. C. and Glanzman, D. L. (2003). Learning in aplysia: looking at synaptic plasticity from both sides. Trends in Neuroscience, 26(12):662-670.
20. Rowe, J. E. and Hidovic, D. (2004). An evolution strategy using a continuous version of the gray-code neighbourhood distribution. In GECCO (1), pages 725-736.
21. Schultz, W., Apicella, P., and Ljungberg, T. (1993). Responses of Monkey Dopamine Neurons to Reward and Conditioned Stimuli during Successive Steps of Learning a Delayed Response Task. The Journal of Neuroscience, 13:900-913.
22. Schultz, W., Dayan, P., and Montague, P. R. (1997). A Neural Substrate for Prediction and Reward. Science, 275:1593-1598.
23. Smith, T., Husbands, P., Philippides, A., and O'Shea, M. (2002). Neuronal Plasticity and Temporal Adaptivity: GasNet Robot Control Networks. Adaptive Behaviour, 10:161-183.
24. Soltoggio, A., Dürr, P., Mattiussi, C., and Floreano, D. (2007). Evolving Neuromodulatory Topologies for Reinforcement Learning-like Problems. In Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2007.
25. Sporns, O. and Alexander, W. H. (2002). Neuromodulation and plasticity in an autonomous robot. Neural Networks, 15:761- 774.
26. Sun, Z.-Y. and Schacher, S. (1998). Binding of Serotonin to Receptors at Multiple Sites Is Required for Structural Plasticity Accompanying Long-TermFacilitation of Aplysia Sensorimotor Synapses. The Journal of Neuroscience, 18(11):3991-4000.
27. Ziemke, T. and Thieme, M. (2002). Neuromodulation of Reactive Sensorimotor Mappings as Short-Term Memory Mechanism in Delayed Response Tasks. Adaptive Behavior, 10:185-199.