Evolving neuromodulatory topologies for reinforcement learning-like problems

2007 IEEE Congress on Evolutionary Computation, CEC 2007

Volumn , Issue , 2007, Pages 2471-2478

  Soltoggio, Andrea ^a   Dürr, Peter ^b   Mattiussi, Claudio ^b   Floreano, Dario ^b  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b EPFL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

LIVING CREATURES; NEUROMODULATORY TOPOLOGIES;

MATHEMATICAL MODELS; NEURAL NETWORKS; PROBLEM SOLVING; REINFORCEMENT LEARNING;

TOPOLOGY;

EID: 79955215481     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CEC.2007.4424781     Document Type: Conference Paper

References (30)

1. A. C. Roberts and D. L. Glanzman, "Learning in aplysia: looking at synaptic plasticity from both sides," Trends in Neuroscience, vol. 26, no. 12, pp. 662-670, December 2003.
2. C. H. Bailey, M. Giustetto, Y.-Y. Huang, R. D. Hawkins, and E. R. Kandel, "Is heterosynaptic modulation essential for stabilizing hebbian plasticity and memory?" Nature Reviews Neuroscience, vol. 1, no. 1, pp. 11-20, October 2000.
3. E. Marder and V. Thirumalai, "Cellular, synaptic and network effects of neuromodulation," Neural Networks, vol. 15, pp. 479-493, 2002.
4. J.-M. Fellous and C. Linster, "Computational models of neuromodulation," Neural Computation, vol. 10, pp. 771-805, 1998.
5. P. R. Montague, S. E. Hyman, and J. D. Cohen, "Computational roles for dopamine in behavioural control," Nature, vol. 4, pp. 2-9, 2004.
6. N. D. Daw, J. P. O'Doherty, P. Dayan, B. Seymour, and R. J. Dolan, "Cortical substrates for exploratory decisions in humans," Nature, vol. 441, no. 15, pp. 876-879, June 2006.
7. W. Schultz, P. Apicella, and T. Ljungberg, "Responses of Monkey Dopamine Neurons to Reward and Conditioned Stimuli during Successive Steps of Learning a Delayed Response Task," The Journal of Neuroscience, vol. 13, pp. 900-913, 1993.
8. P. R. Montague, P. Dayan, and T. J. Sejnowski, "A Framework for Mesencephalic Dopamine Systems Based on Predictive Hebbian Learning," The Journal of Neuroscience, vol. 16, no. 5, pp. 1936-1947, March 1996.
9. W. Schultz, P. Dayan, and P. R. Montague, "A Neural Substrate for Prediction and Reward," Science, vol. 275, pp. 1593-1598, 1997.
10. W. Schultz, "Predictive Reward Signal of Dopamine Neurons," Journal of Neurophysiology, vol. 80, pp. 1-27, 1998.
11. R. S. Sutton and A. G. Barto, Time-Derivative Models of Pavlonian Reinforcement. MIT Press, 1990, ch. 12, pp. 497-537.
12. -, Reinforcement Learning: An Introduction. MITPress,Cambridge, MA, USA, 1998.
13. P. R. Montague, P. Dayan, C. Person, and T. J. Sejnowski, "Bee foraging in uncertain environments using predictive hebbian learning," Nature, vol. 377, pp. 725-728, October 1995.
14. R. E. Suri, J. Bargas, and M. A. Arbib, "Modeling functions of striatal dopamine modulation in learning and planning," Neuroscience, vol. 103, no. 1, pp. 65-85, 2001.
15. Y. Niv, D. Joel, I. Meilijson, and E. Ruppin, "Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviours," Adaptive Behaviour, vol. 10, no. 1, pp. 5-24, 2002.
16. K. Doya, "Metalearning and neuromodulation," Neural Networks, vol. 15, no. 4-6, pp. 495-506, 2002.
17. D. Joel, Y. Niv, and E. Ruppin, "Actor-critic models of the basal ganglia: new anatomical and computational perspectives," Neural Networks, vol. 15, pp. 535-547, 2002.
18. O. Sporns and W. H. Alexander, "Neuromodulation and plasticity in an autonomous robot," Neural Networks, vol. 15, pp. 761-774, 2002.
19. K. Doya and U. Eiji, "The Cyber Rodent Project: Exploration and Adaptive Mechanisms for Self-Preservation and Self-Reproduction," Adaptive Behavior, vol. 13, no. 2, pp. 149-160, 2005.
20. C. Mattiussi, "Evolutionary synthesis of analog networks," Ph.D. dissertation, Laboratory of Intelligent System (LIS) - EPFL - Lausanne, Switzerland, Lausanne, 2005. [Online]. Available: http://library.epfl.ch/theses/ ?nr=3199
21. C. Mattiussi and D. Floreano, "Analog Genetic Encoding for the Evolution of Circuits and Networks," IEEE Transactions on Evolutionary Computation, vol. to appear, 2007.
22. P. Dürr, C. Mattiussi, and D. Floreano, "Neuroevolution with Analog Genetic Encoding," in PPSN 2006, vol. 9, 2006, pp. 671-680. [Online]. Available: http://ppsn2006.raunvis.hi.is/
23. D. Marbach, C. Mattiussi, and D. Floreano, "Bio-mimetic Evolutionary Reverse Engineering of Genetic Regulatory Networks," in EvoBIO: Fifth European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics, vol. LNCS 4447, 2007, pp. 155-165.
24. X. Yao, "Evolving artificial neural networks," Proceedings of the IEEE, vol. 87, no. 9, pp. 1423-1447, September 1999.
25. K. O. Stanley and R. Miikkulainen, "Evolving neural networks through augmenting topologies," Evolutionary Computation, vol. 10, no. 2, pp. 99-127, May 2002.
26. M. Hammer, "An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees," Nature, vol. 366, pp. 59-63, November 1993.
27. R. Menzel, "Memory dynamics in the honeybee," Journal of Comparative Physiology A: Neuroethology, Sensory, Neural, and Behavioural Physiology, vol. 185, pp. 323-340, 1999.
28. -, "Searching for the Memory Trace in a Mini-Brain, the Honeybee," Learning and Memory, vol. 8, pp. 53-62, 2001.
29. R. Menzel and M. Giurfa, "Cognitive architecture of a mini-brain: the honeybee," Trends in Cognitive Sciences, vol. 5, no. 2, pp. 62-71, February 2001.
30. T. Bäck, D. B. Fogel, and Z. Michalevicz, Eds., Handbook of Evolutionary Computation. Oxford: Oxford University Press, 1997.