The neuronal replicator hypothesis

Neural Computation

Volumn 22, Issue 11, 2010, Pages 2809-2857

  Fernando, Chrisantha ^a,b,c   Goldstein, Richard ^c   Szathmáry, Eörs ^a,d,e  

^a INSTITUTE FOR ADVANCED STUDY   (Hungary)

^b UNIVERSITY OF SUSSEX   (United Kingdom)

^c NATIONAL INSTITUTE FOR MEDICAL RESEARCH   (United Kingdom)

^d PARMENIDES FOUNDATION   (Germany)

^e EÖTVÖS LORÁND UNIVERSITY   (Hungary)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ANIMAL; BIOLOGICAL MODEL; BRAIN; COGNITION; EVOLUTION; HUMAN; LETTER; NERVE CELL; PHYSIOLOGY;

ALGORITHMS; ANIMALS; BIOLOGICAL EVOLUTION; BRAIN; COGNITION; HUMANS; MODELS, NEUROLOGICAL; NEURONS;

EID: 78149312643     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00031     Document Type: Letter

References (162)

1. Abraham, W. C., & Bear, M. F. (1996). Metaplasticity: The plasticity of synaptic plasticity. Trends in Neurosciences, 19, 126-130.
2. Abraham, W. C., & Robins, A. (2005). Memory retention-the synaptic stability verses plasticity dilemma. Trends in Neurosciences, 28(2), 73-78.
3. Adams, P. (1998). Hebb and Darwin. J. Theor. Biol., 195(4), 419-438.
4. Altenberg, L. (1994). The evolution of evolvability in genetic programming. In K. E. Kinnear (Ed.), Advances in genetic programming (pp. 47-74). Cambridge, MA: MIT Press.
5. Amit, D. J. (1995). The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behav. Brain Res., 18, 617-657.
6. Amit, D. J., Bernacchia, A., & Yakovlev, J. (2003). Multiple-object working memory: A model for behavioural performance. Cereb. Cortex, 13, 435-443.
7. Anderson, J. R. (2007). How can the human mind occur in the physical universe? New York: Oxford University Press.
8. Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996). Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science, 273, 1868-1871.
9. Atmar, W. (1994). Notes on the simulation of evolution. IEEE Trans. Neural Networks, 5(1), 130-148.
10. Aunger, R. (2002). The electric meme: A new theory of how we think. New York: Free Press.
11. Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 47-89). Orlando, FL: Academic Press.
12. Baldwin, M. J. (1898). On selective thinking. Psychological Review, 5(1), 4.
13. Baldwin, M. J. (1909). The influence of Darwin on theory of knowledge and philosophy. Psychological Review, 16, 207-218.
14. Barak, O. (2007). Persistent activity in neural networks with dynamic synapses. PLoS Computational Biology, 3(2), e35.
15. Barto, A. G. (1995). Adaptive critics and the basal ganglia. In J. C. Houk, J. Davis, & D. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215-232). Cambridge, MA: MIT Press.
16. Barto, A. G., Sutton, R. S., & Anderson, J. C. (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13(5), 834-846.
17. Becker, S. A. (2005). A computational principle for hippocampal learning and neurogenesis. Hippocampus, 15, 722-738.
18. Beyer, H.-G. (2001). The theory of evolution strategies. Berlin: Springer.
19. Bongard, J., & Lipson, H. (2007). Automated reverse engineering of nonlinear dynamical systems. Proc. Natl. Acad. Sci. USA, 104, 9943-9948.
20. Bongard, J., Zykov, V., & Lipson, H. (2006). Resilient machines through continuous self-modeling. Science, 314, 1118-1121.
21. Bremermann, H. J. (1958). The evolution of intelligence: The nervous system as a model of its environment. (Tech. Rep. no. 1). Seattle: University ofWashington, Department of Mathematics.
22. Brunel, N. (2003). Dynamics and plasticity of stimulus-selective persistent activity in cortical network models. Cerebral Cortex, 13, 1151-1161.
23. Butz, M., Worgotter, F., & Van Ooyen, A. (2009). Activity-dependent structural plasticity. Brain Research Reviews, 60(2), 287-305.
24. Byrne, C. C., & Kurland, J. A. (2001). Self-deception in an evolutionary game. Journal of Theoretical Biology, 212, 457-480.
25. Campbell, D. T. (1974). The philophy of Karl. R. Popper. In P. A. Schillpp (Ed.), Evolutionary epistemology (pp. 412-463). Chicago: University of Chicago Press.
26. Carpenter, G., & Grossberg, S. (1988). The ART of adaptive pattern recognition by a self-organizing neural network. Computer, 21(3), 77-88.
27. Changeux, J. P., Courrege, P., & Danchin, A. (1973). A theory of the epigenesis of neuronal networks by selective stabilization of synapses. Proc. Natl. Acad. Sci. USA, 70, 2974-2978.
28. Chklovskii, D. B., Mel, B.W., & Swoboda, K. (2004). Cortical rewiring and information storage. Nature, 431, 782-788.
29. Clune, J., Misevic, D., Ofria, C., Lenski, R. E., Elena, S. F., & Sanjuán, R. (2008). Natural selection fails to optimize mutation rates for long-term adaptation on rugged fitness landscapes. PLoS Computational Biology, 4(9), e1000187.
30. Conrad, M. (1983). Adaptability: The significance of variability frommolecule to ecosystem. New York: Plenum Press.
31. Cooper, W. (2001). The evolution of reason: Logic as a branch of biology. Cambridge: Cambridge University Press.
32. Crick, F. H. C. (1989). Neuronal Edelmanism. Trends Neurosci., 12, 240-248.
33. Crick, F. H. C. (1990). Reply. Trends Neurosci., 13, 13-14.
34. Dawkins, R. (1982). The extended phenotype: The gene as the unit of selection. NewYork: Freeman.
35. Dehaene, S., & Changeux, J. P. (1997). A hierarchical neuronal network for planning behavior. Proc. Natl. Acad Sci. USA, 94(24), 13293-13298.
36. Dehaene, S., Changeux, J. P., & Nadal, J. P. (1987). Neural networks that learn temporal sequences by selection. Proc. Natl. Acad. Sci. USA, 84(9), 2727-2731.
37. Dehaene, S., Kerszberg, M., & Changeux, J. P. (1998). A neuronal model of a global workspace in effortful cognitive tasks. Proc. Natl. Acad. Sci. USA, 95(24), 14529-14534.
38. Dennett, D. C. (1981). Brainstorms. Cambridge, MA: MIT Press.
39. Dennett, D. C. (1995). Darwin's dangerous idea. New York: Simon & Schuster.
40. Dominey, P. F. (1995). Complex sensory-motor sequence learning based on recurrent state representations and reinforcement learning. Biological Cybernetics, 73, 265-274.
41. Doya, K., Samejima, K., Katagiri, K., & Kawato, M. (2002). Multiple model-based reinforcement learning. Neural Computation, 14(6), 1347-1369.
42. Durstewitz, D., & Seamans, J. K. (2006). Beyond bistability: Biophysics and temporal dynamics of working memory. Neuroscience, 139, 119-133.
43. Edelman, G. M. (1987). Neural Darwinism: The theory of neuronal group selection. New York: Basic Books.
44. Edelman, G. M. (1994). The evolution of somatic selection: The antibody Tale. Genetics, 138, 975-981.
45. Eigen, M. (1971). Selforganization of matter and the evolution of biological macromolecules. Naturwissenschaften, 58(10), 465-523.
46. Elfwing, S. (2007). Embodied evolution of learning ability. Unpublished doctoral dissertation, Stockholm, KTH.
47. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14(2), 179-211.
48. Fernando, C., Karishma, K. K., & Szathmáry, E. (2008). Copying and evolution of neuronal topology. PLoS ONE, 3(11), e3775.
49. Fernando, C., & Szathmáry, E. (2009a). Chemical, neuronal and linguistic replicators. In M. Pigliucci & G. Müller (Eds.), Towards an extended evolutionary synthesis. Cambridge, MA: MIT Press.
50. Fernando, C., & Szathmáry, E. (2009b). Natural selection in the brain. In B. Glatzeder, V. Goel, & A. von Müller (Eds.), Toward a theory of thinking. Berlin: Springer.
51. Fogel, D. B. (2006). Evolutionary computation: Toward a new Philosophy of machine intelligence. Hoboken, NJ: Wiley-Interscience.
52. Fox, M. D., Corbetta, M., Snyder, A. Z., Vincent, J. L., & Raichle, M. E. (2006). Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proc. Natl. Acad. Sci. USA, 103, 10046-10051.
53. Fransen, E., Tahvildari, B., Egorov, A. V., Hasselmo, M. E., & Alonso, A. A. (2006). Mechanism of graded persistent cellular activity of entorhinal cortex layer V neurons. Neuron, 49, 735-746.
54. Fraser, A. S. (1957). Simulation of genetic systems by automatic digital computers. I. Introduction. Australian J. Biological Sciences, 10, 484-491.
55. Friston, K. J., & Stephan, K. E. (2007). Free-energy and the brain. Synthese, 159, 417-458.
56. Gabora, L. (2005).Creative thought as a non-Darwinian evolutionary process. Journal of Creative Behavior, 39(4), 65-87.
57. Gánti, T. (2003). The principles of life. New York: Oxford University Press.
58. Goldman, M. S., Levine, J. H., Major, G., Tark, D. W., & Seung, H. S. (2003). Robust persistent neural activity in amodel integrator with multiple hysteretic dendrites per neuron. Cereb. Cortex, 13, 1185-1195.
59. Gomez, F., Schmidhuber, J., & Miikkulainen, R. (2008). Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research, 9, 937-965.
60. Hadamard, J. (1945). The psychology of invention in the mathematical field. New York: Dover.
61. Hofer, S. B., Mrsic-Flogel, T. D., Bonfoeffer, T., & Hubener, M. (2009). Experience leaves a lasting structure trace in cortical circuits. Nature, 457, 313-317.
62. Hofstadter, D. R., & Mitchell, M. (1994). The copycat project: A model of mental fluidity and analogy-making. In K. Holyoak & J. Bamden (Eds.), Advances in connectionist and neural computation (Vol. 2, pp. 31-112). Norwood, NJ: Ablex.
63. Holcman, D., & Tsodyks, M. (2006). The emergence of up and down states in cortical networks. PLoS Computational Biology, 2(3), e23.
64. Holtmaat, A., & Sovoboda, K. (2009). Experience-dependent structural plasticity in the mammalian brain. Nature Reviews Neurosicence, 10, 647-658.
65. Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis, & D. G. Belser (Eds.), Models of information processing in the basal ganglia. Cambridge, MA: MIT Press.
66. Houk, J. C., Bastianen, C., Fansler, A., Fishback, A., Fraser, D., Reber, P. J., et al. (2007). Action selection and refinement in subcortical loops through basal ganglia and cerebellum. Phil. Trans. Roy. Soc. B, 29, 1573-1583.
67. Hutter, M. (2005). Universal artificial intelligence: Sequential decisions based on algorithmic probability. Berlin: Springer.
68. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14, 1539-1572.
69. Izhikevich, E.M. (2006). Polychronization: Computation with spikes. Neural Computation, 18(2), 245-282.
70. Izhikevich, E.M. (2007a). Dynamical systems in neuroscience: The geometry of excitability and bursting. Cambridge, MA: MIT Press.
71. Izhikevich, E. M. (2007b). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17, 2443-2452.
72. Izhikevich, E. M., Gally, J. A., & Edelman, G. M. (2004). Spike-timing dynamics of neuronal groups. Cereb. Cortex, 14(8), 933-944.
73. Jacobs, R., Jordan, M., Nowlan, S., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
74. James, W. (1890). The principles of psychology. New York: Dover.
75. Jones, A. G., Arnold, S. J., & Bürger, R. (2007). The mutation matrix and the evolution of evolvability. Evolution, 61, 727-745.
76. Kaelbling, L. P., Littman, M. L., & Moore, A. P. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.
77. Kakade, S., & Dayan, P. (2002). Dopamine: Generalization and bonuses. Neural Networks, 15, 549-559.
78. Kashtan, N., & Alon, U. (2005). Spontaneous evolution of modularity and network motifs. Proc. Natl. Acad. Sci. USA, 102(39), 13773-13778.
79. Kemp, C., & Tenenbaum, J. B. (2008). The discovery of structural form. Proc. Natl. Acad. Sci. USA, 105(31), 10687-10692.
80. Kenet, T., Bibitchkov, D., Tsodyks, M., Grinvald, A., & Arieli, A. (2003). Spontaneously emerging cortical representations of visual attributes. Nature, 425, 954-956.
81. Kirchner, M., & Gerhart, J. (1998). Evolvability. Proc. Natl. Acad. Sci. USA, 95, 8420-8427.
82. Kirchner, M., & Gerhart, J. C. (2005). The plausibility of life. New Haven, CT: Yale University Press.
83. Kohler, W. (1925). The mentality of apes. New York: K. Paul, Trench, Trubner & Co.
84. Kondo, T., & Ito, K. (2004). A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control. Robotics and Autonomous Systems, 46(2), 111-124.
85. Lenski, R. E., Ofria, C., Collier, T. C., & Adami, C. (1999). Genomic complexity, robustness, and genetic interactions in digital organisms. Nature, 400, 661-664.
86. Loewenstein, Y., & Sompolinsky, H. (2003). Temporal integration by calcium dynamics in a model neuron. Nature Neurosci., 6, 961-967.
87. Mahfoud, S. (1995). Niching methods for genetic algorithms. Unpublished doctoral dissertation, University of Illinois.
88. Marcus, G. F. (2001). The algebraic mind: Integrating connectionism and cognitive science. Cambridge, MA: MIT Press.
89. Marr, D. (1983). Vision: A computational investigation into the human representation and processing of visual information. New York: Freeman.
90. Maynard Smith, J. (1986). The problems of biology.New York: Oxford University Press.
91. Maynard Smith, J. (1998). Evolutionary genetics. New York: Oxford University Press.
92. Mechens, C. K., Romo, R., & Brody, C. D. (2005). Flexible control of mutual inhibition: A neural model of two-interval discrimination. Science, 307, 1121-1124.
93. Michalewicz, Z., & Fogel, D. B. (2004). How to solve it: Modern heuristics. Berlin: Springer-Verlag.
94. Michel, O. (2004). Webots: Professional mobile robot simulation. Journal of Advanced Robotics Systems, 1, 39-42.
95. Minsky, M. (1986). The society of mind. New York: Simon and Schuster.
96. Mongillo, G., Barak, O., & Tsodyks, M. (2008). Synaptic theory of working memory. Science, 319, 1543-1546.
97. Monod, J. (1971). Chance and necessity: An essay on the natural philosophy of modern biology. New York: Knopf.
98. Morimoto, J., & Doya, K. (2001). Acquisition of stand-up behaviour by a real robot using hierarchical reinforcement learning. Robotics and Autonomous Systems, 36, 37-51.
99. Muller, H. J. (1966). The gene material as the initiator and organizing basis of life. American Naturalist, 100, 493-517.
100. Nadel, L., Samsonovich, A., Ryan, L., & Moscovitch, M. (2000). Multiple trace theory of human memory: Computational, neuroimaging, and neuropsychological results. Hippocampus, 10, 352-368.
101. Niv, Y. (2009). Reinforcement learning in the brain. Journal ofMathematical Psychology, 53, 139-154.
102. Niv, Y., Joel, D., Meilijson, I., & Ruppin, E. (2002). Evolution of reinforcement learning in uncertain environments:Asimple explanation for complex foraging behaviors. Adaptive Behavior, 10(5-23).
103. Oja, E. (1982). Simplified neuron model as a principal component analyzer. Journal of Mathematical Biology, 15(3), 267-273.
104. O'Reilly, R. C., & Munakata, Y. (2000). Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
105. Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007). Intrinsic motivation systems for autonomoysmental development. IEEE Transactions on Evolutionary Computation, 11(2), 265-286.
106. Parter, M., Kashtan, N., & Alon, U. (2008). Facilitated variation:Howevolution learns from past environments to generalize to new environments. PLoS Computational Biology, 4(11), e1000206.
107. Pennartz, C. M. (1995). The ascending neuromodulatory systems in learning by reinforcement: Comparing computational conjectures with experimental findings. Brain. Res. Rev., 21, 219-245.
108. Pigliucci, M. (2008). Is evolvability evolvable? Nature Reviews Genetics, 9, 75-82.
109. Poirazi, P., & Mel, B. W. (2001). Impact of active dendrites and structural plasticity on the memory capacity of neural tissue. Neuron, 29, 779-796.
110. Price, G. R. (1970). Selection and covariance. Nature, 227, 520-521.
111. Raichle, M. E. (2006). The brain's dark energy. Science, 314, 1249-1250.
112. Raichle, M. E., & Mintun, M.A. (2006). Brain work and brain imaging. Annual Review of Neuroscience, 29, 449-476.
113. Rechenberg, I. (1994). Evolutionstrategie 94. Stuttgart: Frommann-Holzboog.
114. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). Is the short-latency dopamine response too short to signal reward error? Trends in Neuroscience, 22, 146-151.
115. Samejima, K., & Omori, T. (1999). Adaptive internal state space construction method for reinforcement learning of a real-world agent. Neural Networks, 12, 1143-1155.
116. Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005). Representation of actionspecific reward values in the striatum. Science, 310, 1337-1340.
117. Schmidhuber, J. (1991). Curious model-building control systems. In Proc. International Joint Conference on Neural Networks. San Francisco: Morgan Kaufmann.
118. Schmidhuber, J. (1999). A general method for incremental self-improvement and multi-agent learning. In X. Yao (Ed.), Evolutionary computation: Theory and applications. Singapore: World Scientific.
119. Schmidhuber, J. (2000). Evolutionary computation versus reinforcement learning. In Proceedings of the IEEE Industrial Electronics Society. Piscataway, NJ: IEEE.
120. Schmidhuber, J. (2006). Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science, 18(2), 173-187.
121. Schmidhuber, J. (2009). Ultimate cognition à la Gödel. Cognitive Computation, 1(2), 177-193.
122. Schmidhuber, J., Wierstra, D., & Gomez, F. (2005). Evolino: Hybrid neuroevolution / optimal linear search for sequence learning. In Proceedings of the 19th International Joint Conference on Artificial Intelligence (pp. 853-858). Boca Raton, FL: CRC Press.
123. Schmidhuber, J., Zhao, Q., & Schraudolph, N. (1997). Reinforcement learning with self-modifying policies. In S. Thrün & L. Pratt (Eds.), Learning to learn. Norwood, MA: Kluwer.
124. Schmidhuber, J., Zhao, Q., & Wiering, M. (1997). Shifting inductive bias with successstory algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning, 28, 105-130.
125. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27.
126. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593-1599.
127. Schwefel, H. P. (2000). Advantages (and disadvantages) of evolutionary computation over other approaches. In T. Back, D. B. Fogel, & T. Michalewicz (Eds.), Evolutionary computation. NewYork: Taylor and Francis.
128. Seung, S. H. (2003). Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron, 40, 1063-1973.
129. Simonton, D. K. (1995). Foresight in insight? A Darwinian answer. In R. J. Sternberg & J. E. Davidson (Eds.), The nature of insight. Cambridge, MA: MIT Press.
130. Skinner, B. F. (1976). About behaviourism. New York: Vintage.
131. Soltoggio, A., Dürr, P., Mattlussi, C., & Floreano, D. (2007). Evolving neuromodulatory topologies for reinforcement learning-like problems. In Proceedings of the 2007 IEEE Congress on Evolutionary Computation (pp. 2471-2478). Piscataway, NJ: IEEE.
132. Sporns, O., & Edelman, G. M. (1993). Solving Bernstein's problem: A proposal for the development of coordinated movement by selection. Child Development, 64, 960-981.
133. Stanley, K. O., & Miikkulainen, R. (2002). Efficient reinforcement learning through evolving neural networks topologies. In Proceedings of the Genetic and Evolutionary Computation Conference. San Francisco: Morgan Kaufmann.
134. Steels, L., & De Beule, J. (2006a). A (very) brief introduction to fluid construction grammar. In Proceedings of the 3rd International Workshop on Scalable Natural Language. N.p.
135. Steels, L., & De Beule, J. (2006b). Unify and merge in fluid construction grammar. In Symbol Grounding and Beyond: Proceedings of the Third International Workshop on the Emergence and Evolution of Linguistic Communication (pp. 197-223). Berlin: Springer.
136. Sternberg, R. J., & Davidson, J. E. (Eds.). (1995). The nature of insight. Cambridge, MA: MIT Press.
137. Suri, R. E., & Schultz, W. (1999). A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91(3), 871-890.
138. Sutton, S.R., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
139. Szathmáry, E., & Maynard Smith, J. (1997). Fromreplicators to reproducers: The first major transitions leading to life. Journal of Theoretical Biology, 187(4), 555-571.
140. Szinta, I., & Lorincz, A. (2007). Learning to play using low-complexity rule-based polices: Illustrations through Ms. Pac-Man. Journal ofArtificial Intelligence Research, 30, 659-684.
141. Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York: Macmillan.
142. Togelius, J., Schaul, T., Wierstra, D., Igel, C., Gomez, F., & Schmidhuber, J. (in press). Ontogenetic and phylogenetic reinforcement Learning. Kuenstliche Intelligenz.
143. Tononi, G., Sporns, O., & Edelman, G. M. (1992). Reentry and the problem of integrating multiple brain areas: Simulation of dynamic integration in the visual system. Cerebral Cortex, 2, 310-335.
144. Toussaint, M. (2003). The evolution of genetic representations and modular adaptation. Unpublished doctoral dissertation. Universitat Bochum.
145. Tsodyks, M., Kenet, T., Grinvald, A., & Arieli, A. (1999). Linking spontaneous activity of single cortical neurons and the underlying functional architecture. Science, 286, 1943-1946.
146. Voegtlin, T. (2002). Recursive self-organizing maps. Neural Networks, 15, 979-991.
147. Wagner, G. P., & Altenberg, L. (1996). Complex adaptations and evolution of evolvability. Evolution, 50, 329-347.
148. Wagner, U., Gais, S., Haider, H., Verleger, R., & Born, J. (2004). Sleep inspires insight. Nature, 427, 352-355.
149. Wang, J. (1993). A recurrent neural network for real-time matrix inversion. Applied Mathematics and Computation, 55, 89-100.
150. Wang, X.-J. (1999). Synaptic basis of cortical persistent activity: The importance of NMDA receptors to working memory. Journal of Neuroscience, 19(21), 9587-9603.
151. Wang, X.-J. (2001). Synaptic reverberation underlying mnemonic persistent activity. Trends in Neuroscience, 24, 455-463.
152. Watkins, C. (1989). Learning from delayed rewards. Unpublished doctoral dissertation, Cambridge University.
153. Watson, R. A. (2006). Compositional evolution: The impact of sex, symbiosis, and modularity on the gradualist framework of evolution. Cambridge, MA: MIT Press.
154. Watson, R. A., Buckley, C. L., & Mills, R. (2009). The effect of Hebbian learning on optimisation in Hopfield networks (Tech. Rep. ECS). University of Southampton.
155. Watson, R. A., Hornby, G. S., & Pollack, J. B. (1998). Modelling building-block interdependency. InProceedings of the 5th International Conference on Parallel Problem Solving from Nature. San Francisco: Morgan Kaufmann.
156. Whiteson, S., Taylor, M. E., & Stone, P. (2007). Empirical studies in action selection with reinforcement learning. Adaptive Behavior, 15, 33-50.
157. Wiering, M., & Schmidhuber, J. (1997). HQ-Learning. Adaptive Behavior, 6(2), 219-246.
158. Wierstra, D., Foerster, A., Peters, J., & Schmidhuber, J. (2007). Solving deep memory POMDPs with Recurrent Policy Gradients. In Proceedings of the International Conference on Artificial Neural Networks. Berlin: Springer.
159. Wiskott, L., Rasch, M. J., & Kempermann, G. (2006). A functional hypothesis for adult hippocampal neurogenesis: Avoidance of catastrophic interference in the dentate gyrus. Hippocampus, 16(3), 329-343.
160. Worgötter, F., & Porr, B. (2005). Temporal sequence learning, prediction, and control: A review of different models and their relation to biological mechanisms. Neural Computation, 17, 245-319.
161. Zachar, I., & Szathmáry, E. (2010). A new replicator: A theoretical framework for analysing replication. BMC Biology, 8, 21.
162. Zipser, D., Kehoe, B., Littlewort, G., & Fuster, J. (1993). A spiking network model of short-term active memory. Journal of Neuroscience, 13, 3406-3420.