Forced moves or good tricks in design space? Landmarks in the evolution of neural mechanisms for action selection

Adaptive Behavior

Volumn 15, Issue 1, 2007, Pages 9-31

  Prescott, Tony J ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Action selection; Behavior based robots; Computational neuroscience; Design space; Nervous system evolution; Precambrian trace fossils

Indexed keywords

ANIMALIA; BILATERIA; CNIDARIA; SCYPHOZOA; SPHENISCIDAE; VERTEBRATA;

EID: 33847290342     PISSN: 10597123     EISSN: 17412633     Source Type: Journal    
DOI: 10.1177/1059712306076252     Document Type: Review

References (123)

1. Adoutte, A., Balavoine, G., Lartillot, N., Lespinet, O., Prud'homme, B., & de Rosa, R. (2000). The new animal phylogeny: Reliability and implications. Proceedings of the National Academy of Sciences of the United States of America, 97, 4453-4456.
2. Albert, J. (1999). Computational modeling of an early evolutionary stage of the nervous system. Biosystems, 54 (1-2). 77-90.
3. Baguna, J., & Riutort, M. (2004). Molecular phylogeny of the Platyhelminthes. Canadian Journal of Zoology - Revue Canadienne De Zoologie, 82 (2). 168-193.
4. Baguna, J., Ruiz-Trillo, I., Paps, J., Loukota, M., Ribera, C., Jondelius, U., & Riutort, M. (2001). The first bilaterian organisms: Simple or complex? New molecular evidence. International Journal of Developmental Biology, 45, S133 - S134.
5. Balavoine, G., de Rosa, R., & Adoutte, A. (2002). Hox clusters and bilaterian phylogeny. Molecular Phylogenetics and Evolution, 24 (3). 366-373.
6. Barrington, E. J. W. (1967). Invertebrate Structure and Function. Sunbury-on-Thames, UK: Nelson.
7. Berntson, G. G., & Micco, D. J. (1976). Organization of brainstem behavioral systems. Brain Research Bulletin, 1, 471-483.
8. Berridge, K. C. (1994). The development of action patterns. In J. A. Hogan & J. J. Bolhuis (Eds.), Causal mechanisms of behavioural development (pp. 147-180). Cambridge, UK: CUP.
9. Brooks, R. A. (1986). A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation, 2, 14-23.
10. Brooks, R. A. (1991). New approaches to robotics. Science, 253, 1227-1232.
11. Brooks, R. A. (1994). Coherent behaviour from many adaptive processes. In D. Cliff, P. Husbands, J.-A. Meyer, & S. W. Wilson (Eds.), From animals to animats 3: Proceedings of the third International Conference on the Simulation of Adaptive Behavior (pp. 22-29). Cambridge, MA: MIT Press.
12. Bryson, J. (2000). Cross-paradigm analysis of autonomous agent architecture. Journal of Experimental and Theoretical Artificial Intelligence, 12 (2). 165-189.
13. Butler, A. B., & Hodos, W. (1996). Comparative vertebrate neuroanatomy. New York: Wiley-Liss.
14. Chevalier, G., & Deniau, J. M. (1990). Disinhibition as a basic process in the expression of striatal functions. Trends in Neurosciences, 13, 277-280.
15. Cisek, P. (in press). Cortical mechanisms of action selection: The affordance competition hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences.
16. Crimes, T. P. (1992). Changes in the trace fossil biota across the proterozoic phanerozoic boundary. Journal of the Geological Society, 149, 637-646.
17. Crimes, T. P., & Anderson, M. M. (1985). Trace fossils from late Precambrian - early Cambrian strata of southeastern Newfoundland (Canada): Temporal and environmental implications. Journal of Paleontology, 59, 310-343.
18. Davis, W. J. (1979). Behavioral hierarchies. Trends in Neuro-sciences, 2, 5-7.
19. Deco, G., & Rolls, E. T. (2005). Attention, short-term memory, and action selection: A unifying theory. Progress in Neurobiology, 76, 236-256.
20. Dennett, D. (1995). Darwin's dangerous idea. London: Penguin Books.
21. DeSouza, G. N., & Kak, A. C. (2004). A subsumptive, hierarchical, and distributed vision-based architecture for smart robotics. IEEE Transactions on Systems Man and Cybernetics Part B: Cybernetics, 34, 1988-2002.
22. Duncan, J., Humphreys, G., & Ward, R. (1997). Competitive brain activity in visual attention. Current Opinion in Neurobiology, 7, 255-261.
23. Eaton, R. C., Hofve, J. C., & Fetcho, J. R. (1995). Beating the competition - the reliability hypothesis for Mauthner axon size. Brain Behavior and Evolution, 45, 183-194.
24. Eaton, R. C., Lee, R. K. K., & Foreman, M. B. (2001). The Mauthner cell and other identified neurons of the brainstem escape network of fish. Progress in Neurobiology, 63, 467-485.
25. Elliott, C. J. H., & Susswein, A. J. (2002). Comparative neuroethology of feeding control in molluscs. Journal of Experimental Biology, 205, 877-896.
26. Erwin, D. H., & Davidson, E. H. (2002). The last common bilaterian ancestor. Development, 129, 3021-3032.
27. Esch, T., & Kristan, W. B. (2002). Decision-making in the leech nervous systern. Integrative and Comparative Biology, 42, 716-724.
28. Finlay, B. L., & Darlington, R. B. (1995). Linked regularities in the development and evolution of mammalian brains. Science, 268, 1578-1584.
29. Fransen, E., & Lansner, A. (1998). A model of cortical associative memory based on a horizontal network of connected columns. Network, 9, 235-264.
30. Gabor Miklos, G. L., Campbell, K. S. W., & Kankel, D. R. (1994). The rapid emergence of bio-electronic novelty, neuronal architectures, and organismal performance. In R. J. Greenspan (Ed.), Flexibility and constraint in behavioural systems (pp. 269-293). Chichester, UK: Wiley.
31. Gallistel, C. R. (1980). The organization of action: A new synthesis. Hillsdale, NJ: Lawrence Erlbaum.
32. Grillner, S., Hellgren, J., Menard, A., Saitoh, K., & Wikstrom, M. A. (2005). Mechanisms for selection of basic motor programs - roles for the striatum and pallidum. Trends in Neurosciences, 28, 364-370.
33. Grimmelikhuijzen, C. J. P., & Westfall, L. A. (1995). The nervous systems of Cnidarians. In O. Breidbach & W. Kutsch (Eds.), The Nervous Systems of Invertebrates: An Evolutionary and Comparative Approach (pp. 7-24). Basel, Switzerland: Birkhauser Verlag.
34. Groenewegen, H. J., Wright, C. I., & Beijer, A. V. J. (1996). The nucleus accumbens: Gateway for limbic structures to reach the motor system ? Progress in Brain Research, 107, 485-511.
35. Gruber, S. A., & Ewer, D., W. (1962). Observations on the myo-neural physiology of the polyclad, Planocera gilchristi. Journal of Experimental Biology, 39, 459-477.
36. Gurney, K., Prescott, T. J., Wickens, J., & Redgrave, P. (2004). Computational models of the basal ganglia: From membranes to robots. Trends in Neurosciences, 27, 453-459.
37. Halstead, L. B. (1965). Pssammosteiformes (Agnatha): A review with descriptions of new material from the lower Devonian of Poland. I. General part. Palaeontologia Polonica, 13 (vii). 1-135.
38. Hamner, W. M. (1995). Sensory ecology of scyphomedusae. Marine and Freshwater Behaviour and Physiology, 26 (2-4). 101-118.
39. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (in press). Toward an executive without a homunculus: Computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society B: Biological Sciences.
40. Hodos, W. (1982). Some perspectives on the evolution of intelligence and the brain. In D. R. Griffin (Ed.), Animal mind-human mind (pp. 33-35). Berlin: Springer-Verlag.
41. Horridge, A. (1956). The nervous system of the ephyra larva of Aurellia aurita. Quarterly Journal of Microscopical Science, 97, 59-74.
42. Horridge, G. A. (1968). The origins of the nervous system. In G. H. Bourne (Ed.), The structure and function of nervous tissue (Vol. 1, pp. 1-31). New York: Academic Press.
43. Humphries, M. D., Gurney, K., & Prescott, T. J. (2006). The brainstem reticular formation is a small-world, not scale-free network. Proceedings of the Royal Society B: Biological Science, 273, 503-531.
44. Humphries, M. H., Gurney, K., & Prescott, T. J. (in press). Is there a brainstem substrate for action selection? Philosophical Transactions of the Royal Society B: Biological Sciences.
45. Josephson, R. K. (1974). Cnidarian neurobiology. In L. Muscatine & H. M. Lenhoff (Eds.), Coelenterate biology: Reviews and new perspectives (pp. 245-273). New York: Academic Press.
46. Josephson, R. K., & Mackie, G. O. (1965). Multiple pacemakers and the behaviour of the hydroid Tubularia. Journal of Experimental Biology, 43, 293-332.
47. Jung, L. J., & Scheller, R. H. (1991). Peptide processing and targeting in the neuronal secretory pathway. Science, 251, 1330-1335.
48. Kandel, E. R., Krasne, F. B., Strumwasser, F., & Truman, J. W. (1979). Cellular mechanisms in the selection and modulation of behavior. Neurosciences Research Program Bulletin, 17, 529-710.
49. Knoll, A. H., & Carroll, S. B. (1999). Early animal evolution: Emerging views from comparative biology and geology. Science, 284, 2129-2137.
50. Koopowitz, H. (1970). Feeding behaviour and the role of the brain in the polyclad flatworm, Planocera gilchristi. Animal Behaviour, 18, 31-35.
51. Koopowitz, H. (1986). On the evolution of central nervous systems - implications from polyclad turbellarian neurobiology. Hydrobiologia, 132, 79-87.
52. Koopowitz, H. (1989). Polyclad neurobiology and the evolution of central nervous systems. In P. A. V. Anderson (Ed.), Evolution of the first nervous systems (pp. 315-327). New York: Plenum Press.
53. Koopowitz, H., & Keenan, L. (1982). The primitive brains of platyhelminthes. Trends in Neurosciences, 5, 77-79.
54. Koopowitz, H., Silver, D., & Rose, G. (1975). Neuronal plasticity and recovery of function in a polyclad flatworm. Nature, 256, 737-738.
55. Koopowitz, H., Silver, D., & Rose, G. (1976). Primitive nervous systems, control and recovery of feeding behavior in the polyclad flatworm, Notoplana acticola. Biological Bulletin, 150, 411-425.
56. Kovac, M. P., & Davis, W. J. (1980). Neural mechanisms underlying behavioral choice in pleurobranchaea. Journal of Neurophysiology, 43, 469-487.
57. Krawczyk, D. C. (2002). Contributions of the prefrontal cortex to the neural basis of human decision-making. Neuro-science and Biobehavioral Reviews, 26, 631-664.
58. Kristan, W. B., & Shaw, B. K. (1997). Population coding and behavioral choice. Current Opinion in Neurobiology, 7, 826-831.
59. Kupfermann, I., & Weiss, K. R. (2001). Motor program selections in simple model systems. Current Opinion in Neuro-biology, 11, 673-677.
60. Lacalli, T. C. (1996). Frontal eye circuitry, rostral sensory pathways and brain organization in amphioxus larvae - evidence from 3D reconstructions. Philosophical Transactions of the Royal Society B: Biological Sciences, 351, 243-263.
61. Lingenhohl, K., & Friauf, E. (1994). Giant-neurons in the rat reticular-formation - a sensorimotor interface in the elementary acoustic startle circuit. Journal of Neuroscience, 14, 1176-1194.
62. Lovick, T. A. (1972). The behavioral repertoire of precollicular decerebrate rats. Journal of Physiology, 226, 4P - 6P.
63. Lukashin, A. V., Amirikian, B. R., Mozhaev, V. L., Wilcox, G. L., & Georgopoulos, A. P. (1996). Modeling motor cortical operations by an attractor network of stochastic neurons. Biological Cybernetics, 74, 255-261.
64. Mackie, G. O. (1990). The elementary nervous-system revisited. American Zoologist, 30, 907-920.
65. Mackie, G. O. (2004a). Central neural circuitry in the jellyfish Aglantha: A model 'simple nervous system'. Neurosignals, 13, 5-19.
66. Mackie, G. O. (2004b). Epithelial conduction: Recent findings, old questions, and where do we go from here ? Hydrobiologia, 530-531, 73-80.
67. Mackie, G. O., & Meech, R. W. (2000). Central neural circuitry in the jellyfish Aglantha digitale. III. The rootlet and pacemaker systems. Journal of Experimental Biology, 203, 1797-1807.
68. Maes, P. (1995). Modelling adaptive autonomous agents. In C. G. Langton (Ed.), Artificial life: An overview (pp. 135-162). Cambridge, MA: MIT Press.
69. Mallatt, J., & Chen, J. Y. (2003). Fossil sister group of craniates: Predicted and found. Journal of Morphology, 258, 1-31.
70. Mataric, M. J. (1990). A distributed model for mobile robot environment-learning and navigation (Masters thesis No. AITR-1228). Cambridge, MA: MIT AI Lab.
71. Mataric, M. J. (1997). Behavior-based control: Examples from navigation, learning, and group behavior. Experimental and Theoretical Artificial Intelligence, 9, 323-336.
72. McFarland, D. (1971). Feedback mechanisms in animal behaviour. London: Academic Press.
73. McFarland, D. (1989). Problems of animal behaviour. Harlow, UK: Longman.
74. McHaffie, J. G., Stanford, T. R., Stein, B. E., Coizet, V., & Redgrave, P. (2005). Subcortical loops through the basal ganglia. Trends in Neurosciences, 28, 401-407.
75. Medina, L., & Reiner, A. (1995). Neurotransmitter organization and connectivity of the basal ganglia in vertebrates - implications for the evolution of basal ganglia. Brain Behavior and Evolution, 46, 235-258.
76. Meech, R. W. (1989). The electrophysiology of swimming in the jellyfish Aglantha digitale. In P. A. V. Anderson (Ed.), Evolution of the first nervous systems (pp. 281-298). New York: Plenum Press.
77. Meech, R. W., & Mackie, G. O. (2001). Two escape swim synaptic inputs in the jellyfish Aglantha digitale. Journal of Physiology - London, 536, 132P - 132P.
78. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuro-science, 24, 167-202.
79. Mink, J. W., & Thach, W. T. (1993). Basal ganglia intrinsic circuits and their role in behavior. Current Opinion in Neuro-biology, 3, 950-957.
80. Montes González, F., Prescott, T. J., Gurney, K., Humphries, M., & Redgrave, P. (2000). An embodied model of action selection mechanisms in the vertebrate brain. In J. A. Meyer (Ed.), From animals to animats 6: Proceedings of the sixth International Conference on the Simulation of Adaptive Behavior (pp. 157-166). Cambridge, MA: MIT Press.
81. Nilsson, D. E., Gislen, L., Coates, M. M., Skogh, C., & Garm, A. (2005). Advanced optics in a jellyfish eye. Nature, 435, 201-205.
82. Northcutt, R. G. (1994). An immunohistochemical study of the telencephalon and the diencephalon in a myxinoid jawless fish, the pacific hagfish, Eptatretus stouti. Brain, Behavior, and Evolution, 43, 140-161.
83. Northcutt, R. G. (1996). The agnathan ark - the origin of craniate brains. Brain Behavior and Evolution, 48, 237-247.
84. Northcutt, R. G. (2002). Understanding vertebrate brain evolution. Integrative and Comparative Biology, 42, 743-756.
85. Northcutt, R. G. (2005). The new head hypothesis revisited. Journal of Experimental Zoology Part B: Molecular and Developmental Evolution, 304B, 274-297.
86. Prescott, T. J., Bryson, J. J., & Seth, A. (in press). Modelling natural action selection: an introduction to the theme issue. Philosophical Transactions of the Royal Society B: Biological Sciences.
87. Prescott, T. J., Gurney, K., & Redgrave, P. (2002). Basal Ganglia. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (2nd ed., pp. 147-151). Cambridge, MA: MIT Press.
88. Prescott, T. J., & Ibbotson, C. (1997). A robot trace-maker: Modeling the fossil evidence of early invertebrate behavior. Artificial Life, 3, 289-306.
89. Prescott, T. J., Montes González, F., Humphries, M. D., Gurney, K., & Redgrave, P. (2006). A robot model of the basal ganglia: Behaviour and intrinsic processing. Neural Networks, 19, 31-61.
90. Prescott, T. J., Redgrave, P., & Gurney, K. N. (1999). Layered control architectures in robots and vertebrates. Adaptive Behavior, 7, 99-127.
91. Raikova, O. I., Reuter, M., Gustafsson, M. K. S., Maule, A. G., Halton, D. W., & Jondelius, U. (2004). Evolution of the nervous system in Paraphanostoma (Acoela). Zoologica Scripta, 33, 71-88.
92. Raup, D. M., & Seilacher, A. (1969). Fossil foraging behaviour: computer simulation. Science, 166, 994-995.
93. Redgrave, P., Prescott, T., & Gurney, K. N. (1999). The basal ganglia: A vertebrate solution to the selection problem ? Neuroscience, 89, 1009-1023.
94. Renart, A., Parga, N., & Rolls, E. T. (1999). Backward projections in the cerebral cortex: implications for memory storage. Neural Computation, 11, 1349-1388.
95. Reuter, M. (1989). From innovation to integration: Trends of the integrative systems in microturbellerians. In M. K. S. Gustafsson & M. Reuter (Eds.), The early brain: Proceedings of the Symposium "Invertebrate Neurobiology" (pp. 161-178). Abo, Finland: Abo Academy Press.
96. Reuter, M., & Gustafsson, M. K. S. (1995). The flatworm nervous system: Pattern and phylogeny. In O. Breidbach & W. Kutsch (Eds.), The nervous systems of invertebrates: An evolutionary and comparative approach (pp. 25-59). Basel, Switzerland: Birkhauser Verlag.
97. Ridderinkhof, K. R., van den Wildenberg, W. P., Segalowitz, S. J., & Carter, C. S. (2004). Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain and Cognition, 56, 129-140.
98. Ringo, J. L. (1991). Neuronal interconnection as a function of brain size. Brain, Behaviour, and Evolution, 38, 1-6.
99. Romanes, G. J. (1885). Jellyfish, starfish, and sea urchins, being a research on primitive nervous systems. New York: D. Appleton.
100. Rosenblatt, J. K. (1997). DAMN: A distributed architecture for mobile navigation. Journal of Experimental and Theoretical Artificial Intelligence, 9, 339-360.
101. Rosenblatt, J. K. (2000). Optimal selection of uncertain actions by maximizing expected utility. Autonomous Robots, 9, 17-25.
102. Ruiz-Trillo, I., Riutort, M., Fourcade, H. M., Baguna, J., & Boore, J. L. (2004). Mitochondrial genome data support the basal position of Acoelomorpha and the polyphyly of the Platyhelminthes. Molecular Phylogenetics and Evolution, 33, 321-332.
103. Ruiz-Trillo, I., Riutort, M., Littlewood, D. T. J., Herniou, E. A., & Baguna, J. (1999). Acoel flatworms: Earliest extant bilaterian metazoans, not members of platyhelminthes. Science, 283, 1919-1923.
104. Rushworth, M. F., Buckley, M. J., Gough, P. M., Alexander, I. H., Kyriazis, D., McDonald, K. R., & Passingham, R. E. (2005). Attentional selection and action selection in the ventral and orbital prefrontal cortex. Journal of Neuro-science, 25, 11628-11636.
105. Saper, C. B., Scammell, T. E., & Lu, J. (2005). Hypothalamic regulation of sleep and circadian rhythms. Nature, 437, 1257-1263.
106. Satterlie, R. A. (2002). Neuronal control of swimming in jellyfish: a comparative story. Canadian Journal of Zoology, 80, 1654-1669.
107. Satterlie, R. A., & Nolen, T. G. (2001). Why do cubomedusae have only four swim pacemakers ? Journal of Experimental Biology, 204, 1413-1419.
108. Schall, J. D., Stuphorn, V., & Brown, J. W. (2002). Monitoring and control of action by the frontal lobes. Neuron, 36, 309-322.
109. Scheibel, A. B. (1984). The brainstem reticular core and sensory function. In J. M. Brookhart, & V. B. Mountcastle (Eds.), Handbook of physiology, section 1: The nervous system (pp. 213-256). Bethesda, MD: American Physiological Society.
110. Scheibel, M. E., & Scheibel, A. B. (1967). Anatomical basis of attention mechanisms in vertebrate brains. In G. C. Quarton, M. T. & F. O. Schmitt (Eds.), The neurosciences: A study program (pp. 577-602). New York: Rockefeller University Press.
111. Schmitt, F. O. (1984). Molecular regulators of brain function - a new view. Neuroscience, 13, 991-1001.
112. Shu, D.-G., Luo, H.-L., & Conway Morris, S. (1999). Lower Cambrian vertebrates from south China. Nature, 402, 42-46.
113. Snaith, S., & Holland, O. (1990). An investigation of two mediation strategies suitable for behavioural control in animals and animats. In J.-A. Meyer & S. Wilson (Eds.), From animals to animats: Proceedings of the first International Conference Simulation of Adaptive Behavior (pp. 255-262). Cambridge, MA: MIT Press.
114. Spencer, A. N., Przysiezniak, J., & Acosta-Urquidi, J. (1989). Presynaptic spike-broadening reduces junctional potential amplitude. Nature, 340, 636-638.
115. Utete, S. W., Barshan, B., & Ayrulu, B. (1999). Voting as validation in robot programming. International Journal of Robotics Research, 18, 401-413.
116. Valentine, J. W. (1994). Late precambrian bilaterians - grades and clades. Proceedings of the National Academy of Sciences of the United States of America, 91, 6751-6757.
117. Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of 'small-world' networks. Nature, 393, 440-442.
118. Webb, B., & Consi, T. R. (2001). Biorobotics. Cambridge, MA: MIT Press.
119. Weber, C., Wermter, S., & Elshaw, M. (2006). A hybrid generative and predictive model of the motor cortex. Neural Networks, 19, 339-353.
120. Windhorst, U. (1996). On the role of recurrent inhibitory feedback in motor control. Progress in Neurobiology, 49, 517-587.
121. Wishaw, I. Q. (1990). The decorticate rat. In B. Kolb & R. C. Tees (Eds.), The cerebral cortex of the rat (pp. 239-267). Cambridge, MA: MIT Press.
122. Woods, J. W. (1964). Behavior of chronic decerebrate rats. Journal of Neurophysiology, 27, 635-644.
123. Yavuz, H., & Bradshaw, A. (2002). A new conceptual approach to the design of hybrid control architecture for autonomous mobile robots. Journal of Intelligent and Robotic Systems, 34, 1-26.