Towards a cognitive robotics methodology for reward-based decision-making: Dynamical systems modelling of the Iowa Gambling task

Connection Science

Volumn 22, Issue 3, 2010, Pages 247-289

  Lowe, Robert ^a   Ziemke, Tom ^a  

^a UNIVERSITY OF SKÖVDE   (Sweden)

Author keywords

Dynamic field theory; Dynamical systems; Prefrontal cortex; Reversal learning; Reward prediction learning; Somatic markers

Indexed keywords

DYNAMIC FIELD THEORIES; PREFRONTAL CORTEX; REVERSAL LEARNING; REWARD PREDICTION LEARNING; SOMATIC MARKERS;

DECISION MAKING; DYNAMICAL SYSTEMS; ROBOTICS;

BEHAVIORAL RESEARCH;

EID: 77954890367     PISSN: 09540091     EISSN: 13600494     Source Type: Journal    
DOI: 10.1080/09540091.2010.489742     Document Type: Article

References (145)

1. Alexander, W.H., and Sporns, O. (2002), 'An Embodied Model of Learning, Plasticity, and Reward', Adaptive Behavior, 10, 143-159.
2. Amari, S. (1977), 'Dynamics of Pattern Formation in Lateral-inhibition Type Neural Fields', Biological Cybernetics, 27, 77-87.
3. Armony, J.L., Servan-Schreiber, D., Cohen, J.D., and LeDoux, J.E. (1995), 'AnAnatomically Constrained Neural Network Model of Fear Conditioning', Behavioral Neuroscience, 109, 246-257.
4. Armony, J.L., Servan-Schreiber, D., Romanski, L.M., Cohen, J.D., and LeDoux, J.E. (1997), 'Stimulus Generalization of Fear Responses: Effects of Auditory Cortex Lesions in a Computational Model and in Rats', Cerebral Cortex, 7, 157-165.
5. Ashby, W.R. (1960), Design for a Brain: The Origin of Adaptive Behaviour, Hatfield, UK: Chapman and Hall.
6. Avila-Garcìa, O. (2004), 'Towards Emotional Modulation of Action Selection in Motivated Autonomous Robots', Ph.D. thesis, University of Hertfordshire, Dept. of Computer Science.
7. Avila-Garcìa, O., and Cañamero, L. (2005), 'Hormonal Modulation of Perception in Motivation-based Action Selection Architectures', in Proceedings of Agents that Want and Like: Motivational and Emotional Roots of Cognition and Action, Symposium of the AISB'05 Convention, University of Hertfordshire, pp. 9-17.
8. Balkenius, C., and Morén, J. (2001), 'Emotional Learning: A Computational Model of the Amygdala', Cybernetics and Systems, 32, 611-636.
9. Barandiaran, X., and Moreno, A. (2006), 'On what Makes Certain Dynamical Systems Cognitive:A Minimally Cognitive Organization Program', Adaptive Behavior, 14, 171-185.
10. Barandiaran, X., and Moreno, A. (2008), 'Adaptivity: From Metabolism to Behavior', Adaptive Behavior, 16, 325-344.
11. Bechara, A. (2004), 'The Role of Emotion in Decision-making: Evidence from Neurological Patients with Orbitofrontal Damage', Brain and Cognition, 55, 30-40.
12. Bechara, A., and Damasio, A.R. (2005), 'The Somatic Marker Hypothesis: A Neural Theory of Economic Decision', Game and Economic Behaviour, 52, 336-372.
13. Bechara, A., Damasio, A.R., Damasio, H., and Anderson, S.W. (1994), 'Insensitivity to Future Consequences Following Damage to Human Prefrontal Cortex', Cognition, 50, 7-15.
14. Bechara, A., Damasio, H., Damasio, A.R., and Lee, G.P. (1999), 'Different Contributions of the Human Amygdala and the Ventromedial Prefrontal Cortex to Decision-making', Journal of Neuroscience, 19, 5473-5481.
15. Bechara, A., Damasio, H., Tranel, D., and Damasio, A.R. (1997), 'Deciding Advantageously Before Knowing the Advantageous Strategy', Science, 275, 1293-1295.
16. Bechara, A., Tranel, D., and Damasio, H. (2000), 'Characterization of the Decision-making Deficit of Patients with Ventromedial Prefrontal Cortex Lesions', Brain, 123, 2189-2202.
17. Bechara, A., Tranel, D., Damasio, H., and Damasio, A.R. (1996), 'Failure to RespondAutonomically to Anticipated Future Outcomes Following Damage to Prefrontal Cortex', Cerebral Cortex, 6, 215-225.
18. Beer, R.D. (1995), 'On the Dynamics of Small Continuous-time Recurrent Neural Networks', Adaptive Behavior, 3, 469-509.
19. Beer, R.D. (2000), 'Dynamical Approaches to Cognitive Science', Trends in Cognitive Sciences, 4, 91-99.
20. Berridge, K.C. (2003), 'Comparing the Emotional Brains of Humans and Other Animals', in Handbook of Affective Sciences, eds. R.J. Davidson, K.R. Scherer and H.H. Goldsmith, Oxford: Oxford University Press, pp. 25-52.
21. Bicho, E., and Schöner, G. (2000), 'Target Representation on an Autonomous Vehicle with Low-level Sensors', The International Journal of Robotics Research, 19, 424-447.
22. Bouton, M.E. (2007), Learning and Behavior: A Contemporary Synthesis, Sunderland, MA: Sinauer Associates.
23. Bowman, C.H., and Turnbull, O.H. (2004), 'Emotion-based Learning on a Simplified Card Game: The Iowa and Bangor Gambling Tasks', Brain and Cognition, 55, 277-282.
24. Braitenberg, V. (1984), Vehicles: Experiments in Synthetic Psychology, Cambridge, MA: MIT Press.
25. Buss, A.T., and Spencer, J.P. (2008), 'The Role of Space in Children's Rule-use', in Cognitive Development, Chair A.T. Buss, Paper Symposium Presented at the 30th Annual Meeting of the Cognitive Science Society,Washington, DC.
26. Camperi, M., andWang, X. (1998), 'A Model ofVisuospatialWorking Memory in Prefrontal Cortex: Recurrent Network and Cellular Bistability', Journal of Computational Neuroscience, 5, 383-405.
27. Colombetti, G. (2003), 'Complexity as a New Framework for Emotion Theories', Logic and Philosophy of Science, 1, 1-14.
28. Colombetti, G., and Thompson, E. (2005), 'Enacting Emotional Interpretations with Feelings', Behaviour and Brain Science, Open Discussion, 28, 200-201.
29. Colombetti, G., and Thompson, E. (2008), 'The Feeling Body: Towards an Enactive Approach to Emotion', in Developmental Perspectives on Embodiment and Consciousness, eds.W.F. Overton, U. Muller and J.L. Newman, NewYork: Lawrence Erlbaum Associates, pp. 45-68.
30. Craig, A.D. (2003), 'Interoception: The Sense of the Physiological Condition of the Body', Current Opinion in Neurobiology, 13, 500-505.
31. Craig, A.D. (2004), 'Human Feelings:WhyAre Some MoreAware than Others?' Trends in Cognitive Science, 8, 239-241.
32. Damasio, A.R. (1994), Descartes' Error: Emotion, Reason, and the Human Brain, NewYork: GP Putnam's Sons.
33. Damasio, A.R. (2003), Looking for Spinoza: Joy, Sorrow and the Feeling Brain, Orlando, FL: Harcourt.
34. Diamond, A. (2001), 'Looking Closely at Infants' Performance and Experimental Procedures in the A-not-B task', Behaviour and Brain Science, Open Discussion, 24, 38-41.
35. Diamond, A., Prevor, M.B., Callendar, G., and Druin, D.P. (1997), 'Prefrontal Cortex Cognitive Deficits in Children Treated Early and Continuously for PKU', Monographs of the Society for Research in Child Development, 62, 1-207.
36. Dineva, E.E. (2005), 'Dynamic Field Theory of Infant Reaching and its Dependence on Behavioral History and Context', Ph.D. thesis, Ruhr-Universität Bochum, Germany.
37. Dineva, E., Faubel, C., and Schöner, G. (2007), 'Dynamic Field Theory and Embodiment: Robotic Demonstration of Infant Perseveration', Paper Presented at the 14th International Conference on Perception and Action, Yokohama, Japan.
38. Dineva, E., Schöner, G., and Thelen, E. (2010), 'Behavioral History Matters: A Dynamic Field Theory of Spontaneous and Perseverative Errors in Infant Reaching', in preparation.
39. Di Paolo, E.A., and Iizuka, H. (2008), 'How (not) to Model Autonomous Behaviour', BioSystems, 91, 409-423.
40. Dunn, B.D., Dalgleish, T., and Lawrence, D. (2006), 'The Somatic Marker Hypothesis: A Critical Evaluation', Neuroscience and Biobehavioral Reviews, 30, 239-271.
41. Duran, B. (2009), 'Nonlinear Dynamical Systems and Applications to Robotics', Ph.D. thesis, Università Degli Studi Di Genova.
42. Duran, B., Lowe, R., and Ziemke, T. (2010), 'Towards an Extended Dynamic Field Theoretic Model of Iowa Gambling Task Performance', in IEEE 9th international Conference on Development and Learning, Accepted for publication.
43. Edelman, J. (1989), The Remembered Present, NewYork: Basic Books.
44. Ekman, P. (1999). 'Basic Emotions', in Handbook of Cognition and Emotion, eds. T. Dalgleish and M. Power, Sussex: Wiley.
45. Fellows, L.K., and Farah, M.J. (2003), 'Ventromedial Frontal Cortex Mediates Affective Shifting in Humans: Evidence from a Reversal Learning Paradigm', Brain, 126, 1830-1837.
46. Fellows, L.K., and Farah, M.J. (2005a), 'Different Underlying Impairments in Decision-making Following Ventromedial and Dorsolateral Frontal Lobe Damage in Humans', Cerebral Cortex, 15, 58-63.
47. Fellows, L.K., and Farah, M.J. (2005b), 'Dissociable Elements of Human Foresight:A Role for theVentromedial Frontal Lobes in Framing the Future, but not in Discounting Future Rewards', Neuropsychologia, 43, 1214-1221.
48. Freeman,W.J. (2000), How Brains Make Up Their Minds, NewYork: Columbia University Press.
49. Froese, T., and Ziemke, T. (2009), 'Enactive Artificial Intelligence', Artificial Intelligence, 173, 466-500.
50. Fuster, J.M. (2008), The Prefrontal Cortex (4th ed.), London: Elsevier.
51. Goldstein, K. (1934), The Organism, NewYork: Zone Books.
52. Harvey, I.P.H., Cliff, D., Thompson, A., and Jakobi, N. (1994), 'Evolutionary Robotics: The Sussex Approach', Robotics and Autonomous Systems, 20, 207-224.
53. Hatfield, T., Han, J.S., Conley, M., Gallagher, M., and Holland, P. (1996), 'Neurotoxic Lesions of Basolateral, But not Central,Amygdala Interfere with Pavlovian Second-order Conditioning and Reinforcer Devaluation Effects', Journal of Neuroscience, 16, 5256-5265.
54. Hesslow, G. (2002), 'Conscious Thought as Simulation of Behaviour and Perception', Trends in Cognitive Sciences, 6, 242-247.
55. Hooper, C.J., Luciana, M., Conklin, H.M., and Yarger, R.S. (2004), 'Adolescents Performance on the Iowa Gambling Task: Implications for the Development of Decision making and Ventromedial Prefrontal Cortex', Developmental Psychology, 40, 1148-1158.
56. Iizuka, H., and Di Paolo, E.A. (2007), 'Toward Spinozist Robotics: Exploring the Minimal Dynamics of Behavioural Preference', Adaptive Behaviour, 15, 359-376.
57. Jaeger, H. (2001), 'Short Term Memory in Echo State Networks', GMD Report 152.
58. Jeannerod, M. (2001), 'Neural Simulation of Action: A Unifying Mechanism for Motor Cognition', NeuroImage, 14, S103-S109.
59. Johnson, J.S., Spencer, J.P., and Schöner, G. (2008), 'Moving to Higher Ground: The Dynamic Field Theory and the Dynamics of Visual Cognition', New Ideas in Psychology, 26, 227-251.
60. Kelley, A.E. (2005), 'Neurochemical Networks Encoding Emotion and Motivation:An Evolutionary Perspective', in Who Needs Emotions? The Brain Meets the Robot, eds. J.-M. Fellous and M.A.Arbib, NewYork: Oxford University Press.
61. Kelso, J. (1995), Dynamic Patterns: The Self-organization of Brain and Behavior, Cambridge, MA: MIT Press.
62. Kilcross, S., and Coutreau, E. (2003), 'Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats', Cerebral Cortex, 13, 400-408.
63. Kroener, S., Chandler, L.J., Phillips, P.E.M., and Seamans, J.K. (2009), 'Dopamine Modulates Persistent Synaptic Activity and Enhances the Signal-to-Noise Ratio in the Prefrontal Cortex', PLoS ONE, 4, e6507 (14 pages).
64. LeDoux, J.E. (1990), 'Information Flow from Sensation to Emotion: Plasticity in the Neural Computation of Stimulus Value', in Learning and Computational Neuroscience: Foundations of Adaptive Networks, eds. M. Gabriel and J. Moore, Cambridge, MA: MIT Press, pp. 3-51.
65. LeDoux, J.E. (1996), The Emotional Brain, NewYork: Simon & Schuster.
66. Levenson, R.W. (2003), 'Autonomic Specificity and Emotion', in Handbook of Affective Sciences, eds. R.J. Davidson, K.R. Scherer and H.H. Goldsmith, NewYork: Oxford University Press, pp. 212-224.
67. Lewis, M. (2005), 'Bridging Emotion Theory and Neurobiology through Dynamic Systems Modeling', Behavioral and Brain Sciences, 28, 169-194.
68. Lin, C., Chiu, Y., Cheng, C., and Hsieh, J. (2008), 'Brain Maps of Iowa Gambling Task', BMC Neuroscience, 9, 72 (15 pages).
69. Lowe, R. (2007), 'The Evolution of Affective Displays and Related Strategies', Ph.D. thesis, University of Hertfordshire.
70. Lowe, R., Herrera, C., Morse, T., and Ziemke T. (2007), 'The Embodied Dynamics of Emotion, Appraisal and Attention', in Attention in Cognitive Systems. Theories and Systems from an Interdisciplinary Viewpoint, eds. L. Paletta and E. Rome, Berlin: Springer.
71. Lowe, R., Humphries, M., and Ziemke, T. (2009a), 'The Dual-route Hypothesis: Evaluating a Neurocomputational Model of Fear Conditioning in Rats', Connection Science, 21, 15-37.
72. Lowe, R., Mannella, F., Ziemke, T., and Baldassarre, G. (2009b), 'Modelling Coordination of Learning Systems: A Reservoir Systems Approach to Dopamine Modulated Pavlovian Conditioning', in The European Conference of Artificial Life, eds. G. Kampis and E. Szathmary, Berlin: Springer.
73. Lowe, R., Philippe, P., Montebelli, A., Morse, A., and Ziemke, T. (2008), 'Affective Modulation of Embodied Dynamics', in The Role of Emotion in Adaptive Behaviour and Cognitive Robotics, Electronic Proceedings of SAB Workshop, Osaka, Japan, http://www.his.se/icea/emotion-workshop/.
74. Maia, T.V., and McClelland, J. L. (2004), 'A Reexamination of the Evidence for the Somatic Marker Hypothesis: What Participants Really Know in the Iowa Gambling Task', The National Academy of Sciences of the USA, 101, 16,075-16,080.
75. Maia, T.V., and McClelland, J.L. (2005), 'The Somatic Marker Hypothesis: Still Many Questions But No Answers', Trends in Cognitive Science, 9, 162-164.
76. Malone, S.M. (1994), 'The Prefrontal Cortex and its Functions: Toward a More Developmental Neuroscience', Unpublished manuscript, University of Minnesota.
77. Mannella, F., Zappacosta, S., Mirolli, M., and Baldassarre, G. (2008), 'A Computational Model of the Amygdala Nuclei's Role in Second Order Conditioning', in Proceedings of the Tenth International Conference on Simulation of Adaptive Behavior, pp. 321-330.
78. Marcovitch, S., and Zelazo, P.D. (1999), 'The A-not-B error: Results from a Logistic Meta-analysis', Child Development, 70, 1297-313.
79. Melhuish, C., Ieropolous, I., Greenman, J., and Horsfield, I. (2006), 'EnergeticallyAutonomous Robots: Food for Thought', Autonomous Robot, 21, 187-198.
80. Mirolli, M., Mannella, F., and Baldassarre, G. (2010), 'The Roles of Amygdala in the Affective Regulation of Body, Brain and Behaviour', Connection Science, 22, 215-245.
81. Mobini, S., Body, S., Ho, M-Y., Bradshaw, C.M., Szabadi, E., Deakin, J.F.W., and Anderson, I.M. (2002), 'Effects of Lesions of the Orbitofrontal Cortex on Sensitivity to Delayed and Probabilistic Reinforcement', Psychopharmacology, 160, 290-298.
82. Moioli, R.C., Vargas, P.A., Von Zuben, F.J., and Husbands, P. (2008), 'Towards the Evolution of an Artificial Homeostatic System', in IEEE Congress on Evolutionary Computation (CEC 2008), Berlin: Springer, pp. 4023-4030.
83. Montebelli, A., Lowe, R., and Ziemke, T. (2009), 'The Cognitive Body: From Dynamic Modulation to Anticipation', in Anticipatory Behavior in Adaptive Learning Systems: from Sensorimotor to Higher-level Cognitive Capabilities, Berlin: Springer.
84. Moreno, A., Etxeberria, A., and Umerez, J. (2008), 'The Autonomy of Biological Individuals and Artificial Models', BioSystems, 91, 309-319.
85. Morén, J. (2002), 'Learning and Emotion', Ph.D. thesis, Lund University.
86. Morén, J., and Balkenius, C. (2000), 'A Computational Model of Emotional Learning in the Amygdala', in Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, Cambridge, MA: MIT Press, pp. 383-391.
87. Naqvi, N., Tranel, D., and Bechara, A. (2006), 'Visceral and Decision-making Functions of the Ventromedial Prefrontal Cortex', in The Orbitofrontal Cortex, eds. D.H. Zald and S.L. Rausch, New York: Oxford University Press, pp. 325-354.
88. Nolfi, S., and Floreano, D. (2000), Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-organizing Machines, Cambridge, MA: MIT Press/Bradford Books.
89. Overman, W.H., Frassrand, K., Ansel, S., Trawalter, S., Bies, B., and Redmond, A. (2004), 'Performance on the IOWA Card Task by Adolescents and Adults', Neuropsychologia, 42, 1838-1851.
90. Panksepp, J. (1998), Affective Neuroscience, Oxford: Oxford University Press.
91. Panksepp, J. (2000), 'The Neurodynamics of Emotions: An Evolutionary NeurodevelopmentalView', in Emotion, Development, and Self-organization: Dynamic Systems Approaches to Emotional Development, eds. M.D. Lewis and I. Granic, NewYork: Cambridge University Press.
92. Parisi, D. (2004), 'Internal Robotics', Connection Science, 16, 325-338.
93. Piaget, J. (1953), The Origins of Intelligence in the Child, London: Routledge.
94. Porr, B., andWörgötter, F. (2003a), 'Isotropic Sequence Order Learning', Neural Computation, 15, 831-864.
95. Porr, B., and Wörgötter, F. (2003b), 'Isotropic Sequence Order Learning in a Closed Loop Behavioural System', Proceedings of the Royal Society Philosophical Transactions Mathematical, Physical & Engineering Sciences, 361, 2225-2244.
96. Rescorla, R.A. (1980), Pavlovian Second-order Conditioning, NewYork: Erlbaum.
97. Rescorla, R.A., and Wagner, A.W. (1972), 'A Theory of Pavlovian Conditioning: Variations in the Effectiveness of Reinforcement and Nonreinforcement', in Classical Conditioning II: Current Research and Theory, eds. A.H. Black andW.F. Prokasy, NewYork: Appleton-Century-Crofts, pp. 64-99.
98. Roesch, M.R., and Schoenbaum, G. (2006), 'FromAssociations to Expectancies: Orbitofrontal Cortex as Gateway between the Limbic System and Representational Memory', in The Orbitofrontal Cortex, eds. D.H. Zald, and S.L. Rausch, NewYork: Oxford University Press, pp. 199-236.
99. Rolls, E.T. (1996), 'The Orbitofrontal Cortex', Philosophical Transactions of the Royal Society of London (Series B), 351, 1433-1444.
100. Rolls, E.T. (1999), The Brain and Emotion, Oxford: Oxford University Press.
101. Rolls, E.T. (2005a), Emotion Explained, Oxford: Oxford University Press.
102. Rolls, E.T. (2005b), 'What Are Emotions, Why Do We Have Emotions, and what is Their Computational Basis in the Brain?' in Who Needs Emotions? The Brain Meets the Robot, eds. J.-A. Fellous and M.A. Arbib, Oxford: Oxford University Press, pp. 117-146.
103. Rolls E.T., Hornak J., Wade, D., and McGrath, J. (1994), 'Emotion-related Learning in Patients with Social and Emotional Changes Associated with Frontal Lobe Damage', Journal of Neurology, Neurosurgery, and Psychiatry, 57, 1518-1524.
104. Saddoris, M.P., and Gallagher, M. (2005), 'Rapid Associative Encoding in BasolateralAmygdala Depends on Connections with Orbitofrontal Cortex', Neuron, 46, 321-331.
105. Saglimbeni, F., and Parisi, D. (2009), 'Input from the External Environment and Input from Within the Body', in The European Conference of Artificial Life, eds. G. Kampis and E. Szathmary, Berlin: Springer.
106. Scherer, K.R. (2000), 'Emotions as Episodes of Subsytem Synchronization Driven by Non-linear Appraisal Processes', in Emotion, Development, and Self-organization, eds. M.D. Lewis and I. Granic, Cambridge: Cambridge University Press, pp. 70-99.
107. Scherer, K.R. (2001), 'The Nature and Study of Appraisal: A Review of the Issues', in Appraisal Processes in Emotion: Theory, Methods, Research, eds. K.R. Scherer, A. Schorr and T. Johnstone, Oxford: Oxford University Press, pp. 369-393.
108. Schoenbaum, G., Saddoris, M.P., and Stalnaker, T.A. (2007), 'Reconciling the Roles of Orbitofrontal Cortex in Reversal Learning and the Encoding of Outcome Expectancies', Annals of the NewYork Academy of Science, 1121, 320-335.
109. Schoenbaum, G., Setlow, B., Saddoris, M.P., and Gallagher, M. (2003a), 'Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex During Cue Sampling Depends upon Input from Basolateral Amygdala', Neuron, 39, 855-867.
110. Schoenbaum, G., Setlow, B., Nugent, L., Saddoris, M.P., and Gallagher, M. (2003b), 'Lesions of Orbitofrontal Cortex and Basolateral Amygdala Complex Disrupt Acquisition of Odor-guided Discriminations and Reversals', Learning Memory, 10, 129-140.
111. Schöner, G., and Dineva, E. (2007), 'Dynamic Instabilities as Mechanisms for Emergence', Developmental Science, 10, 69-74.
112. Schultz, W. (1998), 'Predictive Reward Signal of Dopamine Neurons', Journal of Neurophysiology, 80, 1-27.
113. Schultz, W. (2007), 'Multiple Dopamine Functions at DifferentTime Courses', Annual Reviews Neuroscience, 30, 259-88.
114. Schutte, A.R., Spencer, J.P., and Schöner, G. (2003), 'Testing the Dynamic Field Theory:Working Memory for Locations Becomes more Spatially Precise over Development', Child Development, 74, 1393-1417.
115. Simmering, V.R., Schutte, A.R., and Spencer, J.P. (2008), 'Generalizing the Dynamic Field Theory of Spatial Cognition across Real and Developmental Time Scales', Brain Research, 1202, 68-86.
116. Smith, L.B., Spencer, J., and Thelen, E. (1999a), 'General Processes of Goal-directed Reaching make the A-not-B error', Presented at the Biennial Meeting, Society for Research in Child Development, Albuquerque.
117. Smith, L.B., Thelen, E., Titzer, R., and McLin, D. (1999b), Knowing in the Context of Acting: The Task Dynamics of the A-not-B error', Psychological Review, 106, 235-260.
118. Spencer, J.P., Perone, S., and Johnson, J.S. (2009), 'Dynamic Field Theory and Embodied Cognitive Dynamics', in Toward A Unified Theory Of Development, eds. J.P. Spencer, M.S.C. Thomas and J.L. McClelland, Oxford: Oxford University Press, pp. 86-118.
119. Sporns, O., and Alexander, W.H. (2002), 'Neuromodulation and Plasticity in an Autonomous Robot', Neural Networks, 15, 761-774.
120. Sporns, O., and Alexander, W.H. (2003), 'Neuromodulation in a Learning Robot: Interactions between Neural Plasticity and Behavior', Proceedings IJCNN, 4, 2789-2794.
121. Suri, R.E. (2002), 'TD Models of Reward Predictive Responses in Dopamine Neurons', Neural Networks, 15, 523-533.
122. Sutton, R., and Barto, A. (1981), 'Towards a Modern Theory of Adaptive Networks: Expectation and Prediction', Psychological Review, 88, 135-170.
123. Sutton, R.S., and Barto, A.G. (1990), 'Time-derivative Models of Pavlovian Reinforcement', in Learning and Computational Neuroscience: Foundation of Adaptive Networks, eds. M. Gabriel and J. Moore, Cambridge, MA: MIT Press.
124. Tani, J. (2003), 'Learning to Generate Articulated Behaviour through the Bottom-up and the Top-down Interaction Processes', Neural Networks, 16, 11-23.
125. Tani, J., and Nolfi, S. (1999), 'Learning to Perceive theWorld as Articulated: An Approach for Hierarchical Learning in Sensory-motor Systems', Neural Networks, 12, 1131-1141.
126. Thelen, E., Schöner, G., Scheier, C., and Smith, L. (2001), 'The Dynamics of Embodiment: A Dynamic Field Theory of Infant Perseverative Reaching', Behavioural and Brain Sciences, 24, 1-86.
127. Thelen, E., and Smith, L.B. (1994), A Dynamic Systems Approach to the Development of Cognition and Action, Cambridge, MA: MIT Press.
128. Thompson, E. (2007), Mind in Life, Cambridge, MA: Harvard University Press.
129. Tranel, D., Bechara, A., and Damasio, A.R. (1999), 'Decision Making and the Somatic Marker Hypothesis', in The New Cognitive Neurosciences, ed. M.S. Gazzaniga, Cambridge, MA: MIT Press, pp. 1047-1061.
130. Van den Bos, R., Houx, B.B., and Spruijt, B (2005), 'The Effect of Reward Magnitude Differences on Choosing Disadvantageous Decks in the Iowa Gambling Task', Biological Psychology, 71, 155-161.
131. Van den Bos, R., Lasthuis, W., den Heijer, E., Van der Harst, J., and Spruijt, B. (2006), 'Toward a Rodent Model of the Iowa Gambling Task', Behavior Research Methods, 38, 470-478.
132. Vijayraghavan, S., Wang, M., Birnbaum, S.G., Williams, G.V., and Arnsten, A.F. (2007), 'Inverted-U Dopamine D1 Receptor Actions on Prefrontal Neurons Engaged inWorking Memory', Nature Neuroscience, 10, 376-384.
133. Wallis, J.D. (2007), 'Orbitofrontal Cortex and its Contribution to Decision-making', Annual Reviews Neuroscience, 30, 31-56.
134. Wang, X. (2006), 'A Microcircuit Model of Prefrontal Functions: Ying and Yang of Reverberatory Neurodynamics in Cognition', in The Prefrontal Lobes: Development, Function and Pathology, eds. J. Risberg and J. Grafman, Cambridge: Cambridge University Press, pp. 92-127.
135. Wellman, H.M., Cross, D., and Bartsch, K. (1986), 'Infant Search and Object Permanence:AMeta-analysis of the A-not-B Error', Monographs of the Society for Research in Child Development, 51, 1-67.
136. Wellman, H.M., Cross, D., and Bartsch, K. (1987), 'A Meta-analysis of Research on Stage 4 Object Permanence: The A-not-B Error', Monographs of the Society for Research in Child Development, 51(214).
137. Wilson, M. (2002), 'Six Views of Embodied Cognition', Psychonomic Bulletin and Review, 9, 625-636.
138. Winstanley, C.A., Theobald, D.E.H., Cardinal, R.N., and Robbins, T.W. (2004), 'Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice', Journal of Neuroscience, 24, 4718-4722.
139. Wood, R., and Di Paolo, E.A. (2007), 'New Models for Old Questions: Evolutionary Robotics and the 'A not B' error', Proceedings of 9th European Conference on Artificial Life, pp. 1141-1150.
140. Wörgötter, F., and Porr, B. (2005), 'Temporal Sequence Learning, Prediction, and Control -A Review of Different Models and their Relation to Biological Mechanisms', Neural Computation, 17, 245-319.
141. Zald, D.H. (2007), 'Orbital Versus Dorsolateral Prefrontal Cortex: Anatomical Insights into Content versus Process Differentiation Models of the Prefrontal Cortex', Annals of the New York Academy of Sciences, 1121, 395-406.
142. Zeeb, F.D., Robbins, T.W., and Winstanley, C.A. (2009), 'Serotonergic and Dopaminergic Modulation of Gambling Behavior as Assessed Using a Novel Rat Gambling Task', Neuropsychopharmacology, 34, 2329-2343.
143. Zelazo, P.D., Müller, U., Frye, D., and Marcovitch, S. (2003), The Development of Executive Function in Early Childhood, Monographs of the Society for Research in Child Development, 68(3), 1-155.
144. Ziemke, T. (2003), 'What's that Thing Called Embodiment?'in Proceedings of the 25th Annual Conference of the Cognitive Science Society, eds. R. Alterman and D. Kirsh, Mahwah, NJ: Lawrence Erlbaum, pp. 1305-1310.
145. Ziemke, T., and Lowe, R. (2009), 'On the Role of Emotion in Embodied Cognitive Architectures: From Organisms to Robots', Cognitive Computation, 1, 104-117.