The interplay of Pavlovian and instrumental processes in devaluation experiments: A computational embodied neuroscience model tested with a simulated rat

Modelling Perception with Artificial Neural Networks

Volumn , Issue , 2010, Pages 93-113

  Mannella, Francesco ^a   Mirolli, Marco ^a   Baldassarre, Gianluca ^a  

^a INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 78249234248     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511779145.006     Document Type: Chapter

References (56)

1. Armony, J. L., Servan-Schreiber, D., Romanski, L. M. & LeDoux, D. J. J. E. 1997. Stimulus generalization of fear responses: effects of auditorycortexlesions in a computational model and in rats. Cereb Cortex 7(2), 157–165.
2. Baldassarre, G. 2008. Self-organization as phase transition in decentralized groups of robots: a study based on Boltzmann entropy. In Advances in Applied Self-Organizing Systems (ed. M. Prokopenko), pp. 127–146. Springer-Verlag.
3. Balleine, B. W., Killcross, A. S. & Dickinson, A. 2003. The effect of lesions of the basolateral amygdala on instrumental conditioning. J Neurosci 23(2), 666–675.
4. Balleine, B. W. & Killcross, S. 2006. Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci 29(5), 272–279.
5. Barto, A., Singh, S. & Chentanez, N. 2004. Intrinsically motivated learning of hierarchical collections of skills. In International Conference on Developmental Learning (ICDL), LaJolla, CA.
6. Baxter, M. G. & Murray, E. A. 2002a. The amygdala and reward. Nat Rev Neurosci 3(7), 563–573.
7. Baxter, M. G. & Murray, E. A. 2002b. The amygdala and reward. Nature Rev Neurosci 3(7), 563–573.
8. Blair, H. T., Sotres-Bayon, F., Moita, M. A. P. & LeDoux, J. E. 2005. The lateral amygdala processes the value of conditioned and unconditioned aversive stimuli. Neuroscience 133(2), 561–569.
9. Brody, C., Pouget, A., Shadlen, M. & Zador, A. (Eds.) 2004. Abstracts of Papers Presented at the 2004 Meeting on Computational & System Neuroscience. Cold Spring Harbor Laboratory.
10. Camazine, S., Deneubourg, J. L., Franks, N. R., Sneyd, J., Theraulaz, G. & Bonabeau, E. (Ed.) 2001. Self-organization in Biological Systems. Princeton University Press.
11. Cardinal, R. N., Parkinson, J. A., Hall, J. & Everitt, B. J. 2002. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26(3), 321–352.
12. Cisek, P. 2007. Cortical mechanisms of action selection: the affordance competition hypothesis. Phil Trans R Soc B 362(1485), 1585–1599.
13. Clark, A. 1997. Being There: Putting Brain, Body and World Together Again. MIT Press.
14. Darwin, C. 1859. The Origin of Species. Retrieved from http://www.literature.org/authors/darwincharles/the-origin-of-species/index.html.
15. Dayan, P. & Balleine, B. 2002. Reward, motivation and reinforcement learning. Neuron 36, 285–298.
16. Domjan, M. 2006. Principles of Learning and Behaviour. Thomson Wadsworth.
17. Haber, S. N., Fudge, J. L. & McFarland, N. R. 2000. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20(6), 2369–2382.
18. Holland, O. & McFarland, D. 2001. Artificial Ethology. Oxford University Press.
19. Houk, J. C., Adams, J. L. & Andrew, G. B. 1995. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In Models of Information Processing in the Basal Ganglia (ed. J. C. Houk, J. L. Davids & D. G. Beiser), pp. 249–270. MIT Press.
20. Knight, D. C., Nguyen, H. T. & Bandettini, P. A. 2005. The role of the human amygdala in the production of conditioned fear responses. Neuroimage 26(4), 1193–1200.
21. Kobayashi, Y. & Okada, K.-I. 2007. Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann N Y Acad Sci 1104, 310–323.
22. Langton, C. (Ed.) 1987. The First International Conference on the Simulation and Synthesis of Living Systems (ALifeI).
23. Lieberman, D. A. 1993. Behavior and Cognition. Brooks/Cole.
24. Mannella, F., Mirolli, M. & Baldassarre, G. 2007. The role of amygdala in devaluation: a model tested with a simulated robot. In Proceedings of the Seventh International Conference on Epigenetic Robotics (ed. L. Berthouze, C. G. Prince, M. Littman, H. Kozima & C. Balkenius), pp. 77–84. University of Lund.
25. Mannella, F., Zappacosta, S. & Baldassarre, G. 2008. A computational model of the amygdala nuclei’s role in second order conditioning. In Proceedings of the Tenth International Conference on Simulation of Adaptive Behavior: From Amimals to Animals 10 (ed. M. A. J. Tani, J. Hallam & J.-A. Meyer). Springer-Verlag.
26. Maren, S. 2005. Building and burying fear memories in the brain. Neuroscientist 11(1), 89–99.
27. McDonald, A. J. 1998. Cortical pathways to the mammalian amygdala. Prog Neurobiol 55(3), 257–332.
28. Meyer, J.-A. & Wilson, S. W. (Ed.) 1991. From Animals to Animats 1: Proceedings of the First International Conference on Simulation of Adaptive Behaviour. MIT Press.
29. Mogenson, G. J., Jones, D. L. & Yim, C. Y. 1980. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14(2–3), 69–97.
30. Morén, J. & Balkenius, C. 2000. A computational model of emotional learning in the amygdala. In From Animals to Animats 6: Proceedings of the 6th International Conference on the Simulation of Adaptive Behaviour (ed. J.-A. Meyer, A. Berthoz, D. Floreano, H. L. Roitblat & S. W. Wilson). MIT Press.
31. Newell, A. 1973. You can’t play 20 questions with nature and win: projective comments on the papers of this symposium. In Visual Information Processing (ed. W. G. Chase), pp. 283–308. Academic Press.
32. Niv, Y., Daw, N. D., Joel, D. & Dayan, P. 2007. Tonic dopamine: opportunity costs and the control of response vigor. J Psychopharmacol 191(3), 507–520.
33. Nolfi, S. 2006. Behaviour as a complex adaptive system: on the role of self-organization in the development of individual and collective behaviour. ComplexUs 2(3–4), 195–203.
34. Nolfi, S. & Floreano, D. 2000. Evolutionary Robotics: The Biology, Intelligence, and Technology. MIT Press.
35. O’Reilly, R., Frank, M., Hazy, T. & Watz, B. 2007. PVLV: The primary value and learned value pavlovian learning algorithm. Behav Neurosci 121, 31–49.
36. Packard, M. G. & Knowlton, B. J. 2002. Learning and memory functions of the basal ganglia. Annu Rev Neurosci 25, 563–593.
37. Parisi, D. 2004. Internal robotics. Connection Sci 16(4), 325–338.
38. Parisi, D., Cecconi, F. & Nolfi, S. 1990. Econets: Neural networks that learn in an environment. Network 1, 149–168.
39. Pavlov, I. P. 1927. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press.
40. Pitkänen, A., Jolkkonen, E. & Kemppainen, S. 2000. Anatomic heterogeneity of the rat amygdaloid complex. Folia Morphol. (Warsz) 59(1), 1–23.
41. Prescott, T. J., Gonzalez, F. M., Humphries, M. & Gurney, K. 2003. Towards a methodology for embodied computational neuroscience. In Proceedings of the Symposium on Scientific Methods for the Analysis of Agent-Environment Interaction (AISB2003). AISB Press.
42. Prescott, T. J., Gonzalez, F. M. M., Gurney, K., Humphries, M. D. & Redgrave, P. 2006. A robot model of the basal ganglia: behavior and intrinsic processing. Neural Netw 19(1), 31–61.
43. Price, J. L. 2003. Comparative aspects of amygdala connectivity. Ann N Y Acad Sci 985(1), 50–58.
44. Redgrave, P., Prescott, T. J. & Gurney, K. 1999. The basal ganglia: a vertebrate solution to the selection problem? J Neurosci 89(4), 1009–1023.
45. Rolls, E. T. 2005. Taste and related systems in primates including humans. Chem Senses 30 Suppl. 1, i76-i77.
46. Schembri, M., Mirolli, M. & Baldassarre, G. 2007. Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In Proceedings of the 6th International Conference on Development and Learning (ICDL) (ed. Y. Demiris, D. Mareschal, B. Scassellati & J. Weng), pp. E1–6. Imperial College London.
47. Schultz, W. 2002. Getting formal with dopamine and reward. Neuron 36, 241–263.
48. Sejnowski, T. J., Koch, C. & Churchland, P. S. 1988. Computational neuroscience. Science 241(4871), 1299–1306.
49. Shi, C. & Davis, M. 1999. Pain pathways involved in fear conditioning measured with fear potentiated startle: lesion studies. J Neurosci 19(1), 420–430.
50. Skinner, B. F. 1938. The Behavior of Organisms. Appleton-Century-Crofts.
51. Sutton, R. S. & Barto, A. G. 1981. Toward a modern theory of adaptive networks: Expectation and prediction. Psychol Rev 88, 135–140.
52. Sutton, R. & Barto, A. 1998. Reinforcement Learning: An Introduction. MIT Press.
53. Thorndike, E. L. 1911. Animal Intelligence. Transaction Publishers.
54. Weng, J., McClelland, J., Pentland, A. et al. 2001. Autonomous mental development by robots and animals. Science 291, 599–600.
55. Yin, H. H. & Knowlton, B. J. 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci 7, 464–476.
56. Zlatev, J. & Balkenius, C. 2001. Introduction: Why epigenetic robotics? In Proceedings of the First International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems (ed. C. Balkenius, J. Zlatev, H. Kozima, K. Dautenhahn & C. Breazeal), University of Lund. pp. 1–4.