Integrating what & how/where with instrumental and Pavlovian learning: A biologically based computational model

Cognition and Neuropsychology: International Perspectives on Psychological Science (Volume 1)

Volumn , Issue , 2010, Pages 71-93

  Pauli, Wolfgang M ^a   Atallah, Hisham E ^b   O'Reilly, R C ^a  

^a UNIVERSITY OF COLORADO   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84555205464     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (78)

1. Ahn, S., & Phillips, A. G. (2003). Independent modulation of basal and feeding- evoked dopamine efflux in the nucleus accumbens and medial prefrontal cortex by the central and basolateral amygdalar nuclei in the rat. Neuroscience, 116, 295-305.
2. Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9, 357-381.
3. Atallah, H. E., Frank, M. J., & O'Reilly, R. C. (2004). Hippocampus, cortex, and basal ganglia: Insights from computational models of complementary learning systems. Neurobiology of Learning and Memory, 82(3), 253-267.
4. Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W, & O'Reilly, R. C. (2006). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience, 10, 126-131.
5. Balleine, B. W., & Ostlund, S. B. (2007). Still at the choice-point: Action selection and initiation in instrumental conditioning. Annals of the New York Academy of Sciences, 1104, 147-171.
6. Barto, A. G., Sutton, R. S., & Anderson, C. W (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, & Cybernetics, 13(5), 834-846.
7. Berns, G. S., & Sejnowski, T. J. (1998). A computational model of how the basal ganglia produces sequences. Journal of Cognitive Neuroscience, 10, 108-121.
8. Brown, J. W, Bullock, D., & Grossberg, S. (2004). How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Networks, 17, 471-510.
9. Cardinal, R. N., Parkinson, J. A., Lachenal, G., Halkerston, K. M., Rudarakanchana, N., Hall, J., et al. (2002). Effects of selective excito toxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behavioral Neuroscience, 116, 553-567.
10. Cheng, K., Saleem, K. S., & Tanaka, K. (1997). Organization of corticostriatal and corticoamygdalar projections arising from the anterior inferotemporal area TE of the macaque monkey: A phaseolus vulgaris leucoagglutinin study. Journal of Neuroscience, 17(20), 7902-7925.
11. Cisek, P. (2007). Cortical mechanisms of action selection: The affordance competition hypothesis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 362(1485), 1585-1599.
12. Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews. Neuroscience, 3, 201-215.
13. Corbit, L. H., & Balleine, B. W (2005). Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian- instrumental transfer. Journal of Neuroscience, 25(4), 962-970.
14. Dayan, P., Niv, Y, Seymour, B., & Daw, N. D. (2006). The misbehavior of value and the discipline of the will. Neural Networks, 19(8), 1153-1160.
15. Dominey, P., Arbib, M., & Joseph, J.-P. (1995). A model of corticostriatal plasticity for learning oculomotor associations and sequences. Journal of Cognitive Neuroscience, 7, 311-336.
16. Featherstone, R. E., & McDonald, R. J. (2004). Dorsal striatum and stimulusresponse learning: Lesions of the dorsolateral, but not dorsomedial, striatum impair acquisition of a simple discrimination task. Behavioural Brain Research, 150(1-2), 15-23.
17. Fiehler, K., Burke, M., Bien, S., Röder, B., & Rösler, F. (2008). The human dorsal action control system develops in the absence of vision. Cerebral Cortex, 19(1), 1-12.
18. Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003). Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nature Neuroscience, 6, 968-973.
19. Floresco, S. B., Yang, C. R., Phillips, A. G., & Blaha, C. D. (1998). Basolateral amygdala stimulation evokes glutamate receptor-dependent dopamine efflux in the nucleus accumbens of the anaesthetized rat. European Journal of Neuroscience, 10(4), 1241-1251.
20. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: A neuro- computational account of cognitive deficits in medicated and non-medicated parkinsonism. Journal of Cognitive Neuroscience, 17, 51-72.
21. Frank, M. J. (2006). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making. Neural Networks, 19, 1120-1136.
22. Frank, M. J., Samanta, J., Moustafa, A. A., & Sherman, S. J. (2007). Hold your horses: Impulsivity, deep brain stimulation, and medication in parkinsonism. Science, 318, 1309-1312.
23. Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306, 1940-1943.
24. Fudge, J. L., & Haber, S. N. (2000). The central nucleus of the amygdala projection to dopamine subpopulations in primates. Neuroscience, 97, 479-494.
25. Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15(1), 20-25.
26. Graybiel, A. M. (1998). The basal ganglia and chunking of action repertoires. Neurobiology of Learning and Memory, 70(1-2), 119-136.
27. Gurney, K., Prescott, T. J., & Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84, 401-410.
28. Han, J. S., McMahan, R. W, Holland, P., & Gallagher, M. (1997). The role of an amygdalo-nigrostriatal pathway in associative learning. Journal of Neuroscience, 17(10), 3913-3919.
29. Hatfield, T., Han, J. S., Conley, M., & Holland, P. (1996). Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. Journal of Neuroscience, 16, 5256-5265.
30. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2006). Banishing the homunculus: Making working memory work. Neuroscience, 139, 105-118.
31. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2007). Towards an executive without a homunculus: Computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 362(1), 105-118.
32. Himmelbach, M., & Karnath, H.-O. (2005). Dorsal and ventral stream interaction: Contributions from optic ataxia. Journal of Cognitive Neuroscience, 17(4), 632-640.
33. Holland, P. C., & Gallagher, M. (2003). Double dissociation of the effects of lesions of basolateral and central amygdala on conditioned stimulus-potentiated feeding and Pavlovian-instrumental transfer. European Journal of Neuroscience, 17(8), 1680-1694.
34. Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 233-248). Cambridge, MA: MIT Press.
35. Houk, J. C., Bastianen, C., Fansler, D., Fishbach, A., Fraser, D., Reber, P. J., et al. (2007). Action selection and refinement in subcortical loops through basal ganglia and cerebellum. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 362(1485), 1573-1583.
36. Houk, J. C., & Wise, S. P. (1995). Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: Their role in planning and controlling action. Cerebral Cortex, 5, 95-110.
37. Humphries, M. D., Stewart, R. D., & Gurney, K. N. (2006). A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. Journal of Neuroscience, 26(50), 12921-12942.
38. James, T. W, Culham, J., Humphrey, G. K., Milner, A. D., & Goodale, M. A. (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: An fMRI study. Brain, 126(11), 2463-2475.
39. Joel, D., & Weiner, I. (2000). The connections of the dopaminergic system with the striatum in rats and primates: An analysis with respect to the functional and compartmental organization of the striatum. Neuroscience, 96, 451.
40. Johnson, L. R., Aylward, R. L., Hussain, Z., & Totterdell, S. (1995). Input from the amygdala to the rat nucleus accumbens: Its relationship with tyrosine hydroxylase immunoreactivity and identified neurons. Neuroscience, 61(4), 851-865.
41. Kantak, K. M., Green-Jordan, K., Valencia, E., Kremin, T., & Eichenbaum, H. B. (2001). Cognitive task performance after lidocaine-induced inactivation of different sites within the basolateral amygdala and dorsal striatum. Behavioral Neuroscience, 115, 589-601.
42. Klein, R. M. (2000). Inhibition of return. Trends in Cognitive Sciences, 4(4), 138-147.
43. Levitt, J. B., Lewis, D. A., Yoshioka, T., & Lund, J. S. (1993). Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 & 46). Journal of Comparative Neurology, 338, 360-376.
44. Loh, M., Pasupathy, A., Miller, E. K., & Deco, G. (2008). Neurodynamics of the prefrontal cortex during conditional visuomotor associations. Journal of Cognitive Neuroscience, 20(3), 421-431.
45. Mink, J. W. (1996). The basal ganglia: Focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50, 381-425.
46. Murray, E. A. (2007). The amygdala, reward and emotion. Trends in Cognitive Sciences, 11(11), 489-497.
47. O'Doherty, J. (2004). Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14(6), 769-776.
48. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669), 452-454.
49. O'Reilly, R. C. (1996). Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8(5), 895-938.
50. O'Reilly, R. C. (1998). Six principles for biologically based computational models of cortical cognition. Trends in Cognitive Sciences, 2(11), 455-462.
51. O'Reilly, R. C., & Frank, M. J. (2006). Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation, 18, 283-328.
52. O'Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007). PVLV: The primary value and learned value Pavlovian learning algorithm. Behavioral Neuroscience, 121, 31-49.
53. O'Reilly, R. C., & Munakata, Y (2000). Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
54. Packard, M. G., Hirsh, R., & White, N. M. (1989). Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: Evidence for multiple memory systems. Journal of Neuroscience, 9, 1465-1472.
55. Packard, M. G., & McGaugh, J. L. (1992). Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: Further evidence for multiple memory systems. Behavioral Neuroscience, 106(3), 439-446.
56. Parkinson, J. A., Robbins, T. W, & Everitt, B. J. (2000). Dissociable roles of the central and basolateral amygdala in appetitive emotional learning. European Journal of Neuroscience, 12, 405-413.
57. Pasupathy, A., & Miller, E. K. (2005). Different time courses for learning-related activity in the prefrontal cortex and striatum. Nature, 433, 873-876.
58. Petrides, M. (2005). Lateral prefrontal cortex: Architectonic and functional organization. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 360(1456), 781-795.
59. Petrides, M., & Pandya, D. N. (1999). Dorsolateral prefrontal cortex: Comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. European Journal of Neuroscience, 11(3), 1011-1036.
60. Petrides, M., & Pandya, D. N. (2002). Comparative cytoarchitectonic analysis of the human and the macaque ventrolateral prefrontal cortex and corticocortical connection patterns in the monkey. European Journal of Neuroscience, 16(2), 291-310.
61. Pickens, C. L., Saddoris, M. P., Gallagher, M., & Holland, P. C. (2005). Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task. Behavioral Neuroscience, 119(1), 317-322.
62. Poldrack, R. A., Prabhakaran, V., Seger, C. A., & Gabrieli, J. D. (1999). Striatal activation during acquisition of a cognitive skill. Neuropsychology, 13(4), 564-574.
63. Rouillard, C., & Freeman, A. S. (1997). Effects of electrical stimulation of the central nucleus of the amygdala on the in vivo electrophysiological activity of rat nigral dopaminergic neurons. Synapse, 21, 348-356.
64. Sakagami, M., & Tsutsui, K. (1999). The hierarchical organization of decision making in the primate prefrontal cortex. Neuroscience Research, 34, 79.
65. Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1999). Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. Journal of Neuroscience, 19, 1876-1884.
66. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1.
67. Schultz, W, Apicella, P, & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13, 900-913.
68. Semba, K., & Fibiger, H. (1992). Afferent connections of the laterodorsal and the pedunculopontine tegmental nuclei in the rat: A retro- and antero-grade transport and immunohistochemical study. Journal of Comparative Neurology, 323, 387-410.
69. Squire, L. R. (1992). Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans. Psychological Review, 99, 195-231.
70. Sutton, R. S. (1988). Learning to predict by the method of temporal diferences. Machine Learning, 3, 9-44.
71. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
72. Toni, I., Rowe, J., Stephan, K. E., & Passingham, R. E. (2002). Changes of corticostriatal effective connectivity during visuomotor learning. Cerebral Cortex, 12(10), 1040-1047.
73. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, & R. J. W. Mansfield (Eds.), The analysis of visual behavior (pp. 549-586). Cambridge, MA: MIT Press.
74. Vogels, R. (1999). Categorization of complex visual images by rhesus monkeys. Part I: Behavioural study. European Journal of Neuroscience, 11, 1223-1238.
75. Wickens, J. R., Kotter, R., & Alexander, M. E. (1995). Effects of local connectivity on striatal function: Simulation and analysis of a model. Synapse, 20, 281-298.
76. Williams, Z. M., & Eskandar, E. N. (2006). Selective enhancement of associative learning by microstimulation of the anterior caudate. Nature Neuroscience, 9, 562-568.
77. Winstanley, C. A., Theobald, D. E. H., Cardinal, R. N., & Robbins, T. W (2004). Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. Journal of Neuroscience, 24(20), 4718-4722.
78. Yeterian, E. H., & Pandya, D. N. (1995). Corticostriatal connections of extrastriate visual areas in rhesus monkeys. Journal of Comparative Neurology, 352(3), 436-457.