Action selection performance of a reconfigurable Basal Ganglia inspired model with Hebbian-Bayesian Go-NoGo connectivity

Frontiers in Behavioral Neuroscience

Volumn , Issue SEPTEMBER, 2012, Pages

  Berthet, Pierre ^a,b,c   Kotaleski, Jeanette Hellgren ^a,c   Lansner, Anders ^a,b,c  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^b STOCKHOLM UNIVERSITY   (Sweden)

^c SBI Stockholm Brain Institute   (Sweden)

Author keywords

Basal ganglia; Bayesian inference; BCPNN; Behaviour selection; Direct indirect pathway; Dopamine; Hebbian Bayesian plasticity; Reinforcement learning

Indexed keywords

ARTICLE; BASAL GANGLION; BAYES THEOREM; BRAIN FUNCTION; CONTROLLED STUDY; DOPAMINERGIC TRANSMISSION; LEARNING; MATHEMATICAL MODEL; MENTAL TASK; MOTOR CONTROL; NERVE CELL PLASTICITY; PROBABILITY; REINFORCEMENT; STATISTICAL ANALYSIS;

EID: 84866713557     PISSN: 16625153     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnbeh.2012.00065     Document Type: Article

References (87)

1. Albin, R. L., Young, A. B., and Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in neurosciences 12, 366-375. Available at: http://linkinghub.elsevier.com/retrieve/pii/016622368990074X [Accessed April 6, 2011].
2. Alexander, G., DeLong, M., and Strick, P. L. (1986). Parallel organization of functionally segregate700 d circuits linking basal ganglia and cortex. Annual review of Neuroscience 9, 357-381. Available at: http://www.annualreviews.org/doi/pdf/10.1146/annurev.ne.09.030186.002041 [Accessed October 25, 2011].
3. Aosaki, T., Graybiel, A., and Kimura, M. (1994). Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265, 412-415. Available at: http://www.sciencemag.org/content/265/5170/412.short [Accessed September 28, 2011].
4. Bar-Gad, I., Morris, G., and Bergman, H. (2003). Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Progress in neurobiology 71, 439-73. Available at: http://www.ncbi.nlm.nih.gov/pubmed/15013228.
5. Berns, G. S., McClure, S. M., Pagnoni, G., and Montague, P. R. (2001). Predictability modulates human brain response to reward. The Journal of neuroscience: the official journal of the Society for Neuroscience 21, 2793-8. Available at: http://www.ncbi.nlm.nih.gov/pubmed/11306631.
6. Bolam, J. P., Hanley, J. J., Booth, P. A., and Bevan, M. D. (2000). Synaptic organisation of the basal ganglia. Journal of anatomy 196 (Pt 4, 527-42. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1468095&tool=pmcentrez&rendertyp e=abstract.
7. Bouton, M. E. (1986). Slow reacquisition following the extinction of conditioned suppression. Learning and Motivation 17, 1-15. Available at: http://linkinghub.elsevier.com/retrieve/pii/0023969086900172.
8. Buesing, L., Bill, J., Nessler, B., and Maass, W. (2011). Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons. PLoS Computational Biology 7, e1002211. Available at: http://dx.plos.org/10.1371/journal.pcbi.1002211 [Accessed November 4, 2011].
9. Calabresi, P., Pisani, A., Mercuri, N. B., and Bernardi, G. (1996). The corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia. Trends in Neurosciences 19, 19-24. Available at: http://www.sciencedirect.com/science/article/pii/S0079612308626783 [Accessed September 28, 2011].
10. Chakravarthy, V. S., Joseph, D., and Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biological cybernetics 103, 237-53. Available at: http://www.ncbi.nlm.nih.gov/pubmed/20644953 [Accessed December 8, 2010].
11. Charlesworth, J. D., Warren, T. L., and Brainard, M. S. (2012). Covert skill learning in a cortical-basal ganglia circuit. Nature 486, 251-5. Available at: http://www.ncbi.nlm.nih.gov/pubmed/22699618 [Accessed July 12, 2012].
12. Chaves, L. M., and Hodos, W. (1998). Color reversal-learning deficits after tectofugal pathway lesions in the pigeon telencephalon. Behavioural brain research 90, 1-12. Available 735 le at: http://www.ncbi.nlm.nih.gov/pubmed/9520209.
13. Cohen, M. X., and Frank, M. J. (2009). Neurocomputational models of basal ganglia function in learning, memory and choice. Behavioural brain research 199, 141-56. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2762323&tool=pmcentrez&rendertyp e=abstract [Accessed March 19, 2012].
14. Crittenden, J. R., and Graybiel, A. M. (2011). Basal Ganglia Disorders Associated with Imbalances in the Striatal Striosome and Matrix Compartments. Frontiers in neuroanatomy 5, 59. Available at: http://www.ncbi.nlm.nih.gov/pubmed/21941467 [Accessed September 24, 2011].
15. Daw, N. D., and Doya, K. (2006). The computational neurobiology of learning and reward. Current opinion in neurobiology 16, 199-204. Available at: http://www.ncbi.nlm.nih.gov/pubmed/16563737.
16. Deneve, S. (2008). Bayesian spiking neurons I: inference. Neural computation 20, 91-117. Available at: http://www.ncbi.nlm.nih.gov/pubmed/18045002.
17. Doig, N. M., Moss, J., and Bolam, J. P. (2010). Cortical and thalamic innervation of direct and indirect pathway medium-sized spiny neurons in mouse striatum. The Journal of Neuroscience 30, 14610. Available at: http://www.jneurosci.org/content/30/44/14610.short [Accessed September 28, 2011].
18. Doya, K. (2007). Reinforcement learning: Computational theory and biological mechanisms. HFSP journal 1, 30-40. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2645553&tool=pmcentrez&rendertyp e=abstract [Accessed June 14, 2011].
19. Doya, K., Ishii, S., Pouget, A., and Rao, R. P. N. (2007). Bayesian Brain. MIT Press.
20. Fiorillo, C. D., Tobler, P. N., and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science (New York, N.Y.) 299, 1898-902. Available at: http://www.ncbi.nlm.nih.gov/pubmed/12649484 [Accessed March 1, 2012].
21. Friston, K. (2005). Hallucinations and perceptual inference. Behavioral and Brain Sciences, 764-766. Available at: http://journals.cambridge.org/abstract_S0140525X05290131 [Accessed March 15, 2012].
22. Fujiyama, F., Sohn, J., Nakano, T., Furuta, T., Nakamura, K. C., Matsuda, W., and Kaneko, T. (2011). Exclusive and common targets of neostriatofugal projections of rat striosome neurons: a single neuron-tracing study using a viral vector. The European Journal of Neuroscience 33, 765 668-677. Available at: http://www.ncbi.nlm.nih.gov/pubmed/21314848 [Accessed August 12, 2011].
23. Gage, G. J., Stoetzner, C. R., Wiltschko, A. B., and Berke, J. D. (2010). Selective activation of striatal fast-spiking interneurons during choice execution. Neuron 67, 466-79. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2920892&tool=pmcentrez&rendertyp e=abstract [Accessed March 26, 2012].
24. George, D. N., Jenkins, T. A, and Killcross, S. (2011). Dissociation of prefrontal cortex and nucleus accumbens dopaminergic systems in conditional learning in rats. Behavioural brain research 225, 47-55. Available at: http://www.ncbi.nlm.nih.gov/pubmed/21741412 [Accessed July 18, 2011].
25. Gerfen, C. R., Engber, T. M., Mahan, L. C., Susel, Z., Chase, T. N., Monsma, F. J., and Sibley, D. R. (1990). D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science (New York, N.Y.) 250, 1429-32. Available at: http://www.ncbi.nlm.nih.gov/pubmed/2147780.
26. Gerfen, C. R., Herkenham, M., and Thibault, J. (1987). The Neostriatal Dopaminergic Mosaic: II. Patch-and Matrix-Directed Mesostriatal Dopaminergic and Non-Dopaminergic Systems Mesostriatal. Journal of Neuroscience 7, 3915-3934.
27. Gillies, A., and Arbuthnott, G. (2000). Computational models of the basal ganglia. Movement disorders: official journal of the Movement Disorder Society 15, 762-70. Available at: http://www.ncbi.nlm.nih.gov/pubmed/11009178.
28. Graybiel, A. M. (1995). Building action repertoires: memory and learning functions of the basal ganglia. Current opinion in neurobiology 5, 733-41. Available at: http://www.ncbi.nlm.nih.gov/pubmed/8805417.
29. Graybiel, A. M. (1990). Neurotransmitters and neuromodulators in the basal ganglia. Trends in neurosciences 13, 244-54. Available at: http://www.ncbi.nlm.nih.gov/pubmed/1695398.
30. Graybiel, A. M. (2004). Network-level neuroplasticity in cortico-basal ganglia pathways. Parkinsonism & related disorders 10, 293-6. Available at: http://www.ncbi.nlm.nih.gov/pubmed/15196508.
31. Graybiel, A. M. (2005). The basal ganglia: learning new tricks and loving it. Current opinion in neurobiology 15, 638-44. Available at: http://www.ncbi.nlm.nih.gov/pubmed/20670826.
32. Griffiths, T. L., and Tenenbaum, J. B. (2006). Optimal predictions in everyday cognition. Psychological science 17, 767-73. Available at: http://www.ncbi.nlm.nih.gov/pubmed/16984293.
33. Grillner, S., Hellgren, J., Ménard, A., Saitoh, K., and Wikström, M. a (2005). Mechanisms for selecti795 on of basic motor programs--roles for the striatum and pallidum. Trends in neurosciences 28, 364-70. Available at: http://www.ncbi.nlm.nih.gov/pubmed/15935487 [Accessed March 9, 2012]. Groman, S. M., Lee, B., London, E. D., Mandelkern, M. a, James, A. S., Feiler, K., Rivera, R.,
34. Groman, S. M., Lee, B., London, E. D., Mandelkern, M. a, James, A. S., Feiler, K., Rivera, R., Dahlbom, M., Sossi, V., Vandervoort, E., et al. (2011). Dorsal striatal D2-like receptor availability covaries with sensitivity to positive reinforcement during discrimination learning. The Journal of neuroscience: the official journal of the Society for Neuroscience 31, 7291-9. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3114883&tool=pmcentrez&rendertyp e=abstract [Accessed July 26, 2011].
35. Gurney, K., Prescott, T., and Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological cybernetics 84, 401-410. Available at: http://www.springerlink.com/index/VHT1NM66H3V2NAQC.pdf [Accessed October 25, 2011].
36. Haber, S. (2003). The primate basal ganglia: parallel and integrative networks. Journal of Chemical Neuroanatomy 26, 317-330. Available at: http://linkinghub.elsevier.com/retrieve/pii/S0891061803001078.
37. Hikosaka, O., Bromberg-Martin, E., Hong, S., and Matsumoto, M. (2008). New insights on the subcortical representation of reward. Current opinion in neurobiology 18, 203-208. Available at: http://www.sciencedirect.com/science/article/pii/S0959438808000603 [Accessed October 25, 2011].
38. Holst, A., and Lansner, A. (1996). A higher order Bayesian neural network with spiking units. International Journal of Neural Systems. Available at: http://dml.cs.byu.edu/~cgc/docs/atdm/BCPNN-96.pdf [Accessed April 25, 2012].
39. Hoover, J. E., and Strick, P. L. (1993). Multiple output channels in the basal ganglia. Science (New York, N.Y.) 259, 819-21. Available at: http://www.ncbi.nlm.nih.gov/pubmed/7679223.
40. Houk, J. C., Adams, J. L., and Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement., eds. J. C. Houk, J. L. Davis, and D. G. Beiser MIT Press Available at: http://books.google.com/books?hl=en&lr=&id=q6RThpQR_aIC&oi=fnd&pg =PA249&dq=A+model+of+how+the+basal+ganglia+generate+and+use+neural+signals+that +predict+reinforcement&ots=zPWtYfGo7n&sig=_hCSGDbgX1FQhULRxep68OL14cE
41. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral cortex (New York, N.Y.: 1991) 17, 2443-52. Available at: http://www.ncbi.nlm.nih.gov/pubmed/17220510 [Accessed July 20, 2011].
42. Jaber, M., Robinson, S., and Missale, C. (1997). Dopamine receptors and brain function. Neuropharmacology 35, 1503-1519. Available 830 ble at: http://www.sciencedirect.com/science/article/pii/S0028390896001001 [Accessed March 16, 2012].
43. Joel, D., Niv, Y., and Ruppin, E. (2002). Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural networks: the official journal of the International Neural Network Society 15, 535-47. Available at: http://www.ncbi.nlm.nih.gov/pubmed/12371510.
44. Johansson, C., and Lansner, A. (2007). Towards cortex sized artificial neural systems. Neural networks: the official journal of the International Neural Network Society 20, 48-61. Available at: http://www.ncbi.nlm.nih.gov/pubmed/16860539 [Accessed April 30, 2012].
45. Judge, P. G., Evans, D. W., Schroepfer, K. K., and Gross, A. C. (2011). Perseveration on a reversal840 learning task correlates with rates of self-directed behavior in nonhuman primates. Behavioural brain research 222, 57-65. Available at: http://www.ncbi.nlm.nih.gov/pubmed/21419808 [Accessed April 27, 2012].
46. Kehoe, J. E., and Macrae, M. (1997). Savings in Animal Learning: Implications for Relapse and Maintenance After Therapy. Behavior therapy 28, 141-155.
47. Kerr, J., and Wickens, J. R. (2001). Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. Journal of Neurophysiology, 117-124. Available at: http://jn.physiology.org/content/85/1/117.short [Accessed September 28, 2011].
48. Kincaid, a E., Zheng, T., and Wilson, C. J. (1998). Connectivity and convergence of single corticostriatal axons. The Journal of neuroscience: the official journal of the Society for Neuroscience 18, 4722-31. Available at: http://www.ncbi.nlm.nih.gov/pubmed/9614246. Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., and Kreitzer, A.
49. Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., and Kreitzer, A.C. (2010). Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622-626. Available at: http://www.nature.com/doifinder/10.1038/nature09159 [Accessed July 7, 2010].
50. Körding, K. P., and Wolpert, D. M. (2004). Bayesian integration in sensorimotor learning. Nature 427, 244-7. Available at: http://www.ncbi.nlm.nih.gov/pubmed/14724638.
51. Lansner, A. (2009). Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations. Trends in neurosciences 32, 178-86. Available at: http://www.ncbi.nlm.nih.gov/pubmed/19187979 [Accessed March 1, 2012].
52. Lau, B., and Glimcher, P. W. (2008). Value representations in the primate striatum during matching behavior. Neuron 58, 451-463. Available at: http://www.sciencedirect.com/science/article/pii/S089662730800175X [Accessed April 27, 2012].
53. Lundqvist, M., Herman, P., and Lansner, A. (2011). Theta and gamma power increases and alpha/beta power decreases with memory load in an attractor network model. Journal of Cognitive Neuroscience, 1-13. Available at: http://www.mitpressjournals.org/doi/abs/865 /10.1162/jocn_a_00029 [Accessed August 20, 2012].
54. Matsuda, W., Furuta, T., Nakamura, K. C., Hioki, H., Fujiyama, F., Arai, R., and Kaneko, T. (2009). Single nigrostriatal dopaminergic neurons form widely spread and highly dense axonal arborizations in the neostriatum. The Journal of neuroscience: the official journal of the Society for Neuroscience 29, 444-453. Available at: http://www.ncbi.nlm.nih.gov/pubmed/19144844 [Accessed August 4, 2011].
55. McCallum, J., Kim, J. H., and Richardson, R. (2010). Impaired Extinction Retention in Adolescent Rats: Effects of D-Cycloserine. Neuropsychopharmacology 35, 2134-2142. Available at: http://www.nature.com/doifinder/10.1038/npp.2010.92 [Accessed May 7, 2012].
56. McHaffie, J. G., Stanford, T. R., Stein, B. E., Coizet, V., and Redgrave, P. (2005). Subcortical loops through the basal ganglia. Trends in neurosciences 28, 401-7. Available at: http://www.ncbi.nlm.nih.gov/pubmed/15982753.
57. Mengual, E., de las Heras, S., Erro, E., Lanciego, J. L., and Giménez-Amaya, J. M. (1999). Thalamic interaction between the input and the output systems of the basal ganglia. Journal of chemical neuroanatomy 16, 187-200. Available at: http://www.ncbi.nlm.nih.gov/pubmed/10422738.
58. Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Progress in neurobiology 50, 381-425. Available at: http://www.ncbi.nlm.nih.gov/pubmed/9004351.
59. Montague, P., Dayan, P., and Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of Neuroscience 16, 1936-1947. Available at: http://www.jneurosci.org/content/16/5/1936.short [Accessed October 20, 2011].
60. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., and Hikosaka, O. (2004). Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269-80. Available at: http://www.ncbi.nlm.nih.gov/pubmed/14741107. Obeso, J. A., Marin, C., Rodriguez-Oroz, C., Blesa, J., Benitez-Temiño, B., Mena-Segovia, J.,
61. Obeso, J. A., Marin, C., Rodriguez-Oroz, C., Blesa, J., Benitez-Temiño, B., Mena-Segovia, J., Rodríguez, M., and Olanow, C. W. (2008). The basal ganglia in Parkinson's disease: current concepts and unexplained observations. Annals of neurology 64 Suppl 2, S30-46. Available at: http://www.ncbi.nlm.nih.gov/pubmed/19127584.
62. O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., and Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron 38, 329-37. Available at: http://www.ncbi.nlm.nih.gov/pubmed/12718865.
63. Parent, A. (1990). Extrinsic connections of the basal ganglia. Trends in Neurosciences 13, 254-258. Available at: http://www.ncbi.nlm.nih.gov/pubmed/1695399.
64. Parent, A., and Hazrati, L. N. (1995). Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidium in basal ganglia circuitry. Brain Research Reviews 900 20, 128-154. Available at: http://www.sciencedirect.com/science/article/pii/016501739400008d [Accessed October 25, 2011].
65. Parush, N., Tishby, N., and Bergman, H. (2011). Dopaminergic Balance between Reward Maximization and Policy Complexity. Frontiers in systems neuroscience 5, 22. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3093748&tool=pmcentrez&rendertyp e=abstract.
66. Pawlak, V., Wickens, J. R., Kirkwood, A., and Kerr, J. N. D. (2010). Timing is not Everything: Neuromodulation Opens the STDP Gate. Frontiers in synaptic neuroscience 2, 146. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3059689&tool=pmcentrez&rendertyp e=abstract [Accessed July 6, 2011].
67. Peters, A., and Yilmaz, E. (1993). Neuronal Organization in Area 17 of Cat Visual Cortex. Cerebral Cortex 3, 49-68. Available at: http://cercor.oxfordjournals.org/cgi/doi/10.1093/cercor/3.1.49 [Accessed August 23, 2012].
68. Planert, H., Szydlowski, S. N., Hjorth, J. J. J., Grillner, S., and Silberberg, G. (2010). Dynamics of synaptic transmission between fast-spiking interneurons and striatal projection neurons of the direct and indirect pathways. The Journal of neuroscience: the official journal of the Society for Neuroscience 30, 3499-507. Available at: http://www.ncbi.nlm.nih.gov/pubmed/20203210 [Accessed July 20, 2011].
69. Redgrave, P., Prescott, T., and Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009-1023. Available at: http://linkinghub.elsevier.com/retrieve/pii/S0306452298003194.
70. Reynolds, J. N. J., and Wickens, J. R. (2002). Dopamine-dependent plasticity of corticostriatal synapses. Neural Networks 15, 507-521. Available at: http://linkinghub.elsevier.com/retrieve/pii/S089360800200045X [Accessed January 25, 2011].
71. Reynolds, J., and Wickens, J. (2000). Substantia nigra dopamine regulates synaptic plasticity and membrane potential fluctuations in the rat neostriatum, in vivo. Neuroscience 99, 199-203. Available at: http://www.sciencedirect.com/science/article/pii/S0306452200002736 [Accessed September 28, 2011].
72. Samejima, K., Ueda, Y., Doya, K., and Kimura, M. (2005). Representation of action-specific reward values in the striatum. Science (New York, N.Y.) 310, 1337-40. Available at: http://www.ncbi.nlm.nih.gov/pubmed/16311337 [Accessed March 7, 2012].
73. Sandberg, A., Lansner, A., Petersson, K. M., and Ekeberg, O. (2002). A Bayesian attractor network with incremental learning. Network 13, 179-194. Available at: http://www.ncbi.nlm.nih.gov/pubmed/12061419.
74. Schmajuk, N. A. (1997). Animal learning and cognition: a neural network approach. 935. Cambridge University Press.
75. Schultz, W. (2007). Behavioral dopamine signals. Trends in neurosciences 30, 203-10. Available at: http://www.ncbi.nlm.nih.gov/pubmed/17400301 [Accessed July 18, 2011].
76. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of neurophysiology 80, 1-27. Available at: http://www.ncbi.nlm.nih.gov/pubmed/9658025.
77. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science (New York, N.Y.) 275, 1593-9. Available at: http://www.ncbi.nlm.nih.gov/pubmed/9054347.
78. Schultz, W., and Dickinson, a (2000). Neuronal coding of prediction errors. Annual review of neuroscience 23, 473-500. Available at: http://www.ncbi.nlm.nih.gov/pubmed/19176813.
79. Schultz, W., Tremblay, L., and Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. Cerebral cortex (New York, N.Y.: 1991) 10, 272-84. Available at: http://www.ncbi.nlm.nih.gov/pubmed/10731222.
80. Shen, W., Flajolet, M., Greengard, P., and Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851. Available at: http://www.sciencemag.org/content/321/5890/848.short [Accessed September 28, 2011].
81. Stephenson-Jones, M., Samuelsson, E., Ericsson, J., Robertson, B., and Grillner, S. (2011). Evolutionary conservation of the basal ganglia as a common vertebrate mechanism for action selection. Current biology: CB 21, 1081-91. Available at: http://www.ncbi.nlm.nih.gov/pubmed/21700460 [Accessed March 1, 2012].
82. Suri, R. E. (2002). 2002 Special issue TD models of reward predictive responses in dopamine neurons. Neural Networks 15, 523-533.
83. Suri, R. E., and Schultz, W. (2001). Temporal difference model reproduces anticipatory neural activity. Neural Computation 13, 841-862. Available at: http://www.mitpressjournals.org/doi/pdf/10.1162/089976601300014376 [Accessed October 25, 2011].
84. Surmeier, D. J., Ding, J., Day, M., Wang, Z., and Shen, W. (2007). D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends in neurosciences 30, 228-35. Available at: http://www.ncbi.nlm.nih.gov/pubmed/17408758.
85. Sutton, R. S., and Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge Univ Pre965 ss Available at: http://journals.cambridge.org/production/action/cjoGetFulltext?fulltextid=34656 [Accessed April 27, 2012].
86. Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., and Pennartz, C. M. a (2004). Putting a spin on the dorsal-ventral divide of the striatum. Trends in neurosciences 27, 468-74. Available at: http://www.ncbi.nlm.nih.gov/pubmed/15271494.
87. Yang, T., and Shadlen, M. N. (2007). Probabilistic reasoning by neurons. Nature 447, 1075-80. Available at: http://www.ncbi.nlm.nih.gov/pubmed/17546027 [Accessed July 24, 2010].