The basal ganglia and cortex implement optimal decision making between alternative actions

Neural Computation

Volumn 19, Issue 2, 2007, Pages 442-477

  Bogacz, Rafal ^a   Gurney, Kevin ^b  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33847145317     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2007.19.2.442     Document Type: Article

References (85)

1. Alexander, G. E., & Crutcher, M. D. (1990). Functional architecture of basal ganglia circuits: Neural substrates of parallel processing. Trends Neurosci., 13(7), 266-271.
2. Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci., 9, 357-381.
3. Amalric, M., & Koob, G. F. (1987). Depletion of dopamine in the caudate nucleus but not in nucleus accumbens impairs reaction-time performance in rats. J. Neurosci., 7(7), 2129-2134.
4. Amalric, M., Moukhles, H., Nieoullon, A., & Daszuta, A. (1995). Complex deficits on reaction time performance following bilateral intrastriatal 6-OHDA infusion in the rat. Eur. J. Neurosci., 7(5), 972-980.
5. Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychol Rev., 105, 442-481.
6. Ashby, F. G., & Spiering, B. J. (2004). The neurobiology of category learning. Behav. Cogn. Neurosci. Rev., 3(2), 101-113.
7. Barnard, G. (1946). Sequential tests in industrial statistics. Journal of Royal Statistical Society Supplement, 8, 1-26.
8. Baum, C. W., & Veeravalli, V. V. (1994). A sequential procedure for multihypothesis testing. IEEE Transactions on Information Theory, 40, 1996-2007.
9. Bevan, M. D., Smith, A. D., & Bolam, J. P. (1996). The substantia nigra as a site of synaptic integration of functionally diverse information arising from the ventral pallidum and the globus pallidus in the rat. Neuroscience, 75(1), 5-12.
10. Bogacz, R. (2006). Optimal decision networks with distributed representation. Manuscript submitted for publication.
11. Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced choice tasks. Psychol. Rev., 113, 700-765.
12. Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1993). Responses of neurons in macaque MT to stochastic motion signals. Vis. Neurosci., 10(6), 1157-1169.
13. Brown, E., Gao, J., Holmes, P., Bogacz, R., Gilzenrat, M., & Cohen, J. D. (2005). Simple networks that optimize decisions. International Journal of Bifurcations and Chaos, 15, 803-826.
14. Brown, J. W., Bullock, D., & Grossberg, S. (2004). How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Netw., 17(4), 471-510.
15. Chevalier, G., Vacher, S., Deniau, J. M., & Desban, M. (1985). Disinhibition as a basic process in the expression of striatal functions. I. The striato-nigral influence on tecto-spinal/tecto-diencephalic neurons. Brain Res., 334(2), 215-226.
16. Crutcher, M. D., & DeLong, M. R. (1984a). Single cell studies of the primate putamen. I. Functional organization. Exp. Brain Res., 53(2), 233-243.
17. Crutcher, M. D., & DeLong, M. R. (1984b). Single cell studies of the primate putamen. II. Relations to direction of movement and pattern of muscular activity. Exp. Brain Res., 53(2), 244-258.
18. Dayan, P. (2001). Levels of analysis in neural modeling. In R. A. Wilson & F. C. Keil (Eds.) Encyclopedia of cognitive science. London: Macmillan.
19. Deniau, J. M., & Chevalier, G. (1985). Disinhibition as a basic process in the expression of striatal functions. II. The striato-nigral influence on thalamocortical cells of the ventromedial thalamic nucleus. Brain Res., 334(2), 227-233.
20. Doya, K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol., 10(6), 732-739.
21. Dragalin, V. P., Tertakovsky, A. G., & Veeravalli, V. V. (1999). Multihypothesis sequential probability ratio tests - part I: asymptotic optimality. IEEE Transactions on Information Theory, 45, 2448-2461.
22. Faull, R. L., & Mehler, W. R. (1978). The cells of origin of nigrotectal, nigrothalamic and nigrostriatal projections in the rat. Neuroscience, 3(11), 989-1002.
23. Frank, M. J. (2005). When and when not to use your subthalamic nucleus: Lessons from a computational model of the basal ganglia. Paper presented at the International Workshop on Modelling Natural Action Selection, Edinburgh.
24. Frank, M. J., Seeberger, L. C., & O'Reilly R, C. (2004). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science, 306(5703), 1940-1943.
25. Georgopoulos, A. P., DeLong, M. R., & Crutcher, M. D. (1983). Relations between parameters of step-tracking movements and single cell discharge in the globus pallidus and subthalamic nucleus of the behaving monkey. J. Neurosci., 3(8), 1586-1598.
26. Gerfen, C. R., & Wilson, C. J. (1996). The basal ganglia. In A. Bjorklund, T. Hokfelt, & L. Swanson (Eds.), Handbook of chemical neuroanatomy (Vol. 12, pp. 369-466). New York: Elsevier.
27. Gerfen, C. R., & Young, W. S. III. (1988). Distribution of striatonigral and striatopallidal peptidergic neurons in both patch and matrix compartments: An in situ hybridization histochemistry and fluorescent retrograde tracing study. Brain Res., 460(1), 161-167.
28. Gold, J. I., & Shadlen, M. N. (2001). Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci., 5(1), 10-16.
29. Gold, J. I., & Shadlen, M. N. (2002). Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward. Neuron, 36(2), 299-308.
30. Gold, J. I., & Shadlen, M. N. (2003). The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands, J. Neurosci., 23(2), 632-651.
31. Gurney, K., Humphries, M., Wood, R., Prescott, T. J., & Redgrave, P. (2004). Testing computational hypotheses of brain systems function: A case study with the basal ganglia. Network, 15(4), 263-290.
32. Gurney, K., Prescott, T. J., & Redgrave, P. (2001a). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern., 84(6), 401-410.
33. Gurney, K., Prescott, T. J., & Redgrave, P. (2001b). A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biol. Cybern., 84(6), 411-123.
34. Gurney, K., Prescott, T. J., Wickens, J. R., & Redgrave, P. (2004). Computational models of the basal ganglia: From robots to membranes. Trends Neurosci., 27(8), 453-159.
35. Hallworth, N. E., Wilson, C. J., & Bevan, M. D. (2003). Apamin-sensitive small conductance calcium-activated potassium channels, through their selective coupling to voltage-gated calcium channels, are critical determinants of the precision, pace, and pattern of action potential generation in rat subthalamic nucleus neurons in vitro. J. Neurosci., 23(20), 7525-7542.
36. Humphries, M. D., & Gurney, K. N. (2002). The role of intra-thalamic and thalamocortical circuits in action selection. Network, 13(1), 131-156.
37. Humphries, M. D., Stewart, R. D., & Gurney, K. N. (in press). A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J. Neurosci.
38. Kha, H. T., Finkelstein, D. I., Tomas, D., Drago, J., Pow, D. V., & Horne, M. K. (2001). Projections from the substantia nigra pars reticulata to the motor thalamus of the rat: Single axon reconstructions and immunohistochemical study. J. Comp. Neurol., 440(1), 20-30.
39. Kim, J. N., & Shadlen, M. N. (1999). Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci., 2(2), 176-185.
40. Kita, H., & Kitai, S. T. (1991). Intracellular study of rat globus pallidus neurons: Membrane properties and responses to neostriatal, subthalamic and nigral stimulation. Brain Res., 564(2), 296-305.
41. Laming, D. R. J. (1968). Information theory of choice reaction time. New York: Wiley.
42. McMillen, T., & Holmes, P. (2006). The dynamics of choice among multiple alternatives. Journal of Mathematical Psychology, 50, 30-57.
43. Medina, L., & Reiner, A. (1995). Neurotransmitter organization and connectivity of the basal ganglia in vertebrates: Implications for the evolution of basal ganglia. Brain Behav. Evol., 46(4-5), 235-258.
44. Miller, E. K., & Cohen, J. D. (2001). Anintegrative theory of prefrontal cortex function. Annu. Rev. Neurosci., 24, 167-202.
45. Mink, J. W. (1996). The basal ganglia: Focused selection and inhibition of competing motor programs. Prog. Neurobiol., 50(4), 381-425.
46. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci., 16(5), 1936-1947.
47. Nakano, K., Kayahara, T., Tsutsumi, T., & Ushiro, H. (2000). Neural circuits and functional organization of the striatum. J. Neurol., 247 (Suppl 5), V1-15.
48. Nambu, A., & Llinas, R. (1994). Electrophysiology of globus pallidus neurons in vitro. J. Neurophysiol., 72(3), 1127-1139.
49. Nambu, A., & Llinas, R. (1997). Morphology of globus pallidus neurons: Its correlation with electrophysiology in guinea pig brain slices. J. Comp. Neurol., 377(1), 85-94.
50. Obeso, J. A., Rodriguez-Oroz, M. C., Rodriguez, M., Lanciego, J. L., Artieda, J., Gonzalo, N., & Olanow, C. W. (2000). Pathophysiology of the basal ganglia in Parkinson's disease. Trends Neurosci., 23(10 Suppl), S8-19.
51. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669), 452-454.
52. Okamoto, H., Isomura, Y., Takada, M., & Fukai, T. (2005). Temporal integration by stochastic dynamics of a recurrent network of bistable neurons. Paper presented at the Computational Cognitive Neuroscience meeting, Washington, DC.
53. O'Reilly, R. C., & Frank, M. J. (2006). Making working memory work: A computational model of learning in the frontal cortex and basal ganglia. Neural Computation, 18, 283-328.
54. Overton, P. G., & Greenfield, S. A. (1995). Determinants of neuronal firing pattern in the guinea-pig subthalamic nucleus: An in vivo and in vitro comparison. J. Neural Transm. Park. Dis. Dement. Sect., 10(1), 41-54.
55. Parent, A., & Hazrati, L. N. (1993). Anatomical aspects of information processing in primate basal ganglia. Trends Neurosci., 16(3), 111-116.
56. Parent, A., & Hazrati, L. N. (1995a). Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop. Brain Res. Rev., 20(1), 91-127.
57. Parent, A., & Hazrati, L. N. (1995b). Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidum in basal ganglia circuitry. Brain Res. Rev., 20(1), 128-154.
58. Parent, A., & Smith, Y. (1987). Organization of efferent projections of the subthalamic nucleus in the squirrel monkey as revealed by retrograde labeling methods. Brain Res., 436(2), 296-310.
59. Parthasarathy, H. B., Schall, J. D., & Graybiel, A. M. (1992). Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J. Neurosci., 12(11), 4468-4488.
60. Platt, M. L., & Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400(6741), 233-238.
61. Prescott, T. J., Montes-Gonzalez, F. M., Gurney, K., Humphries, M. D., & Redgrave, P. (2006). A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Networks, 19(1), 31-61.
62. Ratcliff, R. (1978). A theory of memory retrieval. Psychol. Rev., 83, 59-108.
63. Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and diffusion models of reaction time. Psychol. Rev., 106, 261-300.
64. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). The basal ganglia: A vertebrate solution to the selection problem? Neuroscience, 89(4), 1009-1023.
65. Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005). Representation of action-specific reward values in the striatum. Science, 310(5752), 1337-1340.
66. Schall, J. D. (2001). Neural basis of deciding, choosing and acting. Nat. Rev. Neurosci., 2(1), 33-42.
67. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593-1599.
68. Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol., 86(4), 1916-1936.
69. Shadmehr, R., & Holcomb, H. H. (1997). Neural correlates of motor memory consolidation. Science, 277(5327), 821-825.
70. Simen, P. A., Cohen, J. D., & Holmes, P. (2006). Rapid decision threshold modulation by reward rate in a neural network. Neural Networks, 19(8), 1013-1026.
71. Smith, Y., Bevan, M. D., Shink, E., & Bolam, J. P. (1998). Microcircuitry of the direct and indirect pathways of the basal ganglia. Neuroscience, 86(2), 353-387.
72. Stafford, T., & Gurney, K. (2004). The role of response mechanisms in determining reaction time performance: Pieron's law revisited. Psychon. Bull. Rev., 11, 975-987.
73. Stafford, T., & Gurney, K. (in press). The basal ganglia as the selection mechanism in a cognitve task. Philos. Trans. R. Soc. Lond. B Biol. Sci.
74. Stewart, R. D., Gurney, K., & Humphries, M. D. (2005). A large-scale model of the sensorimotor basal ganglia: Evoked responses and selection properties. Paper presented at the Society for Neuroscience meeting, Washington, DC.
75. Stone, M. (1960). Models for choice reaction time. Psychometrika, 25, 251-260.
76. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.
77. Teichner, W. H., & Krebs, M. J. (1974). Laws of visual choice reaction time. Psychol. Rev., 81, 75-98.
78. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychol. Rev., 108(3), 550-592.
79. Vickers, D. (1970). Evidence for an accumulator model of psychophysical discrimination. Ergonomics, 13, 37-58.
80. Wald, A. (1947). Sequential analysis. New York: Wiley.
81. Wald, A., & Wolfowitz, J. (1948). Optimum character of the sequential probability ratio test. Annals of Mathematical Statistics, 19, 326-339.
82. Wang, X. J. (2002). Probabilistic decision making by slow reverberation in cortical circuits. Neuron, 36(5), 955-968.
83. Wilson, C. J. (1995). The contribution of cortical neurons to firing patterns of striatal spiny neurons. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 22-50). Cambridge, MA: MIT Press.
84. Wilson, C. J., Weyrick, A., Terman, D., Hallworth, N. E., & Bevan, M. D. (2004). A model of reverse spike frequency adaptation and repetitive firing of subthalamic nucleus neurons. J. Neurophysiol., 91(5), 1963-1980.
85. Yu, A. D., & Dayan, P. (2005). Inference, attention, and decision in a Bayesian neural architecture. In L. K. Saul, W. Yair, & L. Bottou (Eds.), Advances in neural information processing systems, 17 (pp. 1577-1584). Cambridge, MA: MIT Press.