A biophysically based neural model of matching law behavior: Melioration by stochastic synapses

Journal of Neuroscience

Volumn 26, Issue 14, 2006, Pages 3731-3744

  Soltani, Alireza ^a   Wang, Xiao Jing ^a  

^a BRANDEIS UNIVERSITY   (United States)

Author keywords

Decision making; Dopamine; Lateral intraparietal cortex; Matching behavior; Melioration; Reward dependent stochastic Hebbian learning

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; BEHAVIOR; BIOPHYSICAL EQUIPMENT; DECISION MAKING; LEARNING; NERVE CELL PLASTICITY; PRIORITY JOURNAL; REINFORCEMENT; REPRODUCIBILITY; REWARD; SPIKE WAVE; STOCHASTIC MODEL; SYNAPSE; TASK PERFORMANCE;

ACTION POTENTIALS; ANIMALS; BEHAVIOR, ANIMAL; BIOPHYSICS; COMPUTER SIMULATION; DECISION MAKING; MODELS, NEUROLOGICAL; MODELS, STATISTICAL; NEURONAL PLASTICITY; NEURONS; PARIETAL LOBE; REINFORCEMENT (PSYCHOLOGY); STOCHASTIC PROCESSES; SYNAPSES; SYNAPTIC TRANSMISSION;

EID: 33645566919     PISSN: 02706474     EISSN: 02706474     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.5159-05.2006     Document Type: Article

References (67)

1. Amit DJ, Fusi S (1994) Dynamic learning in neural networks with material synapses. Neural Comput 6:957-982.
2. Anderson KG, Velkey AJ, Woolverton WL (2002) The generalized matching law as a predictor of choice between cocaine and food in rhesus monkeys. Psychopharmacology (Berl) 163:319-326.
3. Barraclough DJ, Conroy ML, Lee D (2004) Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7:404-410.
4. Baum WM (1974) 2 Types of deviation from matching law-bias and undermatching. J Exp Anal Behav 22:231-242.
5. Baum WM (1979) Matching, undermatching, and overmatching in studies of choice. J Exp Anal Behav 32:269-281.
6. Baum WM (1981) Optimization and the matching law as accounts of instrumental behavior. J Exp Anal Behav 36:387-403.
7. Bi G, Poo M (2001) Synaptic modification by correlated activity: Hebb's postulate revisited. Annu Rev Neurosci 24:139-166.
8. Brunel N, Wang X-J (2001) Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition. J Comput Neurosci 11:63-85.
9. Brunel N, Carusi F, Fusi S (1998) Slow stochastic hebbian learning of classes of stimuli in a recurrent neural network. Network 9:123-152.
10. Compte A, Constantinidis C, Tegner J, Raghavachari S, Chafee MV, Goldman-Rakic PS, Wang X-J (2003) Temporally irregular mnemonic persistent activity in prefrontal neurons of monkeys during a delayed response task. J Neurophysiol 90:3441-3454.
11. Corrado GS, Sugrue LP, Seung HS, Newsome WT (2005) Linear-nonlinear- Poisson models of primate choice dynamics. J Exp Anal Behav 84, 581-617.
12. Davison M, Baum WM (2000) Choice in a variable environment: every reinforcer counts. J Exp Anal Behav 74:1-24.
13. de Villiers PA, Herrnstein RJ (1976) Toward a law of response strength. Psychol Bull 83:1131-1153.
14. Dorris MC, Glimcher PW (2004) Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44:365-378.
15. Doya K (2002) Metalearning and neuromodulation. Neural Netw 15:495-506.
16. Fino E, Glowinski J, Venance L (2005) Bidirectional activity-dependent plasticity at corticostriatal synapses. J Neurosci 25:11279-11287.
17. Fusi S (2002) Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol Cybern 87:459-470.
18. Fusi S, Asaad WF, Miller EK, Wang X-J (2005a) A microcircuit model of arbitrary sensori-motor mapping: learning and forgetting on multiple timescales. Soc Neurosci Abstr 31:813.10.
19. Fusi S, Drew PJ, Abbott LF (2005b) Cascade models of synaptically stored memories. Neuron 45:599-611.
20. Gallistel CR (1994) Foraging for brain stimulation: toward a neurobiology of computation. Cognition 50:151-170.
21. Gallistel CR, Mark TA, King AP, Latham PE (2001) The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. J Exp Psychol Anim Behav Process 27:354-372.
22. Herrnstein RJ (1961) Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav 4:267-272.
23. Herrnstein RJ, Prelec D (1991) Melioration: a theory of distributed choice. J Econ Perspect 5:137-156.
24. Herrnstein RJ, Vaughan WJ (1980) Melioration and behavioral allocation. In: Limits to action: the allocation of individual behavior (Staddon JER, ed), pp 143-176. New York: Academic.
25. Herrnstein RJ, Rachlin H, Laibson DI (1997) The matching law: papers in psychology and economics. Cambridge, MA: Harvard UP.
26. Heyman GM (1979) A Markov model description of changeover probabilities on concurrent variable-interval schedules. J Exp Anal Behav 31:41-51.
27. Heyman GM, Luce D (1979) Operant matching is not a logical consequences of maximizing reinforcement rate. Learn Behav 7:133-140.
28. Hikosaka O, Nakamura K, Nakahara H (2006) Basal ganglia orient eyes to reward. J Neurophysiol 95:567-584.
29. Houston AI, Sumida BH (1987) Learning rules, matching and frequency dependence. J Theor Biol 126:289-308.
30. Huang Y-Y, Simpson E, Kellendonk C, Kandel ER (2004) Genetic evidence for the bidirectional modulation of synaptic plasticity in the prefrontal cortex by D1 receptors. Proc Natl Acad Sci USA 101:3236-3241.
31. Jay TM (2003) Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Prog Neurobiol 69:375-390.
32. Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411-416.
33. Lau B, Glimcher PW (2005a) Caudate neurons encode both saccade and reward information in a free-choice task. Soc Neurosci Abstr 31:400.14.
34. Lau B, Glimcher PW (2005b) Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav 84, 555-579.
35. Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413-417.
36. Leon MI, Shadlen MN (1999) Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24:415-425.
37. Malenka RC, Nicoll RA (1999) Long-term potentiation - a decade of progress? Science 285:1870-1874.
38. Montague PR, Berns GS (2002) Neural economics and the biological substrates of valuation. Neuron 36:265-284.
39. O'Connor DH, Wittenberg GM, Wang SS-H (2005) Graded bidirectional synaptic plasticity is composed of switch-like unitary events. Proc Natl Acad Sci USA 102:9679-9684.
40. Otani S, Daniel H, Roisin M-P, Crepel F (2003) Dopaminergic modulation of long-term synaptic plasticity in rat prefrontal neurons. Cereb Cortex 13:1251-1256.
41. Petersen CC, Malenka RC, Nicoll RA, Hopfield JJ (1998) All-or-none potentiation at CA3-CA1 synapses. Proc Natl Acad Sci USA 95:4732-4737.
42. Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400:233-238.
43. Reynolds JN, Wickens JR (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507-521.
44. Reynolds JN, Hyland BI, Wickens JR (2001) A cellular mechanism of reward-related learning. Nature 413:67-70.
45. Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475-9489.
46. Rorie AE, Newsome WT (2004) The role of area LIP in a direction discrimination task with multiple reward contingencies. Soc Neurosci Abstr 30:20.11.
47. Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337-1340.
48. Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG (1998) Signal timing across the macaque visual system. J Neurophysiol 79:3272-3278.
49. Schultz W (2000) Multiple reward signals in the brain. Nat Rev Neurosci 1:199-207.
50. Schultz W (2006) Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 57:87-115.
51. Schweighofer N, Doya K (2003) Meta-learning in reinforcement learning. Neural Netw 16:5-9.
52. Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-1073.
53. Shadlen MN, Newsome WT (1994) Noise, neural codes and cortical organization. Curr Opin Neurobiol 4:569-579.
54. Shadlen MN, Newsome WT (1996) Motion perception: seeing and deciding. Proc Natl Acad Sci USA 93:628-633.
55. Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916-1936.
56. Shidara M, Richmond BJ (2002) Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296:1709-1711.
57. Softky WR, Koch C (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J Neurosci 13:334-350.
58. Soltani A, Wang X-J (2004) Exploring the neural basis of the matching law in choice behavior: a cortical network model with reward-gated learning. Soc Neurosci Abstr 30:668.14.
59. Staddon JER, Motheral S (1978) On matching and maximization in operant choice experiments. Psychol Rev 85:436-444.
60. Sugrue LP, Corrado GC, Newsome WT (2004a) Matching behavior and representation of value in parietal cortex. Science 304:1782-1787.
61. Sugrue LP, Corrado GC, Newsome WT (2004b) Neural correlates of value in orbitofrontal cortex of the rhesus monkey. Soc Neurosci Abstr 30:671.8.
62. Sugrue LP, Corrado GS, Newsome WT (2005) Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6:363-375.
63. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT.
64. van Vreeswijk C, Sompolinsky H (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274:1724-1726.
65. Vaughan W (1981) Melioration, matching, and maximization. J Exp Anal Behav 36:141-149.
66. Wang X-J (2002) Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36:955-968.
67. Williams BA (1988) Reinforcement, choice, and response strength. In: Steven's handbook of experimental psychology, Ed 2, Vol 2 (Atkison RC, Herrnstein RJ, Lindzey G, Luce RD, eds), pp 167-244. New York: Wiley.