When does reward maximization lead to matching law?

PLoS ONE

Volumn 3, Issue 11, 2008, Pages

  Sakai, Yutaka ^a   Fukai, Tomoki ^b  

^a TAMAGAWA UNIVERSITY   (Japan)

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; BEHAVIOR CONTROL; CONTROLLED STUDY; DECISION MAKING; DECISION SUPPORT SYSTEM; DECISION THEORY; INFORMATION PROCESSING; INFORMATION STORAGE; LEARNING ALGORITHM; LEARNING THEORY; MATHEMATICAL COMPUTING; REWARD; SENSORY STIMULATION; ALGORITHM; ANIMAL; ANIMAL BEHAVIOR; BIOLOGICAL MODEL; THEORETICAL MODEL;

ANIMALIA;

ALGORITHMS; ANIMALS; BEHAVIOR, ANIMAL; CHOICE BEHAVIOR; DECISION MAKING; MODELS, NEUROLOGICAL; MODELS, THEORETICAL; REWARD;

EID: 57049112212     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0003795     Document Type: Article

References (30)

1. Sutton RS, Barto AG (1998) Reinforcement Learning. Cambridge, MA (USA): MIT press. 322 p.
2. Houk JC, Davis JL, Beiser DG (1994) Models of Information Processing in the Basal Ganglia (Computational Neuroscience) Bradford Books. 382 p.
3. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophsiol 80: 1-27.
4. Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, et al. (2004) Prediction of immediate and future rewards differentially recruits corticobasal ganglia loops. Nature Neurosci 7: 887-893.
5. Mazur JE (2005) Learning and Behavior. Prentice Hall. 464 p.
6. Herrnstein RJ (1961) Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav 4: 267-272.
7. Davison M, McCarthy D (1987) The Matching Law: A Research Review Lawrence Erlbaum Assoc Inc. 296 p.
8. Herrnstein RJ (1997) The Matching Law: Papers in Psychology and Economics. Cambridge, MA (USA): Harvard Univ Press. 334 p.
9. Sugrue LP, Corrado GS, Newsome WT (2004) Matching behavior and the representation of value in the parietal cortex. Science 304: 1782-1787.
10. Heyman GM (1979) A Markov model description of changeover probabilities on concurrent variable-interval schedules. J Exp Anal Behav 31: 41-51.
11. Baum WM (1981) Optimization and the matching law as accounts of instrumental behavior. J Exp Anal Behav 36: 387-402.
12. Herrnstein RJ, Heyman GM (1979) Is matching compatible with reinforcement maximization on concurrent variable interval, variable ratio? J Exp Anal Behav 31: 209-223.
13. Mazur J (1981) Optimization theory fails to predict performance of pigeons in a two-response situation. Science 214: 823-825.
14. Vaughan WJ (1981) Melioration, matching, and maximization. J Exp Anal Behav 36: 141-149.
15. DeCarlo LT (1985) Matching and maximizing with variable-time schedules. J Exp Anal Behav 43: 75-81.
16. Herrnstein RJ, Vaughan WJ (1980) Melioration and behavioral allocation. In Staddon J, ed. Limits to action: The allocation of individual behavior. New York (USA): Academic Press.
17. Corrado GS, Sugrue LP, Newsome WT (2005) Linear-nonlinear-Poisson models of primate choice dynamics. J Exp Anal Behav 84: 581-617.
18. Soltani A, Wang X (2006) A biophysically based neural model of matching law behavior: melioration by stochastic synapses. J Neurosci 26: 3731-3744.
19. Loewenstein Y, Seung H (2006) Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proc Natl Acad Sci 103: 15224-15229.
20. Sakai Y, Okamoto H, Fukai T (2006) Computational algorithms and neuronal network models underlying decision processes. Neural Netw 19: 1091-1105.
21. Sakai Y, Fukai T (2008) The actor-critic learning is behind the matching law: Matching vs. optimal behaviors. Neural Comput 20: 227-251.
22. Shanks DR, Tunney RJ, McCarthy JD (2002) A re-examination of probability matching and rational choice. J Behav Dec Making 15: 233-250.
23. Dayan P, Abbott L (2001) Theoretical Neuroscience. Cambridge, MA (USA): MIT press. 360 p.
24. Bertsekas DP, Tsitsiklis JN (1996) Neuro-Dynamic Programming. Belmont, MA (USA): Athena Scientific. 491 p.
25. Baum WM (1979) Matching, undermatching, and overmatching in studies of choice. J Exp Anal Behav 32: 269-81.
26. Davison M, Kerr A (1989) Sensitivity of time allocation to an overall reinforcer rate feedback function in concurrent interval schedules. J Exp Anal Behav 51: 215-231.
27. Alsop B, Davison M (1991) Effects of varying stimulus disparity and the reinforcer ratio in concurrent-schedule and signal-detection procedures. J Exp Anal Behav 56: 67-80.
28. Davison M, Nevin J (1999) Stimuli, reinforcers, and behavior: an integration. J Exp Anal Behav 71: 439-482.
29. Marbach P, Tsitsiklis JN (2001) Simulation-based optimization of Markov reward processes. IEEE Trans Automat Contr 46: 191-209.
30. Konda VR, Tsitsiklis JN (2003) On actor-critic algorithms. SIAM J Contr Optim 42: 1143-1166.