Reinforcement learning and decision making in monkeys during a competitive game

Cognitive Brain Research

Volumn 22, Issue 1, 2004, Pages 45-58

  Lee, Daeyeol ^a   Conroy, Michelle L ^a   McGreevy, Benjamin P ^a   Barraclough, Dominic J ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

Author keywords

Game theory; Mixed strategy; Motivation; Prefrontal cortex; Reward; Zero sum game

Indexed keywords

ALGORITHM; ANIMAL EXPERIMENT; ARTICLE; COMPETITION; COMPUTER; COMPUTER PROGRAM; CONTROLLED STUDY; DECISION MAKING; GAME; HISTORY; MALE; MONKEY; NONHUMAN; OCULOMOTOR SYSTEM; PREDICTION; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; REWARD; SACCADIC EYE MOVEMENT; VISUAL STIMULATION;

ALGORITHMS; ANIMALS; BEHAVIOR, ANIMAL; COMPETITIVE BEHAVIOR; DECISION MAKING; FEEDBACK; LEARNING; LOGISTIC MODELS; MACACA MULATTA; MALE; MODELS, PSYCHOLOGICAL; PROBABILITY; REINFORCEMENT (PSYCHOLOGY); TIME FACTORS;

EID: 9244231144     PISSN: 09266410     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cogbrainres.2004.07.007     Document Type: Article

References (74)

1. A. Baddeley, H. Emslie, J. Kolodny, and J. Duncan Random generation and the executive control of working memory Q. J. Exp. Psychol. 51 1998 819 852
2. D.J. Barraclough, M.L. Conroy, and D. Lee Prefrontal cortex and decision making in a mixed-strategy game Nat. Neurosci. 7 2004 404 410
3. K. Binmore, J. Swierzbinski, and C. Proulx Does minimax work? An experimental study Econ. J. 111 2001 445 464
4. H.C. Breiter, I. Aharon, D. Kahneman, A. Dale, and P. Shizgal Functional imaging of neural responses to expectancy and experience of monetary gains and losses Neuron 30 2001 619 639
5. J.N. Brown, and R.W. Rosenthal Testing the minimax hypothesis: a re-examination of O'Neill's game experiment Econometrica 58 1990 1065 1081
6. D.V. Budescu, and A. Rapoport Subjective randomization in one- and two-person games J. Behav. Decis. Mak. 7 1994 261 278
7. K.P. Burnham, and D.R. Anderson Model Selection and Multimodel Inference. A Practical Information-Theoretic Approach Second edition 2002 Springer-Verlag New York
8. R.R. Bush, and F. Mosteller Stochastic Models for Learning 1955 Wiley New York
9. C.F. Camerer Behavioral Game Theory: Experiments in Strategic Interaction 2003 Princeton Univ. Press Princeton
10. R.H.S. Carpenter A neural mechanism that randomises behavior J. Conscious. Stud. 6 1999 13 22
11. H.-C. Chen, J.W. Friedman, and J.-F. Thisse Boundedly rational Nash equilibrium: a probabilistic choice approach Games Econ. Behav. 18 1996 1832 1854
12. P.-A. Chiappori, S. Levitt, and T. Groseclose Testing mixed-strategy equilibria when players are heterogeneous: the case of penalty kicks in soccer Am. Econ. Rev. 92 2002 1138 1151
13. R. Christensen Log-Linear Models and Logistic Regression Second edition 1997 Springer-Verlag New York
14. T.M. Cover, and J.A. Thomas Elements of Information Theory 1991 Wiley New York
15. R. Elliott, C.D. Frith, and R.J. Dolan Differential neural response to positive and negative feedback in planning and guessing tasks Neuropsychologia 35 1997 1395 1404
16. R. Elliott, J.L. Newman, O.A. Longe, and J.F.W. Deakin Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study J. Neurosci. 23 2003 303 307
17. I. Erev, and A.E. Roth Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria Am. Econ. Rev. 88 1998 848 881
18. S. Ghirlanda, and G. Vallortigara The evolution of brain lateralization: a game-theoretic analysis of population structure Proc. R. Soc. Lond., B 271 2004 853 857
19. P.W. Glimcher The neurobiology of visual-saccadic decision making Annu. Rev. Neurosci. 26 2003 133 179
20. J.I. Gold, and M.N. Shadlen Neural computations that underlie decisions about sensory stimuli Trends Neurosci. 5 2001 10 16
21. R.J. Herrnstein The Matching Law: Papers in Psychology and Economics 1997 Harvard Univ. Press Cambridge
22. T. Ikeda, and O. Hikosaka Reward-dependent gain and bias of visual responses in primate superior colliculus Neuron 39 2003 693 700
23. S. Ito, V. Stuphorn, J.W. Brown, and J.D. Schall Performance monitoring by the anterior cingulate cortex during saccade countermanding Science 302 2003 120 122
24. J.H. Kagel, and A.E. Roth Handbook of Experimental Economics 1995 Princeton Univ Press Princeton
25. J.H. Kagel, R.C. Battalio, and L. Green Economic Choice Theory: An Experimental Analysis of Animal Behavior 1995 Cambridge Univ. Press Cambridge
26. J. Kahan, and D.J. Goehring Responsiveness in two-person zero-sum games Behav. Sci. 18 1973 27 33
27. D. Kahneman, and A. Tversky Choices, Values, and Frames 2000 Cambridge Univ. Press Cambridge
28. R. Kawagoe, Y. Takikawa, and O. Hikosaka Expectation of reward modulates cognitive signals in the basal ganglia Nat. Neurosci. 1 1998 411 416
29. W. Lee Decision Theory and Human Behavior 1971 Wiley New York
30. M.I. Leon, and M.N. Shadlen Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque Neuron 24 1999 415 425
31. B. Liberman Experimental studies of conflict in some two-person and three-person games J.H. Criswell H. Solomon P. Suppes Mathematical Methods in Small Group Processes 1962 Stanford Univ. Press Stanford 203 220
32. M.L. Littman Markov games as a framework for multi-agent reinforcement learning Machine Learning: Proc. 11th Int. Conf. 1994 Morgan Kaufmann San Francisco 157 163
33. L.L. Lopes, and G.C. Oden Distinguishing between random and nonrandom events J. Exper. Psychol., Learn., Mem., Cogn. 13 1987 392 400
34. D. Malcolm, and B. Liberman The behavior of responsive individuals playing a two-person, zero-sum game requiring the use of mixed strategies Psychon. Sci. 2 1965 373 374
35. A.N. McCoy, J.C. Crowley, G. Haghighian, H.L. Dean, and M.L. Platt Saccade reward signals in posterior cingulate cortex Neuron 40 2003 1031 1040
36. R.D. McKelvey, and T.R. Palfrey Quantal response equilibria for normal form games Games Econ. Behav. 7 1996 6 38
37. D.M. Messick Interdependent decision strategies in zero-sum games: a computer-controlled study Behav. Sci. 12 1967 33 48
38. G.A. Miller Note on the bias of information estimates H. Quastler Information Theory in Psychology 1955 Free Press Glencoe 95 100
39. D. Mookherjee, and B. Sopher Learning behavior in an experimental matching pennies game Games Econ. Behav. 7 1994 62 91
40. J.F. Nash Equilibrium points in n-person games Proc. Natl. Acad. Sci. 36 1950 48 49
41. A. Neuringer Can people behave "randomly?": the role of feedback J. Exp. Psychol. Gen. 115 1986 62 75
42. R.S. Nickerson The production and perception of randomness Psychol. Rev. 109 2002 330 357
43. J. Ochs Games with unique, mixed strategy equilibria: an experimental study Games Econ. Behav. 10 1995 202 217
44. J. O'Doherty, M.L. Kringelbach, E.T. Rolls, J. Hornak, and C. Andrews Abstract reward and punishment representation in the human orbitofrontal cortex Nat. Neurosci. 4 2001 95 102
45. B. O'Neill Nonmetric test of the minimax theory of two-person zerosum games Proc. Natl. Acad. Sci. U. S. A. 84 1987 2106 2109
46. B. O'Neill Comments on Brown and Rosenthal's reexamination Econometrica 59 1991 503 507
47. I. Palacios-Huerta Professionals play minimax Rev. Econ. Stud. 70 2003 395 415
48. M.L. Platt, and P.W. Glimcher Neural correlates of decision variables in parietal cortex Nature 400 1999 233 238
49. N. Ramnani, and R.C. Miall Instructed delay activity in the human prefrontal cortex is modulated by monetary reward expectation Cereb. Cortex 13 2003 318 327
50. A. Rapoport, and R.B. Boebel Mixed strategies in strictly competitive games: a further test of the minimax hypothesis Games Econ. Behav. 4 1992 261 283
51. A. Rapoport, and D.V. Budescu Generation of random series in two-person strictly competitive games J. Exp. Psychol. Gen. 121 1992 352 363
52. A. Rapoport, and D.V. Budescu Randomization in individual choice behavior Psychol. Rev. 104 1997 603 617
53. M.R. Roesch, and C.R. Olson Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex J. Neurophysiol. 90 2003 1766 1789
54. R.D. Rogers, A.M. Owen, H.C. Middleton, E.J. Williams, J.D. Picard, B.J. Sakakian, and T.W. Robbins Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex J. Neurosci. 19 1999 9029 9038
55. E.T. Rolls Brain and Emotion 1999 Oxford Univ. Press Oxford
56. R. Romo, and E. Salinas Flutter discrimination: neural codes, perception, memory and decision making Nat. Rev., Neurosci. 4 2003 203 218
57. R. Sarin, and F. Vahid Predicting how people play games: a simple dynamic model of choice Games Econ. Behav. 34 2001 104 122
58. J.D. Schall Neural correlates of decision processes: neural and mental chronometry Curr. Opin. Neurobiol. 13 2003 182 186
59. W. Schultz Predictive reward signal of dopamine neurons J. Neurophysiol. 80 1998 1 27
60. W. Schultz Getting formal with dopamine and reward Neuron 36 2002 241 263
61. W. Schultz Neural coding of basic reward terms of animal learning theory, game theory, microeconomics, and behavioral ecology Curr. Opin. Neurobiol. 14 2004 139 147
62. J.M. Shachat Mixed strategy play and the minimax hypothesis J. Econ. Theory 104 2002 189 226
63. M. Shidara, and B.J. Richmond Anterior cingulate: single neuronal signals related to degree of reward expectancy Science 296 2002 1709 1711
64. G.W. Snedecor, and W.G. Cochran Statistical Methods Eight edition 1989 Iowa State Univ. Press Ames
65. V. Stuphorn, T.L. Taylor, and J.D. Schall Performance monitoring by the supplementary eye field Nature 408 2000 857 860
66. L.P. Sugrue, G.S. Corrado, and W.T. Newsome Matching behavior and the representation of value in parietal cortex Science 304 2004 1782 1787
67. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction 1998 MIT Press Cambridge
68. J.C. Touhey Decision processes, expectations, and adoption of strategies in zero-sum games Hum. Relat. 27 1974 813 824
69. L. Tremblay, and W. Schultz Reward-related neuronal activity during go-no go task performance in primate orbitofrontal cortex J. Neurophysiol. 83 2000 1864 1876
70. G.S. Tune A brief survey of variables that influence random generation Percept. Mot. Skills 18 1964 705 710
71. J. von Neumann, and O. Morgenstern The Theory of Games and Economic Behavior 1944 Princeton Univ. Press Princeton
72. W.A. Wagenaar Generation of random sequences of human subjects: critical survey of literature Psychol. Bull. 77 1972 65 72
73. M. Walker, and J. Wooders Minimax play at Wimbledon Am. Econ. Rev. 91 2001 1521 1538
74. M. Watanabe Reward expectancy in primate prefrontal cortex Nature 382 1996 629 632