Effects of reward expectancy on sequential eye movements in monkeys

Neural Networks

Volumn 19, Issue 8, 2006, Pages 1181-1191

  Sohn, Jeong woo ^a   Lee, Daeyeol ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

Author keywords

Basal ganglia; Cortex; Discounting; Learning; Reaction time; Value function

Indexed keywords

BASAL GANGLIA; CORTEX; DISCOUNTING; VALUE FUNCTION;

BIODIVERSITY; ERROR ANALYSIS; HUMAN REACTION TIME; LEARNING SYSTEMS;

EYE MOVEMENTS;

ANALYTICAL ERROR; ANIMAL EXPERIMENT; ARTICLE; BEHAVIOR; CONTROLLED STUDY; EXPECTATION; EYE FIXATION; EYE MOVEMENT; EYE MOVEMENT CONTROL; EYE TRACKING; LEARNING; MONKEY; MOTIVATION; NONHUMAN; PRIORITY JOURNAL; RESPONSE TIME; REWARD; SACCADIC EYE MOVEMENT; VISUAL STIMULATION;

ANIMALS; BEHAVIOR, ANIMAL; EYE MOVEMENTS; LEARNING; MACACA MULATTA; MALE; MOTIVATION; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REGRESSION ANALYSIS; REWARD; VISUAL FIELDS; VISUAL PERCEPTION;

EID: 33749077275     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2006.04.005     Document Type: Article

References (63)

1. Averbeck B.B., Chafee M.V., Crowe D.A., and Georgopoulos A.P. Neural activity in prefrontal cortex during copying geometrical shapes. I. Single cell encode shape, sequence, and metric parameters. Experimental Brain Research 150 (2003) 127-141
2. Averbeck B.B., Sohn J.-W., and Lee D. Activity in prefrontal cortex during dynamic selection of action sequences. Nature Neuroscience 9 (2006) 276-282
3. Barone P., and Joseph J.P. Prefrontal cortex and spatial sequencing in macaque monkey. Experimental Brain Research 78 (1989) 447-464
4. Barraclough D.J., Conroy M.L., and Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nature Neuroscience 7 (2004) 404-410
5. Becker W., and Jürgens R. An analysis of the saccadic system by means of double step stimuli. Vision Research 19 (1979) 967-983
6. Berns G.S., Cohen J.D., and Mintun M.A. Brain regions responsitive to novelty in the absence of awareness. Science 276 (1997) 1272-1275
7. Bonem M., and Crossman E.K. Elucidating the effects of reinforcement magnitude. Psychological Bulletin 104 (1988) 348-362
8. Bowman E.M., Aigner T.G., and Richmond B.J. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. Journal of Neurophysiology 75 (1996) 1061-1073
9. Chung S.H., and Herrnstein R.J. Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior 10 (1967) 67-74
10. Cisek P., and Kalaska J.F. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45 (2005) 801-814
11. Clower W.T., and Alexander G.E. Movement sequence-related activity reflecting numerical order of components in supplementary and presupplementary motor areas. Journal of Neurophysiology 80 (1998) 1562-1566
12. Crespi L.P. Quantitative variation of incentive and performance in the white rat. American Journal of Psychology 55 (1942) 467-517
13. Davenport J.W. The interaction of magnitude and delay of reinforcement in spatial discrimination. Journal of Comparative and Physiological Psychology 55 (1962) 267-273
14. Doyon J., Song A.W., Karni A., Lalonde F., Adams M.M., and Ungerleider L.G. Experience-dependent changes in cerebellar contributions to motor sequence learning. Proceedings of the National Academy of Sciences of the United States of America 99 (2002) 1017-1022
15. Ferster C.B., and Skinner B.F. Schedules of reinforcement (1957), Appleton-Century-Crofts, New York
16. Grafton S.T., Hazeltine E., and Ivry R. Functional mapping of sequence learning in normal humans. Journal of Cognitive Neuroscience 7 (1995) 497-510
17. Hamilton E.L. The effect of delayed incentive on the hunger drive in the white rat. Genetic Psychology Monograph 5 (1929) 131-207
18. Harker G.S. Delay of reward and performance of an instrumental response. Journal of Experimental Psychology 51 (1956) 303-310
19. Hazeltine E., Grafton S.T., and Ivry R. Attention and stimulus characteristics determine the locus of motor-sequence encoding: a PET study. Brain 120 (1997) 123-140
20. Hull C.L. Principles of behavior (1943), Appleton-Century-Crofts, New York
21. Isoda M., and Tanji J. Contrasting neuronal activity in the supplementary and frontal eye fields during temporal organization of multiple saccades. Journal of Neurophysiology 90 (2003) 3054-3065
22. Isoda M., and Tanji J. Participation of the primate presupplementary motor area in sequencing multiple saccades. Journal of Neurophysiology 92 (2004) 653-659
23. Kawagoe R., Takikawa Y., and Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neuroscience 1 (1998) 411-416
24. Kawashima R., Tanji J., Okada K., Sugiura M., Sato K., Kinomura S., et al. Oculomotor sequence learning: a positron emission tomography study. Experimental Brain Research 122 (1998) 1-8
25. Kermadi I., and Joseph J.P. Activity in the caudate nucleus of monkey during spatial sequencing. Journal of Neurophysiology 74 (1995) 911-933
26. Lee D. Effects of exogenous and endogenous attention on visually guided hand movements. Cognitive Brain Research 8 (1999) 143-156
27. Lee D. Behavioral context and coherent oscillations in the supplementary motor area. Journal of Neuroscience 24 (2004) 4453-4459
28. Lee D., Conroy M.L., McGreevy B.P., and Barraclough D.J. Reinforcement learning and decision making in monkeys during a competitive game. Cognitive Brain Research 22 (2004) 45-58
29. Lee D., and Quessy S. Activity in the supplementary motor area related to learning and performance during a sequential visuomotor task. Journal of Neurophysiology 89 (2003) 1039-1056
30. Leon M.I., and Shadlen M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24 (1999) 415-425
31. Logan F.A. The role of delay of reinforcement in determining reaction potential. Journal of Experimental Psychology 43 (1952) 393-399
32. Logan F.A. Incentive: how the conditions of reinforcement affect the performance of rats (1960), Yale University Press, New Haven
33. Logan F.A. Decision making by rats: delay versus amount of reward. Journal of Comparative and Physiological Psychology 59 (1965) 1-12
34. Marquardt D.W. Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics 12 (1970) 591-612
35. McPeek R.M., and Keller E.L. Short-term priming, concurrent processing, and saccade curvature during a target selection task in the monkey. Vision Research 41 (2001) 785-800
36. Nissen M.J., and Bullemer P. Attentional requirements of learning: evidence from performance measures. Cognitive Psychology 19 (1987) 1-32
37. Pascual-Leone A., Grafman J., and Hallett M. Modulation of cortical motor output maps during development of implicit and explicit knowledge. Science 263 (1994) 1287-1289
38. Pellizzer G., and Hedges J.H. Motor planning: effect of directional uncertainty with discrete spatial cues. Experimental Brain Research 150 (2003) 276-289
39. Perin C.T. A quantitative investigation of the delay-of-reinforcement gradient. Journal of Experimental Psychology 32 (1943) 37-51
40. Platt M.L., and Glimcher P.W. Neural correlates of decision variables in parietal cortex. Nature 400 (1999) 233-238
41. Procyk E., Dominey P.F., Amiez C., and Joseph J.P. The effects of sequence structure and reward schedule on serial reaction time learning in the monkey. Cognitive Brain Research 9 (2000) 239-248
42. Procyk E., and Joseph J.P. Characterization of serial order encoding in the monkey anterior cingulate sulcus. European Journal of Neuroscience 14 (2001) 1041-1046
43. Rauch S.L., Savage C.R., Brown H.D., Curran T., Alpert N.M., Kendrick A., et al. A PET investigation of implicit and explicit sequence learning. Human Brain Mapping 3 (1995) 271-286
44. Rauch S.L., Whalen P.J., Savage C.R., Curran T., Kendrick A., Brown H.D., et al. Striatal recruitment during an implicit sequence learning task as measured by functional magnetic resonance imaging. Human Brain Mapping 5 (1997) 124-132
45. Renner K.E. Influence of deprivation and availability of goal box cues on the temporal gradient of reinforcement. Journal of Comparative and Physiological Psychology 56 (1963) 101-104
46. Roesch M.R., and Olson C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. Journal of Neurophysiology 90 (2003) 1766-1789
47. Ryan T.P. Modern regression methods (1996), Wiley, New York
48. Sakai K., Ramnani N., and Passingham R.E. Learning of sequences of finger movements and timing: frontal lobe and action-oriented representation. Journal of Neurophysiology 88 (2002) 2035-2046
49. Samejima K., Ueda Y., Doya K., and Kimura M. Representation of action-specific reward values in the striatum. Science 310 (2005) 1337-1340
50. Schendan H.E., Searl M.M., Melrose R.J., and Stern C.E. An fMRI study of the role of the medial temporal lobe in implicit and explicit sequence learning. Neuron 37 (2003) 1013-1025
51. Seidler R.D., Purushotham A., Kim S.G., Ugurbil K., Willingham D., and Ashe J. Cerebellum activation associated with performance change but not motor learning. Science 296 (2002) 2043-2046
52. Sgro J.A., Dyal J.A., and Anastasio E.J. Effects of constant delay of reinforcement on acquisition asymptote and resistance to extinction. Journal of Experimental Psychology 73 (1967) 634-636
53. Shidara M., and Richmond B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296 (2002) 1709-1711
54. Shima K., and Tanji J. Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. Journal of Neurophysiology 84 (2000) 2148-2160
55. Simon H.A. Rational choice and the structure of environment. Psychological Review 63 (1956) 129-138
56. Sugrue L.P., Corrado G.S., and Newsome W.T. Matching behavior and the representation of value in the parietal cortex. Science 304 (2004) 1782-1787
57. Sutton R.S., and Barto A.G. Reinforcement learning: an introduction (1998), MIT Press, Cambridge
58. Takikawa Y., Kawagoe R., Itoh H., Nakahara H., and Hikosaka O. Modulation of saccadic eye movements by predicted reward outcome. Experimental Brain Research 142 (2002) 284-291
59. Thorndike E.L. Animal intelligence: experimental studies (1911), MacMillan, New York
60. Watanabe M., Cromwell H.C., Tremblay L., Hollerman J.R., Hikosaka K., and Schultz W. Behavioral reactions reflecting differential reward expectations in monkeys. Experimental Brain Research 140 (2001) 511-518
61. Willingham D.B., Salidis J., and Gabrieli J.D.E. Direct comparison of neural systems mediating conscious and unconscious skill learning. Journal of Neurophysiology 88 (2002) 1451-1460
62. Wolfe J.B. The effect of delayed reward upon learning in the white rat. Journal of Comparative Psychology 17 (1934) 1-21
63. Zeaman D. Response latency as a function of the amount of reinforcement. Journal of Experimental Psychology 39 (1949) 466-483