Dopamine neurons encode errors in predicting movement trigger occurrence

Journal of Neurophysiology

Volumn 113, Issue 4, 2015, Pages 1110-1123

  Pasquereau, Benjamin ^a   Turner, Robert S ^a  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

Author keywords

Anticipation; Dopamine; Hazard rate; Prediction error

Indexed keywords

ANIMAL EXPERIMENT; ANTICIPATION; ARTICLE; CONTROLLED STUDY; DOPAMINERGIC NERVE CELL; FEMALE; HAPLORHINI; MOVEMENT (PHYSIOLOGY); NONHUMAN; PRIORITY JOURNAL; REACTION TIME; SUBSTANTIA NIGRA PARS COMPACTA; TASK PERFORMANCE; ACTION POTENTIAL; ANIMAL; CYTOLOGY; HAND; INNERVATION; MALE; MOTOR PERFORMANCE; PHYSIOLOGY; RHESUS MONKEY; SUBSTANTIA NIGRA;

ACTION POTENTIALS; ANIMALS; ANTICIPATION, PSYCHOLOGICAL; DOPAMINERGIC NEURONS; FEMALE; HAND; MACACA MULATTA; MALE; MOTOR SKILLS; MOVEMENT; REACTION TIME; SUBSTANTIA NIGRA;

EID: 84923197672     PISSN: 00223077     EISSN: 15221598     Source Type: Journal    
DOI: 10.1152/jn.00401.2014     Document Type: Article

References (88)

1. Alegria J. Sequential effects of foreperiod duration as a function of the frequency of foreperiod repetitions. J Mot Behav 7: 243–250, 1975.
2. Bayer HM, Glimcher PW. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47: 129–141, 2005.
3. Benecke R, Rothwell JC, Dick JP, Day BL, Marsden CD. Performance of simultaneous movements in patients with Parkinson’s disease. Brain 109: 739–757, 1986.
4. Berridge KC, Robinson TE. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28: 309–369, 1998.
5. Bestmann S, Harrison LM, Blankenburg F, Mars RB, Haggard P, Friston KJ, Rothwell JC. Influence of uncertainty and surprise on human corticospinal excitability during preparation for action. Curr Biol 18: 775–780, 2008.
6. Bouret S, Ravel S, Richmond BJ. Complementary neural correlates of motivation in dopaminergic and noradrenergic neurons of monkeys. Front Behav Neurosci 6: 40, 2012.
7. Bromberg-Martin ES, Matsumoto M, Hikosaka O. Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron 67: 144–155, 2010a.
8. Bromberg-Martin ES, Matsumoto M, Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68: 815–834, 2010b.
9. Carroll CA, Boggs J, O’Donnell BF, Shekhar A, Hetrick WP. Temporal processing dysfunction in schizophrenia. Brain Cogn 67: 150–161, 2008.
10. Carroll CA, O’Donnell BF, Shekhar A, Hetrick WP. Timing dysfunctions in schizophrenia span from millisecond to several-second durations. Brain Cogn 70: 181-190, 2009.
11. Cheng RK, Ali YM, Meck WH. Ketamine “unlocks” the reduced clock-speed effects of cocaine following extended training: evidence for dopamineglutamate interactions in timing and time perception. Neurobiol Learn Mem 88: 149–159, 2007.
12. Correa A, Lupianez J, Tudela P. Attentional preparation based on temporal expectancy modulates processing at the perceptual level. Psychon Bull Rev 12: 328–334, 2005.
13. Coull JT, Cheng RK, Meck WH. Neuroanatomical and neurochemical substrates of timing. Neuropsychopharmacology 36: 3–25, 2011.
14. Crammond DJ, Kalaska JF. Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity. J Neurophysiol 84: 986–1005, 2000.
15. DeLong MR, Crutcher MD, Georgopoulos AP. Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J Neurosci 3: 1599–1606, 1983.
16. Drew MR, Fairhurst S, Malapani C, Horvitz JC, Balsam PD. Effects of dopamine antagonists on the timing of two intervals. Pharmacol Biochem Behav 75: 9–15, 2003.
17. Elithorn A, Lawrence C. Central inhibition–some refractory observations. Q J Exp Psychol 7: 116–127, 1955.
18. Elvevag B, McCormack T, Gilbert A, Brown GD, Weinberger DR, Goldberg TE. Duration judgements in patients with schizophrenia. Psychol Med 33: 1249–1261, 2003.
19. Fiorillo CD, Newsome WT, Schultz W. The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11: 966–973, 2008.
20. Fiorillo CD, Song MR, Yun SR. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33: 4710–4725, 2013.
21. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898–1902, 2003.
22. Freeman JS, Cody FW, Schady W. The influence of external timing cues upon the rhythm of voluntary movements in Parkinson’s disease. J Neurol Neurosurg Psychiatry 56: 1078–1084, 1993.
23. Genovesio A, Tsujimoto S, Wise SP. Neuronal activity related to elapsed time in prefrontal cortex. J Neurophysiol 95: 3281–3285, 2006.
24. Ghose GM, Maunsell JH. Attentional modulation in visual cortex depends on task timing. Nature 419: 616–620, 2002.
25. Gibbon J. Scalar expectancy-theory and Webers law in animal timing. Psychol Rev 84: 279–325, 1977.
26. Gibbon J, Malapani C, Dale CL, Gallistel CR. Toward a neurobiology of temporal cognition: advances and challenges. Curr Opin Neurobiol 7: 170–184, 1997.
27. Hanes DP, Thompson KG, Schall JD. Relationship of presaccadic activity in frontal eye field and supplementary eye field to saccade initiation in macaque: Poisson spike train analysis. Exp Brain Res 103: 85–96, 1995.
28. Harrington DL, Haaland KY, Hermanowicz N. Temporal processing in the basal ganglia. Neuropsychology 12: 3–12, 1998.
29. Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1: 304–309, 1998.
30. Horvitz JC. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96: 651–656, 2000.
31. Janssen P, Shadlen MN. A representation of the hazard rate of elapsed time in macaque area LIP. Nat Neurosci 8: 234–241, 2005.
32. Jaramillo S, Zador AM. The auditory cortex mediates the perceptual effects of acoustic temporal expectation. Nat Neurosci 14: 246–251, 2011.
33. Jin X, Costa RM. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466: 457–462, 2010.
34. Joshua M, Adler A, Mitelman R, Vaadia E, Bergman H. Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J Neurosci 28: 11673–11684, 2008.
35. Kiyatkin EA. Morphine-induced modification of the functional properties of ventral tegmental area neurons in conscious rat. Int J Neurosci 41: 57–70, 1988.
36. Kobayashi S, Schultz W. Influence of reward delays on responses of dopamine neurons. J Neurosci 28: 7837–7846, 2008.
37. Legendy CR, Salcman M. Bursts and recurrences of bursts in the spike trains of spontaneously active striate cortex neurons. J Neurophysiol 53: 926–939, 1985.
38. Ljungberg T, Apicella P, Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67: 145–163, 1992.
39. Luce RD. Response times: their role in inferring elementary mental organization. New York: Oxford University Press, 1986.
40. Malapani C, Rakitin B, Levy R, Meck WH, Deweer B, Dubois B, Gibbon J. Coupled temporal memories in Parkinson’s disease: a dopamine-related dysfunction. J Cogn Neurosci 10: 316–331, 1998.
41. Martin RF, Bowden DM. A stereotaxic template atlas of the macaque brain for digital imaging and quantitative neuroanatomy. Neuroimage 4: 119–150, 1996.
42. Matell MS, Bateson M, Meck WH. Single-trials analyses demonstrate that increases in clock speed contribute to the methamphetamine-induced horizontal shifts in peak-interval timing functions. Psychopharmacology 188: 201–212, 2006.
43. Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459: 837–841, 2009.
44. Mauk MD, Buonomano DV. The neural basis of temporal processing. Annu Rev Neurosci 27: 307–340, 2004.
45. Mauritz KH, Wise SP. Premotor cortex of the rhesus-monkey – neuronalactivity in anticipation of predictable environmental events. Exp Brain Res 61: 229–244, 1986.
46. Meck WH, Cheng RK, MacDonald CJ, Gainetdinov RR, Caron MG, Cevik MO. Gene dose dependent effects of methamphetamine on interval timing in dopamine-transporter knockout mice. Neuropharmacology 62: 1221–1229, 2012.
47. Miocinovic S, Noecker AM, Maks CB, Butson CR, McIntyre CC. Cicerone: stereotactic neurophysiological recording and deep brain stimulation electrode placement software system. Acta Neurochir Suppl 97: 561–567, 2007.
48. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936–1947, 1996.
49. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9: 1057–1063, 2006.
50. Nagasaki H, Nakamura R, Taniguchi R. Disturbances of rhythm formation in patients with Parkinson’s disease: part II. a forced oscillation model. Percept Mot Skills 46: 79–87, 1978.
51. Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O. Dopamine neurons can represent context-dependent prediction error. Neuron 41: 269–280, 2004.
52. Narayanan NS, Land BB, Solder JE, Deisseroth K, DiLeone RJ. Prefrontal D1 dopamine signaling is required for temporal control. Proc Natl Acad Sci USA 109: 20726–20731, 2012.
53. Niemi P, Naatanen R. Foreperiod and simple reaction-time. Psychol Bull 89:133–162, 1981.
54. Niki H, Watanabe M. Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171: 213–224, 1979.
55. Nomoto K, Schultz W, Watanabe T, Sakagami M. Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30: 10692–10702, 2010.
56. O’Boyle DJ, Freeman JS, Cody FW. The accuracy and precision of timing of self-paced, repetitive movements in subjects with Parkinson’s disease. Brain 119: 51–70, 1996.
57. Pasquereau B, Turner RS. Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci 33: 8288–8300, 2013.
58. Pasquereau B, Turner RS. Primary motor cortex of the parkinsonian monkey: differential effects on the spontaneous activity of pyramidal tract-type neurons. Cereb Cortex 21: 1362–1378, 2011.
59. Pastor MA, Jahanshahi M, Artieda J, Obeso JA. Performance of repetitive wrist movements in Parkinsons-disease. Brain 115: 875–891, 1992.
60. Premereur E, Vanduffel W, Janssen P. Functional heterogeneity of macaque lateral intraparietal neurons. J Neurosci 31: 12307–12317, 2011.
61. Puryear CB, Kim MJ, Mizumori SJ. Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent. Behav Neurosci 124: 234–247, 2010.
62. Rammsayer T, Classen W. Impaired temporal discrimination in Parkinson’s disease: temporal processing of brief durations as an indicator of degeneration of dopaminergic neurons in the basal ganglia. Int J Neurosci 91: 45–55, 1997.
63. Rammsayer TH. On dopaminergic modulation of temporal information processing. Biol Psychol 36: 209–222, 1993.
64. Redgrave P, Gurney K. The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci 7: 967–975, 2006.
65. Renoult L, Roux S, Riehle A. Time is a rubberband: neuronal activity in monkey motor cortex in relation to time estimation. Eur J Neurosci 23: 3098–3108, 2006.
66. Roesch MR, Calu DJ, Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10: 1615–1624, 2007.
67. Romo R, Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63: 592–606, 1990.
68. Roux S, Mackay WA, Riehle A. The pre-movement component of motor cortical local field potentials reflects the level of expectancy. Behav Brain Res 169: 335–351, 2006.
69. Rubia K, Halari R, Christakou A, Taylor E. Impulsiveness as a timing disturbance: neurocognitive abnormalities in attention-deficit hyperactivity disorder during temporal processes and normalization with methylphenidate. Philos Trans R Soc Lond B Biol Sci 364: 1919–1931, 2009.
70. Salamone JD, Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137: 3–25, 2002.
71. Salinas E, Romo R. Conversion of sensory signals into motor commands in primary motor cortex. J Neurosci 18: 499–511, 1998.
72. Satoh T, Nakai S, Sato T, Kimura M. Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23: 9913–9923, 2003.
73. Schultz W. Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 57: 87–115, 2006.
74. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1–27, 1998.
75. Schultz W, Apicella P, Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13: 900–913, 1993.
76. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science 275: 1593–1599, 1997.
77. Schultz W, Romo R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63: 607–624, 1990.
78. Schultz W, Ruffieux A, Aebischer P. The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51: 377–387, 1983.
79. Sutton RS, Barto AG. Reinforcement Learning: an Introduction. Cambridge, MA: MIT Press, 1998, p. 28.
80. Tanji J, Evarts EV. Anticipatory activity of motor cortex neurons in relation to direction of an intended movement. J Neurophysiol 39: 1062–1068, 1976.
81. Tobler PN, Fiorillo CD, Schultz W. Adaptive coding of reward value by dopamine neurons. Science 307: 1642–1645, 2005.
82. Totah NK, Kim Y, Moghaddam B. Distinct prestimulus and poststimulus activation of VTA neurons correlates with stimulus detection. J Neurophysiol 110: 75–85, 2013.
83. Tsunoda Y, Kakei S. Anticipation of future events improves the ability to estimate elapsed time. Exp Brain Res 214: 323–334, 2011.
84. Tsunoda Y, Kakei S. Reaction time changes with the hazard rate for a behaviorally relevant event when monkeys perform a delayed wrist movement task. Neurosci Lett 433: 152–157, 2008.
85. Turner RS, Anderson ME. Pallidal discharge related to the kinematics of reaching movements in two dimensions. J Neurophysiol 77: 1051–1074, 1997.
86. Tysk L. Estimation of time and the subclassification of schizophrenic disorders. Percept Mot Skills 57: 911–918, 1983.
87. Vallesi A, Shallice T, Walsh V. Role of the prefrontal cortex in the foreperiod effect: TMS evidence for dual mechanisms in temporal preparation. Cereb Cortex 17: 466–474, 2007.
88. Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci 5: 483–494, 2004.