Beyond reward prediction errors: The role of dopamine in movement kinematics

Frontiers in Integrative Neuroscience

Volumn 9, Issue MAY, 2015, Pages

  Barter, Joseph W ^a   Li, Suellen ^a   Lu, Dongye ^a   Bartholomew, Ryan A ^a   Rossi, Mark A ^a   Shoemaker, Charles T ^a   Salas Meza, Daniel ^a   Gaidis, Erin ^a   Yin, Henry H ^a  

^a DUKE UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Basal ganglia; Dopamine; Movement; Reward prediction error; Striatum; Substantia nigra

Indexed keywords

CHANNELRHODOPSIN 2 PROTEIN; DOPAMINE; RHODOPSIN; SUCROSE; UNCLASSIFIED DRUG;

ACCELERATION; ANIMAL EXPERIMENT; APPETITE; ARTICLE; AVERSIVE BEHAVIOR; BASAL GANGLION; CELL ACTIVATION; CONTROLLED STUDY; CORRELATIONAL STUDY; FEMALE; KINEMATICS; MALE; MOTOR CONTROL; MOUSE; MOVEMENT (PHYSIOLOGY); NERVE CELL; NIGRONEOSTRIATAL SYSTEM; NONHUMAN; OPTOGENETICS; PHOTOSTIMULATION; PREDICTION; PROTEIN EXPRESSION; PROTEIN FUNCTION; REWARD; SINGLE UNIT ACTIVITY; SUBSTANTIA NIGRA PARS COMPACTA; VELOCITY;

EID: 84930204365     PISSN: 16625145     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnint.2015.00039     Document Type: Article

References (76)

1. Alchian, A. A., and Allen, W. R. (1977). Exchange and Production: Competition, Coordination, and Control. Belmont, CA: Wadsworth Publishing Company.
2. Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357-381. doi: 10.1146/annurev.ne.09.030186.002041
3. Barter, J. W., Castro, S., Sukharnikova, T., Rossi, M. A., and Yin, H. H. (2014). The role of the substantia nigra in posture control. Eur. J. Neurosci. 39, 1465-1473. doi: 10.1111/ejn.12540
4. Barter, J. W., Li, S., Sukharnikova, T., Rossi, M. A., Bartholomew, R. A., and Yin, H. H. (2015). Basal ganglia outputs map instantaneous position coordinates during behavior. J. Neurosci. 35, 2703-2716. doi: 10.1523/JNEUROSCI.3245-14.2015
5. Boyden, E. S., Zhang, F., Bamberg, E., Nagel, G., and Deisseroth, K. (2005). Millisecond-timescale, genetically targeted optical control of neural activity. Nat. Neurosci. 8, 1263-1268. doi: 10.1038/nn1525
6. Brown, P. (2006). "Bad oscillations in Parkinson's disease," in Parkinson's Disease and Related Disorders (Viena: Springer), 27-30.
7. Cagniard, B., Beeler, J. A., Britt, J. P., McGehee, D. S., Marinelli, M., and Zhuang, X. (2006). Dopamine scales performance in the absence of new learning. Neuron 51, 541-547. doi: 10.1016/j.neuron.2006.07.026
8. Cannon, C. M., and Palmiter, R. D. (2003). Reward without dopamine. J. Neurosci. 23, 10827-10831.
9. Carelli, R. M., and West, M. O. (1991). Representation of the body by single neurons in the dorsolateral striatum of the awake, unrestrained rat. J. Comp. Neurol. 309, 231-249. doi: 10.1002/cne.903090205
10. Chuhma, N., Tanaka, K. F., Hen, R., and Rayport, S. (2011). Functional connectome of the striatal medium spiny neuron. J. Neurosci. 31, 1183-1192. doi: 10.1523/JNEUROSCI.3833-10.2011
11. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., and Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85-88. doi: 10.1038/nature10754
12. Connelly, W. M., Schulz, J. M., Lees, G., and Reynolds, J. N. (2010). Differential short-term plasticity at convergent inhibitory synapses to the substantia nigra pars reticulata. J. Neurosci. 30, 14854-14861. doi: 10.1523/JNEUROSCI.3895-10.2010
13. Costa, R. M., Lin, S. C., Sotnikova, T. D., Cyr, M., Gainetdinov, R. R., Caron, M. G., et al. (2006). Rapid alterations in corticostriatal ensemble coordination during acute dopamine-dependent motor dysfunction. Neuron 52, 359-369. doi: 10.1016/j.neuron.2006.07.030
14. Cui, G., Jun, S. B., Jin, X., Pham, M. D., Vogel, S. S., Lovinger, D. M., et al. (2013). Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238-242. doi: 10.1038/nature11846
15. de Jesús Aceves, J., Rueda-Orozco, P. E., Hernández, R., Plata, V., Ibañez-Sandoval, O., Galarraga, E., et al. (2011). Dopaminergic presynaptic modulation of nigral afferents: its role in the generation of recurrent bursting in substantia nigra pars reticulata neurons. Front. Syst. Neurosci. 5:6. doi: 10.3389/fnsys.2011.00006
16. DeLong, M. R. (1990). Primate models of movement disorders of basal ganglia origin. Trends Neurosci. 13, 281-285. doi: 10.1016/0166-2236(90)90110-V
17. Dommett, E., Coizet, V., Blaha, C. D., Martindale, J., Lefebvre, V., Walton, N., et al. (2005). How visual stimuli activate dopaminergic neurons at short latency. Science 307, 1476-1479. doi: 10.1126/science.1107026
18. Fan, D., Rich, D., Holtzman, T., Ruther, P., Dalley, J. W., Lopez, A., et al. (2011). A wireless multi-channel recording system for freely behaving mice and rats. PLoS ONE 6:e22033. doi: 10.1371/journal.pone.0022033
19. Fan, D., Rossi, M. A., and Yin, H. H. (2012). Mechanisms of action selection and timing in substantia nigra neurons. J. Neurosci. 32, 5534-5548. doi: 10.1523/JNEUROSCI.5924-11.2012
20. Ferrier, D. (1876). The Functions of the Brain. New York, NY: GP Putnam's Sons. doi: 10.1037/12860-000
21. Georgopoulos, A. P., Schwartz, A. B., and Kettner, R. E. (1986). Neuronal population coding of movement direction. Science233, 1416-1419. doi: 10.1126/science.3749885
22. Gerfen, C. R., and Surmeier, D. J. (2011). Modulation of striatal projection systems by dopamine. Annu. Rev. Neurosci. 34, 441-466. doi: 10.1146/annurev-neuro-061010-113641
23. Grace, A. A., and Bunney, B. S. (1984). The control of firing pattern in nigral dopamine neurons: single spike firing. J. Neurosci. 4, 2866-2876.
24. Green, A. M., and Angelaki, D. E. (2010). Internal models and neural computation in the vestibular system. Exp. Brain Res. 200, 197-222. doi: 10.1007/s00221-009-2054-4
25. Haber, S. N., Fudge, J. L., and McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369-2382.
26. Hefti, F., Melamed, E., Sahakian, B. J., and Wurtman, R. J. (1980). Circling behavior in rats with partial, unilateral nigro-striatal lesions: effect of amphetamine, apomorphine, and DOPA. Pharmacol. Biochem. Behav. 12, 185-188. doi: 10.1016/0091-3057(80)90353-6
27. Hikosaka, O. (2007). Basal ganglia mechanisms of reward-oriented eye movement. Ann. N.Y. Acad. Sci. 1104, 229-249. doi: 10.1196/annals.1390.012
28. Isomura, Y., Takekawa, T., Harukuni, R., Handa, T., Aizawa, H., Takada, M., et al. (2013). Reward-modulated motor information in identified striatum neurons. J. Neurosci. 33, 10209-10220. doi: 10.1523/JNEUROSCI.0381-13.2013
29. Jin, X., and Costa, R. M. (2010). Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457-462. doi: 10.1038/nature09263
30. Kaneda, K., Isa, K., Yanagawa, Y., and Isa, T. (2008). Nigral inhibition of GABAergic neurons in mouse superior colliculus. J. Neurosci. 28, 11071-11078. doi: 10.1523/JNEUROSCI.3263-08.2008
31. Kang, Y., and Kitai, S. (1993). A whole cell patch-clamp study on the pacemaker potential in dopaminergic neurons of rat substantia nigra compacta. Neurosci. Res. 18, 209-221. doi: 10.1016/0168-0102(93)90056-V
32. Kim, N., Barter, J. W., Sukharnikova, T., and Yin, H. H. (2014). Striatal firing rate reflects head movement velocity. Eur. J. Neurosci. 40, 3481-3490. doi: 10.1111/ejn.12722
33. Kravitz, A. V., Freeze, B. S., Parker, P. R., Kay, K., Thwin, M. T., Deisseroth, K., et al. (2010). Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622-626. doi: 10.1038/nature09159
34. Lammel, S., Steinberg, E. E., Földy, C., Wall, N. R., Beier, K., Luo, L., et al. (2015). Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron 85, 429-438. doi: 10.1016/j.neuron.2014.12.036
35. Leblois, A., Wendel, B. J., and Perkel, D. J. (2010). Striatal dopamine modulates basal ganglia output and regulates social context-dependent behavioral variability through D1 receptors. J. Neurosci. 30, 5730-5743. doi: 10.1523/JNEUROSCI.5974-09.2010
36. Lein, E. S., Hawrylycz, M. J., Ao, N., Ayres, M., Bensinger, A., Bernard, A., et al. (2006). Genome-wide atlas of gene expression in the adult mouse brain. Nature 445, 168-176. doi: 10.1038/nature05453
37. Ljungberg, T., Apicella, P., and Schultz, W. (1992). Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67, 145-163.
38. Madisen, L., Mao, T., Koch, H., Zhuo, J. M., Berenyi, A., Fujisawa, S., et al. (2012). A toolbox of Cre-dependent optogenetic transgenic mice for light-induced activation and silencing. Nat. Neurosci. 15, 793-802. doi: 10.1038/nn.3078
39. Masino, T. (1992). Brainstem control of orienting movements: intrinsic coordinate systems and underlying circuitry. Brain Behav. Evol. 40, 98-111. doi: 10.1159/000113906
40. McClure, S. M., Daw, N. D., and Read Montague, P. (2003). A computational substrate for incentive salience. Trends Neurosci. 26, 423-428. doi: 10.1016/S0166-2236(03)00177-2
41. Mountcastle, V. B., Lynch, J., Georgopoulos, A., Sakata, H., and Acuna, C. (1975). Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space. J. Neurophysiol. 38, 871-908.
42. Niv, Y., Daw, N. D., Joel, D., and Dayan, P. (2007). Tonic dopamine: opportunity costs and the control of response vigor.Psychopharmacology 191, 507-520. doi: 10.1007/s00213-006-0502-4
43. Osborne, L. C., Lisberger, S. G., and Bialek, W. (2005). A sensory source for motor variation. Nature 437, 412-416. doi: 10.1038/nature03961
44. Paladini, C. A., Iribe, Y., and Tepper, J. M. (1999). GABA A receptor stimulation blocks NMDA-induced bursting of dopaminergic neurons in vitro by decreasing input resistance. Brain Res. 832, 145-151. doi: 10.1016/S0006-8993(99)01484-5
45. Paladini, C., and Roeper, J. (2014). Generating bursts (and pauses) in the dopamine midbrain neurons. Neuroscience 282, 109-121. doi: 10.1016/j.neuroscience.2014.07.032
46. Pan, W.-X., Brown, J., and Dudman, J. T. (2013). Neural signals of extinction in the inhibitory microcircuit of the ventral midbrain. Nat. Neurosci. 16, 71-78. doi: 10.1038/nn.3283
47. Paninski, L., Fellows, M. R., Hatsopoulos, N. G., and Donoghue, J. P. (2004). Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J. Neurophysiol. 91, 515-532. doi: 10.1152/jn.00587.2002
48. Powers, W. T., Clark, R. K., and McFarland, R. L. (1960). A general feedback theory of human behavior. Percept. Mot. Skills11, 71-88. doi: 10.2466/pms.1960.11.1.71
49. Puryear, C. B., Kim, M. J., and Mizumori, S. J. (2010). Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent. Behav. Neurosci. 124, 234. doi: 10.1037/a0018865
50. Robinson, D. (1981). The use of control systems analysis in the neurophysiology of eye movements. Annu. Rev. Neurosci. 4, 463-503. doi: 10.1146/annurev.ne.04.030181.002335
51. Roesch, M. R., Calu, D. J., and Schoenbaum, G. (2007). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615-1624. doi: 10.1038/nn2013
52. Romo, R., and Schultz, W. (1990). Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol. 63, 592-606.
53. Rossi, M. A., Fan, D., Barter, J. W., and Yin, H. H. (2013a). Bidirectional modulation of substantia nigra activity by motivational state. PLoS ONE 8:e71598. doi: 10.1371/journal.pone.0071598
54. Rossi, M. A., Go, V., Murphy, T., Fu, Q., Morizio, J., and Yin, H. H. (2015). A wirelessly controlled implantable LED system for deep brain optogenetic stimulation. Front. Integr. Neurosci. 9:8. doi: 10.3389/fnint.2015.00008
55. Rossi, M. A., Sukharnikova, T., Hayrapetyan, V. Y., Yang, L., and Yin, H. H. (2013b). Operant self-stimulation of dopamine neurons in the substantia nigra. PLoS ONE 8:e65799. doi: 10.1371/journal.pone.0065799
56. Rossi, M. A., and Yin, H. H. (2015). Elevated dopamine alters consummatory pattern generation and increases behavioral variability during learning. Front. Integr. Neurosci. 9:37. doi: 10.3389/fnint.2015.00037
57. Schultz, W. (1998a). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27.
58. Schultz, W. (1998b). The phasic reward signal of primate dopamine neurons. Adv. Pharmacol. 42, 686-690. doi: 10.1016/S1054-3589(08)60841-8
59. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599. doi: 10.1126/science.275.5306.1593
60. Schultz, W., Ruffieux, A., and Aebischer, P. (1983). The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp. Brain Res. 51, 377-387. doi: 10.1007/BF00237874
61. Sparta, D. R., Stamatakis, A. M., Phillips, J. L., Hovelsø, N., van Zessen, R., and Stuber, G. D. (2012). Construction of implantable optical fibers for long-term optogenetic manipulation of neural circuits. Nat. Protoc. 7, 12-23. doi: 10.1038/nprot.2011.413
62. Taylor, D. M., Tillery, S. I. H., and Schwartz, A. B. (2002). Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829-1832. doi: 10.1126/science.1070291
63. Tepper, J., Martin, L., and Anderson, D. (1995). GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J. Neurosci. 15, 3092-3103.
64. Tepper, J. M., and Lee, C. R. (2007). GABAergic control of substantia nigra dopaminergic neurons. Prog. Brain Res. 160, 189-208. doi: 10.1016/S0079-6123(06)60011-3
65. Todorov, E., and Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226-1235. doi: 10.1038/nn963
66. Wang, D. V., and Tsien, J. Z. (2011a). Convergent processing of both positive and negative motivational signals by the VTA dopamine neuronal populations. PLoS ONE 6:e17047. doi: 10.1371/journal.pone.0017047
67. Wang, D. V., and Tsien, J. Z. (2011b). Conjunctive processing of locomotor signals by the ventral tegmental area neuronal population. PLoS ONE 6:e16528. doi: 10.1371/journal.pone.0016528
68. Wang, S., Tan, Y., Zhang, J.-E., and Luo, M. (2013). Pharmacogenetic activation of midbrain dopamine neurons produces hyperactivity. Neurosci. Bull. 29, 1-8. doi: 10.1007/s12264-013-1327-x
69. White, N. M. (1986). Control of sensorimotor function by dopaminergic nigrostriatal neurons: influence on eating and drinking. Neurosci. Biobehav. Rev. 10, 15-36. doi: 10.1016/0149-7634(86)90030-8
70. Yin, H. H. (2013). "Restoring purpose in behavior," in Computational and Robotic Models of the Hierarchical Organization of Behavior (Berlin: Springer), 319-347.
71. Yin, H. H. (2014a). How the basal ganglia output generates behavior. Adv. Neurosci. 2014:768313. doi: 10.1155/2014/768313
72. Yin, H. H. (2014b). Action, time and the basal ganglia. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369:20120473. doi: 10.1098/rstb.2012.0473
73. Yin, H. H., Knowlton, B. J., and Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181-189. doi: 10.1111/j.1460-9568.2004.03095.x
74. Yin, H. H., Mulcare, S. P., Hilario, M. R., Clouse, E., Holloway, T., Davis, M. I., et al. (2009). Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333-341. doi: 10.1038/nn.2261
75. Yin, H. H., Ostlund, S. B., Knowlton, B. J., and Balleine, B. W. (2005). The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513-523. doi: 10.1111/j.1460-9568.2005.04218.x
76. Zhou, F. M., and Lee, C. R. (2011). Intrinsic and integrative properties of substantia nigra pars reticulata neurons.Neuroscience 198, 69-94. doi: 10.1016/j.neuroscience.2011.07.061