The dissociable effects of punishment and reward on motor learning

Nature Neuroscience

Volumn 18, Issue 4, 2015, Pages 597-602

  Galea, Joseph M ^a   Mallia, Elizabeth ^b   Rothwell, John ^b   Diedrichsen, Jörn ^c  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTATION; ADULT; ARTICLE; CONTROLLED STUDY; DOUBLE DISSOCIATION; FEMALE; HAND MOVEMENT; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; MEMORY CONSOLIDATION; MONEY; MOTIVATION; MOTOR LEARNING; MOVEMENT (PHYSIOLOGY); NEGATIVE FEEDBACK; NEUROFEEDBACK; NORMAL HUMAN; POSITIVE FEEDBACK; PRIORITY JOURNAL; PUNISHMENT; RESPONSE TIME; REWARD; TASK PERFORMANCE; VISUAL FEEDBACK; VISUOMOTOR COORDINATION; YOUNG ADULT; ADAPTIVE BEHAVIOR; LONG TERM MEMORY; MOTOR ACTIVITY; PHYSIOLOGY; PSYCHOMOTOR PERFORMANCE;

ADAPTATION, PSYCHOLOGICAL; ADULT; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; LEARNING; MALE; MOTIVATION; MOTOR ACTIVITY; PSYCHOMOTOR PERFORMANCE; PUNISHMENT; RETENTION (PSYCHOLOGY); REWARD; YOUNG ADULT;

EID: 84925614882     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.3956     Document Type: Article

References (54)

1. Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603-616 (2002).
2. Frank, M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive defcits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 17, 51-72 (2005).
3. den Ouden, H.E. et al. Dissociable effects of dopamine and serotonin on reversal learning. Neuron 80, 1090-1100 (2013).
4. Robinson, O.J., Frank, M.J., Sahakian, B.J. & Cools, R. Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage 51, 1459-1467 (2010).
5. Frank, M.J., Seeberger, L.C. & O'Reilly, R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940-1943 (2004).
6. Wächter, T., Lungu, O.V., Liu, T., Willingham, D.T. & Ashe, J. Differential effect of reward and punishment on procedural learning. J. Neurosci. 29, 436-443 (2009).
7. Abe, M. et al. Reward improves long-term retention of a motor memory through induction of offine memory gains. Curr. Biol. 21, 557-562 (2011).
8. Wolpert, D.M., Diedrichsen, J. & Flanagan, J.R. Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 739-751 (2011).
9. Doya, K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732-739 (2000).
10. Mazzoni, P. & Krakauer, J.W. An implicit plan overrides an explicit strategy during visuomotor adaptation. J. Neurosci. 26, 3642-3645 (2006).
11. Shadmehr, R. & Krakauer, J.W. A computational neuroanatomy for motor control. Exp. Brain Res. 185, 359-381 (2008).
12. Huang, V.S. & Krakauer, J.W. Robotic neurorehabilitation: a computational motor learning perspective. J. Neuroeng. Rehabil. 6, 5 (2009).
13. Krakauer, J.W. Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr. Opin. Neurol. 19, 84-90 (2006).
14. Galea, J.M., Vazquez, A., Pasricha, N., Orban de Xivry, J.J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761-1770 (2011).
15. Rabe, K. et al. Adaptation to visuomotor rotation and force feld perturbation is correlated to different brain areas in patients with cerebellar degeneration. J. Neurophysiol. 101, 1961-1971 (2009).
16. Moulton, E.A. et al. Aversion-related circuitry in the cerebellum: responses to noxious heat and unpleasant images. J. Neurosci. 31, 3795-3804 (2011).
17. Ernst, M. et al. Decision-making in a risk-taking task: a PET study. Neuropsychopharmacology 26, 682-691 (2002).
18. McCormick, D.A. & Thompson, R.F. Cerebellum: essential involvement in the classically conditioned eyelid response. Science 223, 296-299 (1984).
19. Hester, R., Murphy, K., Brown, F.L. & Skilleter, A.J. Punishing an error improves learning: the infuence of punishment magnitude on error-related neural activity and subsequent learning. J. Neurosci. 30, 15600-15607 (2010).
20. Hadipour-Niktarash, A., Lee, C.K., Desmond, J.E. & Shadmehr, R. Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J. Neurosci. 27, 13413-13419 (2007).
21. Richardson, A.G. et al. Disruption of primary motor cortex before learning impairs memory of movement dynamics. J. Neurosci. 26, 12466-12470 (2006).
22. Beierholm, U. et al. Dopamine modulates reward-related vigor. Neuropsychopharmacology 38, 1495-1503 (2013).
23. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507-520 (2007).
24. Awenowicz, P.W. & Porter, L.L. Local application of dopamine inhibits pyramidal tract neuron activity in the rodent motor cortex. J. Neurophysiol. 88, 3439-3451 (2002).
25. Hosp, J.A., Pekanovic, A., Rioult-Pedotti, M.S. & Luft, A.R. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J. Neurosci. 31, 2481-2487 (2011).
26. Hosp, J.A. & Luft, A.R. Dopaminergic meso-cortical projections to M1: role in motor learning and motor cortex plasticity. Front. Neurol. 4, 145 (2013).
27. Shmuelof, L. et al. Overcoming motor "forgetting" through reinforcement of learned actions. J. Neurosci. 32, 14617-14621 (2012).
28. Thabit, M.N. et al. Momentary reward induce changes in excitability of primary motor cortex. Clin. Neurophysiol. 122, 1764-1770 (2011).
29. Krakauer, J.W. Motor learning and consolidation: the case of visuomotor rotation. Adv. Exp. Med. Biol. 629, 405-421 (2009).
30. Donchin, O., Francis, J.T. & Shadmehr, R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J. Neurosci. 23, 9032-9045 (2003).
31. Thoroughman, K.A. & Shadmehr, R. Learning of action through adaptive combination of motor primitives. Nature 407, 742-747 (2000).
32. Taylor, J.A. & Ivry, R.B. The role of strategies in motor learning. Ann. NY Acad. Sci. 1251, 1-12 (2012).
33. Torrubia, R., Avila, C., Molto, J. & Caseras, X. The Sensitivity to Punishment and Sensitivity Reward Questionnaire (SPSRQ) as a measure of Gray's anxiety and impulsivity dimensions. Pers. Indiv. Differ. 31, 837-862 (2001).
34. Smith, M.A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).
35. Tseng, Y.W., Diedrichsen, J., Krakauer, J.W., Shadmehr, R. & Bastian, A.J. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 98, 54-62 (2007).
36. Galea, J.M., Ruge, D., Buijink, A., Bestmann, S. & Rothwell, J.C. Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters. J. Neurosci. 33, 3981-3988 (2013).
37. Dayan, P. & Daw, N.D. Decision theory, reinforcement learning, and the brain. Cogn. Affect. Behav. Neurosci. 8, 429-453 (2008).
38. Izawa, J. & Shadmehr, R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 7, e1002012 (2011).
39. Wu, H.G., Miyamoto, Y.R., Castro, L.N., Olveczky, B.P. & Smith, M.A. Temporal structure of motor variability is dynamically regulated and predicts motor learning ability. Nat. Neurosci. 17, 312-321 (2014).
40. Nichols, R.A. Serotonin, presynaptic 5-HT(3) receptors and synaptic plasticity in the developing cerebellum. J. Physiol. (Lond.) 589, 5019-5020 (2011).
41. Kahneman, D. & Tversky, A. Prospect theory-analysis of decision under risk. Econometrica 47, 263-291 (1979).
42. De Martino, B., Camerer, C.F. & Adolphs, R. Amygdala damage eliminates monetary loss aversion. Proc. Natl. Acad. Sci. USA 107, 3788-3792 (2010).
43. Kahneman, D. & Tversky, A. Choices, values, and frames. Am. Psychol. 39, 341-350 (1984).
44. Mercer, J. Prospect theory and political science. Annu. Rev. Polit. Sci. 8, 1-21 (2005).
45. Dayan, E., Averbeck, B.B., Richmond, B.J. & Cohen, L.G. Stochastic reinforcement benefts skill acquisition. Learn. Mem. 21, 140-142 (2014).
46. Haith, A.M. & Krakauer, J.W. Model-based and model-free mechanisms of human motor learning. Adv. Exp. Med. Biol. 782, 1-21 (2013).
47. Orban de Xivry, J.J., Criscimagna-Hemminger, S.E. & Shadmehr, R. Contributions of the motor cortex to adaptive control of reaching depend on the perturbation schedule. Cereb. Cortex 21, 1475-1484 (2011).
48. Huang, V.S., Haith, A., Mazzoni, P. & Krakauer, J.W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787-801 (2011).
49. Patton, J.L. & Mussa-Ivaldi, F.A. Robot-assisted adaptive training: custom force felds for teaching movement patterns. IEEE Trans. Biomed. Eng. 51, 636-646 (2004).
50. Diedrichsen, J., Hashambhoy, Y., Rane, T. & Shadmehr, R. Neural correlates of reach errors. J. Neurosci. 25, 9919-9931 (2005).
51. Tanaka, H., Sejnowski, T.J. & Krakauer, J.W. Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas. J. Neurophysiol. 102, 2921-2932 (2009).
52. Galea, J.M., Sami, S.A., Albert, N.B. & Miall, R.C. Secondary tasks impair adaptation to step-and gradual-visual displacements. Exp. Brain Res. 202, 473-484 (2010).
53. Galea, J.M., Vazquez, A., Pasricha, N., de Xivry, J.J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761-1770 (2011).
54. Ingram, J.N., Flanagan, J.R. & Wolpert, D.M. Context-dependent decay of motor memories during skill acquisition. Curr. Biol. 23, 1107-1112 (2013).