Reward-dependent modulation of movement variability

Journal of Neuroscience

Volumn 35, Issue 9, 2015, Pages 4015-4024

  Pekny, Sarah E ^a   Izawa, Jun ^a,b   Shadmehr, Reza ^a  

^a JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF TSUKUBA   (Japan)

Author keywords

Computational models; Motor control; Parkinson s disease; Reaching

Indexed keywords

ADULT; AGED; ARTICLE; BEHAVIOR ASSESSMENT; CLINICAL ARTICLE; CONTROLLED STUDY; DISEASE SEVERITY; FEMALE; HUMAN; LEARNING DISORDER; MALE; MATHEMATICAL PARAMETERS; MIDDLE AGED; MOTOR DYSFUNCTION; NEUROMODULATION; OUTCOME ASSESSMENT; PARKINSON DISEASE; PRIORITY JOURNAL; REWARD; REWARD DEPENDENT MOTOR VARIABILITY; SENSITIVITY ANALYSIS; UNIFIED PARKINSON DISEASE RATING SCALE; AGING; CONDITIONING; DEPTH PERCEPTION; FEEDBACK SYSTEM; MOVEMENT (PHYSIOLOGY); PATHOPHYSIOLOGY; PHYSIOLOGY; PSYCHOLOGY; PSYCHOMOTOR PERFORMANCE; REACTION TIME;

AGED; AGING; CONDITIONING (PSYCHOLOGY); FEEDBACK, PHYSIOLOGICAL; FEMALE; HUMANS; MALE; MIDDLE AGED; MOVEMENT; PARKINSON DISEASE; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REWARD; SPACE PERCEPTION;

EID: 84924777672     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.3244-14.2015     Document Type: Article

References (51)

1. Agostino R, Sanes JN, Hallett M (1996) Motor skill learning in Parkinson’sdisease. J Neurol Sci 139:218–226.
2. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47:129–141.
3. Bédard P, Sanes JN (2011) Basal ganglia-dependent processes in recalling learned visual-motor adaptations. Exp Brain Res 209:385–393.
4. Bock O, Thomas M, Grigorova V (2005) The effect of rest breaks on human sensorimotor adaptation. Exp Brain Res 163:258–260.
5. Bódi N, Kéri S, Nagy H, Moustafa A, Myers CE, Daw N, Dibó G, Takáts A, Bereczki D, Gluck MA (2009) Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopa-mine agonists on young Parkinson’s patients. Brain 132:2385–2395.
6. Chowdhury R, Guitart-Masip M, Lambert C, Dayan P, Huys Q, Düzel E, Dolan RJ (2013) Dopamine restores reward prediction errors in old age. Nat Neurosci 16:648–653.
7. Churchland MM, Afshar A, Shenoy KV (2006) A central source of movement variability. Neuron 52:1085–1096.
8. Cools R, Barker RA, Sahakian BJ, Robbins TW (2001) Enhanced or impaired cognitive function in Parkinson’s disease as a function of dopami-nergic medication and task demands. Cereb Cortex 11:1136–1143.
9. Fearnley JM, Lees AJ (1991) Ageing and Parkinson’s disease: substantia nigra regional selectivity. Brain 114:2283–2301.
10. Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state.” A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12:189–198.
11. Francis JT (2005) Influence of the inter-reach-interval on motor learning. Exp Brain Res 167:128–131.
12. Frank MJ, Seeberger LC, O’reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940–1943.
13. Frank MJ, Samanta J, Moustafa AA, Sherman SJ (2007) Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science 318:1309–1312.
14. Galea JM, Ruge D, Buijink A, Bestmann S, Rothwell JC (2013) Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters. J Neurosci 33: 3981–3988.
15. Gordon J, Ghilardi MF, Cooper SE, Ghez C (1994) Accuracy of planar reaching movements. II. Systematic extent errors resulting from inertial anisotropy. Exp Brain Res 99:112–130.
16. Haith AM, Reppert TR, Shadmehr R (2012) Evidence for hyperbolic temporal discounting of reward in control of movements. J Neurosci 32: 11727–11736.
17. Herzfeld DJ, Vaswani PA, Marko MK, Shadmehr R (2014) A memory of errors in sensorimotor learning. Science 345:1349–1353.
18. Huang VS, Shadmehr R (2007) Evolution of motor memory during the seconds after observation of motor error. J Neurophysiol 97:3976–3985.
19. Huang VS, Haith A, Mazzoni P, Krakauer JW (2011) Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70:787–801.
20. Izawa J, Shadmehr R (2011) Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol 7:e1002012.
21. Izawa J, Criscimagna-Hemminger SE, Shadmehr R (2012a) Cerebellar contributions to reach adaptation and learning sensory consequences of action. J Neurosci 32:4230–4239.
22. Izawa J, Pekny SE, Marko MK, Haswell CC, Shadmehr R, Mostofsky SH (2012b) Motor learning relies on integrated sensory inputs in ADHD, but over-selectively on proprioception in autism spectrum conditions. Autism Res 5:124–136.
23. Jones KE, Hamilton AF, Wolpert DM (2002) Sources of signal-dependent noise during isometric force production. J Neurophysiol 88:1533–1544.
24. Kao MH, Doupe AJ, Brainard MS (2005) Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433: 638–643.
25. Knowlton BJ, Mangels JA, Squire LR (1996) A neostriatal habit learning system in humans. Science 273:1399–1402.
26. Leow LA, Loftus AM, Hammond GR (2012) Impaired savings despite intact initial learning of motor adaptation in Parkinson’s disease. Exp Brain Res 218:295–304.
27. Leow LA, de Rugy A, Loftus AM, Hammond G (2013) Different mechanisms contributing to savings and anterograde interference are impaired in Parkinson’s disease. Front Hum Neurosci 7:55.
28. Marinelli L, Crupi D, Di Rocco A, Bove M, Eidelberg D, Abbruzzese G, Ghilardi MF (2009) Learning and consolidationof visuo-motor adapta-tionin Parkinson’s disease. Parkinsonism Relat Disord 15:6–11.
29. Marko MK, Haith AM, Harran MD, Shadmehr R (2012) Sensitivity to prediction error in reach adaptation. J Neurophysiol 108:1752–1763.
30. Mazzoni P, Hristova A, Krakauer JW (2007) Why don’t we move faster? Parkinson’s disease, movement vigor, and implicit motivation. JNeurosci 27:7105–7116.
31. Movement Disorder Society Task Force on Rating Scales for Parkinson’s Disease (2003) The Unified Parkinson’s Disease Rating Scale (UPDRS): status and recommendations. Movement Disorders 18:738–750.
32. Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 191:507– 520.
33. Olveczky BP, Andalman AS, Fee MS (2005) Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol 3:e153.
34. Page S, Neuringer A (1985) Variability is an operant. J Exp Psych Anim Behav Proc 11:429–452.
35. Paquet F, Be´dard MA, Levesque M, Tremblay PL, Lemay M, Blanchet PJ, Scherzer P, Chouinard S, Filion J (2008) Sensorimotor adaptation in Parkinson’s disease: evidence for a dopamine dependent remapping disturbance. Exp Brain Res 185:227–236.
36. Pesek-Cotton EF, Johnson JE, Newland MC (2011) Reinforcing behavioral variability: an analysis of dopamine-receptor subtypes and intermittent reinforcement. Pharmacol Biochem Behav 97:551–559.
37. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042–1045.
38. Pizzagalli DA, Evins AE, Schetter EC, Frank MJ, Pajtas PE, Santesso DL, Culhane M (2008) Single dose of a dopamine agonist impairs reinforcement learning in humans: behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacology (Berl) 196:221–232.
39. Rutledge RB, Lazzaro SC, Lau B, Myers CE, Gluck MA, Glimcher PW (2009) Dopaminergic drugs modulate learning rates and perseveration in Parkinson’s patients in a dynamic foraging task. J Neurosci 29:15104–15114.
40. Schott BH, Niehaus L, Wittmann BC, Schütze H, Seidenbecher CI, Heinze HJ, Düzel E (2007) Ageing and early-stage Parkinson’s disease affect separable neural mechanisms of mesolimbic reward processing. Brain 130:2412–2424.
41. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599.
42. Shadmehr R, Orban de Xivry JJ, Xu-Wilson M, Shih TY (2010) Temporal discounting of reward and the cost of time in motor control. J Neurosci 30:10507–10516.
43. Shohamy D, Myers CE, Onlaor S, Gluck MA (2004) Role ofthe basal ganglia in category learning: how do patients with Parkinson’s disease learn? Be-hav Neurosci 118:676–686.
44. Shohamy D, Myers CE, Geghman KD, Sage J, Gluck MA (2006) L-dopa impairs learning, but spares generalization, in Parkinson’s disease. Neu-ropsychologia 44:774–784.
45. Tai LH, Lee AM, Benavidez N, Bonci A, Wilbrecht L (2012) Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat Neurosci 15:1281–1289.
46. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284–291.
47. Tumer EC, Brainard MS (2007) Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong. Nature 450:1240–1244.
48. Vaillancourt DE, Spraker MB, Prodoehl J, Zhou XJ, Little DM (2012) Effects of aging on the ventral and dorsal substantia nigra using diffusion tensor imaging. Neurobiol Aging 33:35–42.
49. Vakil E, Hassin-Baer S, Karni A (2014) A deficit in optimizing task solution but robust and well-retained speed and accuracy gains in complex skill acquisition in Parkinson’s disease: multi-session training on the Tower of Hanoi puzzle. Neuropsychologia 57:12–19.
50. Wu HG, Miyamoto YR, Gonzalez Castro LN, Olveczky BP, Smith MA (2014) Temporal structure of motor variability is dynamically regulated and predicts motor learning ability. Nat Neurosci 17:312–321.
51. Yang Y, Lisberger SG (2014) Role of plasticity at different sites across the time course of cerebellar motor learning. J Neurosci 34:7077–7090.