Rethinking Motor Learning and Savings in Adaptation Paradigms: Model-Free Memory for Successful Actions Combines with Internal Models

Neuron

Volumn 70, Issue 4, 2011, Pages 787-801

  Huang, Vincent S ^a   Haith, Adrian ^b,c   Mazzoni, Pietro ^a   Krakauer, John W ^a,b  

^a Columbia University College of Physicians and Surgeons ^*  (United States)

^b JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c Johns Hopkins University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTATION; ADULT; ARTICLE; CONTROLLED STUDY; FEMALE; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; MEMORY; MOTOR LEARNING; MOTOR PERFORMANCE; NERVE CELL PLASTICITY; NORMAL HUMAN; PRIORITY JOURNAL; REINFORCEMENT; TASK PERFORMANCE; VISUOMOTOR COORDINATION;

ADAPTATION, PHYSIOLOGICAL; ADULT; FEMALE; HUMANS; LEARNING; MALE; MEMORY; MODELS, BIOLOGICAL; MOVEMENT; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; YOUNG ADULT;

EID: 79956192969     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2011.04.012     Document Type: Article

References (88)

1. Abeele S., Bock O. Sensorimotor adaptation to rotated visual input: different mechanisms for small versus large rotations. Exp. Brain Res. 2001, 140:407-410.
2. Adams C.D., Dickinson A. Instrumental Responding Following Reinforcer Devaluation. Q. J. Exp. Psychol. B 1981, 33:109-121.
3. Ajemian R., D'Ausilio A., Moorman H., Bizzi E. Why professional athletes need a prolonged period of warm-up and other peculiarities of human motor learning. J. Mot. Behav. 2010, 42:381-388.
4. Anderson M.C., Bjork E.L., Bjork R.A. Retrieval-induced forgetting: evidence for a recall-specific mechanism. Psychon. Bull. Rev. 2000, 7:522-530.
5. Arce F., Novick I., Mandelblat-Cerf Y., Vaadia E. Neuronal correlates of memory formation in motor cortex after adaptation to force field. J. Neurosci. 2010, 30:9189-9198.
6. Avila I., Reilly M.P., Sanabria F., Posadas-Sánchez D., Chavez C.L., Banerjee N., Killeen P., Castañeda E. Modeling operant behavior in the Parkinsonian rat. Behav. Brain Res. 2009, 198:298-305.
7. Bédard P., Sanes J.N. Basal ganglia-dependent processes in recalling learned visual-motor adaptations. Exp. Brain Res. 2011, 209:385-393.
8. Bock O., Schneider S., Bloomberg J. Conditions for interference versus facilitation during sequential sensorimotor adaptation. Exp. Brain Res. 2001, 138:359-365.
9. Brashers-Krug T., Shadmehr R., Bizzi E. Consolidation in human motor memory. Nature 1996, 382:252-255.
10. Bütefisch C.M., Davis B.C., Wise S.P., Sawaki L., Kopylev L., Classen J., Cohen L.G. Mechanisms of use-dependent plasticity in the human motor cortex. Proc. Natl. Acad. Sci. USA 2000, 97:3661-3665.
11. Chiviacowsky S., Wulf G. Feedback after good trials enhances learning. Res. Q. Exerc. Sport 2007, 78:40-47.
12. Clark L., Lawrence A.J., Astley-Jones F., Gray N. Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry. Neuron 2009, 61:481-490.
13. Classen J., Liepert J., Wise S.P., Hallett M., Cohen L.G. Rapid plasticity of human cortical movement representation induced by practice. J. Neurophysiol. 1998, 79:1117-1123.
14. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8:1704-1711.
15. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
16. Daw N.D., Gershman S.J., Seymour B., Dayan P., Dolan R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 2011, 69:1204-1215.
17. Dayan P. Goal-directed control and its antipodes. Neural Netw. 2009, 22:213-219.
18. Debicki D.B., Gribble P.L. Inter-joint coupling strategy during adaptation to novel viscous loads in human arm movement. J. Neurophysiol. 2004, 92:754-765.
19. Diedrichsen J., White O., Newman D., Lally N. Use-dependent and error-based learning of motor behaviors. J. Neurosci. 2010, 30:5159-5166.
20. Donchin O., Francis J.T., Shadmehr R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J. Neurosci. 2003, 23:9032-9045.
21. Doya K. What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?. Neural Netw. 1999, 12:961-974.
22. Ebbinghaus H. Memory: A Contribution to Experimental Psychology 1913, Teachers College, Columbia University, New York.
23. Ethier V., Zee D.S., Shadmehr R. Spontaneous recovery of motor memory during saccade adaptation. J. Neurophysiol. 2008, 99:2577-2583.
24. Fermin A., Yoshida T., Ito M., Yoshimoto J., Doya K. Evidence for model-based action planning in a sequential finger movement task. J. Mot. Behav. 2010, 42:371-379.
25. Flanagan J.R., Vetter P., Johansson R.S., Wolpert D.M. Prediction precedes control in motor learning. Curr. Biol. 2003, 13:146-150.
26. Frank M.J., Seeberger L.C., O'reilly R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 2004, 306:1940-1943.
27. Galea J.M., Vazquez A., Pasricha N., Orban de Xivry J.J., Celnik P. Dissociating the Roles of the Cerebellum and Motor Cortex during Adaptive Learning: The Motor Cortex Retains What the Cerebellum Learns. Cereb. Cortex. 2010, Published online December 7, 2010.
28. Gandolfo F., Mussa-Ivaldi F.A., Bizzi E. Motor learning by field approximation. Proc. Natl. Acad. Sci. USA 1996, 93:3843-3846.
29. Gläscher J., Daw N., Dayan P., O'Doherty J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 2010, 66:585-595.
30. Haruno M., Wolpert D.M., Kawato M. Mosaic model for sensorimotor learning and control. Neural Comput. 2001, 13:2201-2220.
31. Held R., Rekosh J. Motor-sensory feedback and the geometry of visual space. Science 1963, 141:722-723.
32. Hosp J.A., Pekanovic A., Rioult-Pedotti M.S., Luft A.R. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J. Neurosci. 2011, 31:2481-2487.
33. Huang V.S., Shadmehr R. Evolution of motor memory during the seconds after observation of motor error. J. Neurophysiol. 2007, 97:3976-3985.
34. Huang V.S., Shadmehr R. Persistence of motor memories reflects statistics of the learning event. J. Neurophysiol. 2009, 102:931-940.
35. Huang V.S., Shadmehr R., Diedrichsen J. Active learning: learning a motor skill without a coach. J. Neurophysiol. 2008, 100:879-887.
36. Huntley G.W., Morrison J.H., Prikhozhan A., Sealfon S.C. Localization of multiple dopamine receptor subtype mRNAs in human and monkey motor cortex and striatum. Brain Res. Mol. Brain Res. 1992, 15:181-188.
37. Imamizu H., Uno Y., Kawato M. Internal representations of the motor apparatus: implications from generalization in visuomotor learning. J. Exp. Psychol. Hum. Percept. Perform. 1995, 21:1174-1198.
38. Izawa J., Shadmehr R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 2011, 7:e1002012.
39. Jax S.A., Rosenbaum D.A. Hand path priming in manual obstacle avoidance: evidence that the dorsal stream does not only control visually guided actions in real time. J. Exp. Psychol. Hum. Percept. Perform. 2007, 33:425-441.
40. Joiner W.M., Smith M.A. Long-term retention explained by a model of short-term learning in the adaptive control of reaching. J. Neurophysiol. 2008, 100:2948-2955.
41. Kaelbling L.P., Littman M.L., Moore A.W. Reinforcement learning: A survey. J. Artif. Intell. Res. 1996, 4:237-285.
42. Kakade S., Dayan P. Dopamine: generalization and bonuses. Neural Netw. 2002, 15:549-559.
43. Karpicke J.D., Roediger H.L. The critical importance of retrieval for learning. Science 2008, 319:966-968.
44. Kojima Y., Iwamoto Y., Yoshida K. Memory of learning facilitates saccadic adaptation in the monkey. J. Neurosci. 2004, 24:7531-7539.
45. Kording K.P., Tenenbaum J.B., Shadmehr R. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat. Neurosci. 2007, 10:779-786.
46. Korenberg A.T., Ghahramani Z. A Bayesian view of motor adaptation. Current Psychology of Cognition 2002, 21:537-564.
47. Krakauer J.W., Ghilardi M.F., Ghez C. Independent learning of internal models for kinematic and dynamic control of reaching. Nat. Neurosci. 1999, 2:1026-1031.
48. Krakauer J.W., Pine Z.M., Ghilardi M.F., Ghez C. Learning of visuomotor transformations for vectorial planning of reaching trajectories. J. Neurosci. 2000, 20:8916-8924.
49. Krakauer J.W., Ghez C., Ghilardi M.F. Adaptation to visuomotor transformations: consolidation, interference, and forgetting. J. Neurosci. 2005, 25:473-478.
50. Krutky M.A., Perreault E.J. Motor cortical measures of use-dependent plasticity are graded from distal to proximal in the human upper limb. J. Neurophysiol. 2007, 98:3230-3241.
51. Lackner J.R., Dizio P. Rapid adaptation to Coriolis force perturbations of arm trajectory. J. Neurophysiol. 1994, 72:299-313.
52. Lee J.Y., Schweighofer N. Dual adaptation supports a parallel architecture of motor memory. J. Neurosci. 2009, 29:10396-10404.
53. Luft A.R., Schwarz S. Dopaminergic signals in primary motor cortex. Int. J. Dev. Neurosci. 2009, 27:415-421.
54. MacLeod M.D., Macrae C.N. Gone but not forgotten: the transient nature of retrieval-induced forgetting. Psychol. Sci. 2001, 12:148-152.
55. MacLin O.H., Dixon M.R., Daugherty D., Small S.L. Using a computer simulation of three slot machines to investigate a gambler's preference among varying densities of near-miss alternatives. Behav. Res. Methods 2007, 39:237-241.
56. Malfait N., Gribble P.L., Ostry D.J. Generalization of motor learning based on multiple field exposures and local adaptation. J. Neurophysiol. 2005, 93:3327-3338.
57. Marinelli L., Crupi D., Di Rocco A., Bove M., Eidelberg D., Abbruzzese G., Ghilardi M.F. Learning and consolidation of visuo-motor adaptation in Parkinson's disease. Parkinsonism Relat. Disord. 2009, 15:6-11.
58. Martin T.A., Keating J.G., Goodkin H.P., Bastian A.J., Thach W.T. Throwing while looking through prisms. I. Focal olivocerebellar lesions impair adaptation. Brain 1996, 119:1183-1198.
59. Martin T.A., Keating J.G., Goodkin H.P., Bastian A.J., Thach W.T. Throwing while looking through prisms. II. Specificity and storage of multiple gaze-throw calibrations. Brain 1996, 119:1199-1211.
60. Mazzoni P., Krakauer J.W. An implicit plan overrides an explicit strategy during visuomotor adaptation. J. Neurosci. 2006, 26:3642-3645.
61. Mazzoni P., Hristova A., Krakauer J.W. Why don't we move faster? Parkinson's disease, movement vigor, and implicit motivation. J. Neurosci. 2007, 27:7105-7116.
62. Miall R.C., Jenkinson N., Kulkarni K. Adaptation to rotated visual feedback: a re-examination of motor interference. Exp. Brain Res. 2004, 154:201-210.
63. Orban de Xivry J.J., Criscimagna-Hemminger S.E., Shadmehr R. Contributions of the Motor Cortex to Adaptive Control of Reaching Depend on the Perturbation Schedule. Cereb. Cortex. 2011, Published online December 3, 2010. 10.1093/cercor/bhq192.
64. Pine Z.M., Krakauer J.W., Gordon J., Ghez C. Learning of scaling factors and reference axes for reaching movements. Neuroreport 1996, 7:2357-2361.
65. Rushworth M.F., Nixon P.D., Wade D.T., Renowden S., Passingham R.E. The left hemisphere and the selection of learned actions. Neuropsychologia 1998, 36:11-24.
66. Rutledge R.B., Lazzaro S.C., Lau B., Myers C.E., Gluck M.A., Glimcher P.W. Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J. Neurosci. 2009, 29:15104-15114.
67. Scheidt R.A., Dingwell J.B., Mussa-Ivaldi F.A. Learning to move amid uncertainty. J. Neurophysiol. 2001, 86:971-985.
68. Seitz A.R., Kim D., Watanabe T. Rewards evoke learning of unconsciously processed visual stimuli in adult humans. Neuron 2009, 61:700-707.
69. Shadmehr R., Mussa-Ivaldi F.A. Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 1994, 14:3208-3224.
70. Shadmehr R., Smith M.A., Krakauer J.W. Error correction, sensory prediction, and adaptation in motor control. Annu. Rev. Neurosci. 2010, 33:89-108.
71. Shohamy D., Myers C.E., Grossman S., Sage J., Gluck M.A. The role of dopamine in cognitive sequence learning: evidence from Parkinson's disease. Behav. Brain Res. 2005, 156:191-199.
72. Sing G.C., Smith M.A. Reduction in learning rates associated with anterograde interference results from interactions between different timescales in motor adaptation. PLoS Comput. Biol. 2010, 6. pii: e1000893.
73. Smith M.A., Shadmehr R. Intact ability to learn internal models of arm dynamics in Huntington's disease but not cerebellar degeneration. J. Neurophysiol. 2005, 93:2809-2821.
74. Smith M.A., Ghazizadeh A., Shadmehr R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 2006, 4:e179.
75. Sutton R.S., Barto A.G. Reinforcement learning: An introduction. Adaptive Computation and Machine Learning 1998, MIT Press, Cambridge, MA.
76. Tanaka H., Sejnowski T.J., Krakauer J.W. Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas. J. Neurophysiol. 2009, 102:2921-2932.
77. Thoroughman K.A., Shadmehr R. Learning of action through adaptive combination of motor primitives. Nature 2000, 407:742-747.
78. Tong C., Wolpert D.M., Flanagan J.R. Kinematics and dynamics are not represented independently in motor working memory: evidence from an interference study. J. Neurosci. 2002, 22:1108-1113.
79. Tseng Y.W., Diedrichsen J., Krakauer J.W., Shadmehr R., Bastian A.J. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 2007, 98:54-62.
80. Wächter T., Röhrich S., Frank A., Molina-Luna K., Pekanovic A., Hertler B., Schubring-Giese M., Luft A.R. Motor skill learning depends on protein synthesis in the dorsal striatum after training. Exp. Brain Res. 2010, 200:319-323.
81. Wigmore V., Tong C., Flanagan J.R. Visuomotor rotations of varying size and direction compete for a single internal model in motor working memory. J. Exp. Psychol. Hum. Percept. Perform. 2002, 28:447-457.
82. Wixted J.T. The psychology and neuroscience of forgetting. Annu. Rev. Psychol. 2004, 55:235-269.
83. Wulf G., Shea C.H. Principles derived from the study of simple skills do not generalize to complex skill learning. Psychon. Bull. Rev. 2002, 9:185-211.
84. Wulf G., Shea C., Lewthwaite R. Motor skill learning and performance: a review of influential factors. Med. Educ. 2010, 44:75-84.
85. Yarrow K., Brown P., Krakauer J.W. Inside the brain of an elite athlete: the neural processes that support high achievement in sports. Nat. Rev. Neurosci. 2009, 10:585-596.
86. Zarahn E., Weston G.D., Liang J., Mazzoni P., Krakauer J.W. Explaining savings for visuomotor adaptation: linear time-invariant state-space models are not sufficient. J. Neurophysiol. 2008, 100:2537-2548.
87. Ziemann U., Tergau F., Bruns D., Baudewig J., Paulus W. Changes in human motor cortex excitability induced by dopaminergic and anti-dopaminergic drugs. Electroencephalogr. Clin. Neurophysiol. 1997, 105:430-437.
88. Ziemann U., Muellbacher W., Hallett M., Cohen L.G. Modulation of practice-dependent plasticity in human motor cortex. Brain 2001, 124:1171-1181.