Learning from sensory and reward prediction errors during motor adaptation

PLoS Computational Biology

Volumn 7, Issue 3, 2011, Pages

  Izawa, Jun ^a,b   Shadmehr, Reza ^a  

^a JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF ELECTRO COMMUNICATIONS   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ERRORS; FORECASTING;

ADAPTATION PROTOCOLS; GENERALISATION; HIGH QUALITY; MOTOR ADAPTATION; MOTOR COMMANDS; PREDICTION ERRORS; REMAPPING; RESULT COMPARISON; REWARD-PREDICTION ERROR; SENSORY ORGANS;

SENSORY FEEDBACK;

ADULT; ARTICLE; CONTROLLED STUDY; FEEDBACK SYSTEM; HUMAN; HUMAN EXPERIMENT; LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MOTOR CONTROL; MOTOR PERFORMANCE; PREDICTION; REWARD; SENSORY STIMULATION; STIMULUS RESPONSE; SYSTEMATIC ERROR; TASK PERFORMANCE; ADAPTATION; ANALYSIS OF VARIANCE; MOTOR ACTIVITY; PHYSIOLOGY; PSYCHOMOTOR PERFORMANCE; SENSORY FEEDBACK;

ADAPTATION, PHYSIOLOGICAL; ADULT; ANALYSIS OF VARIANCE; FEEDBACK; FEEDBACK, SENSORY; HUMANS; MOTOR ACTIVITY; PSYCHOMOTOR PERFORMANCE; REWARD; TASK PERFORMANCE AND ANALYSIS;

EID: 79953656304     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1002012     Document Type: Article

References (51)

1. Synofzik M, Thier P, Lindner A, (2006) Internalizing agency of self-action: perception of one's own hand movements depends on an adaptable prediction about the sensory action outcome. J Neurophysiol 96: 1592-1601.
2. Synofzik M, Lindner A, Thier P, (2008) The cerebellum updates predictions about the visual consequences of one's behavior. Curr Biol 18: 814-818.
3. Baddeley RJ, Ingram HA, Miall RC, (2003) System identification applied to a visuomotor task: near-optimal human performance in a noisy changing task. J Neurosci 23: 3066-3075.
4. Berniker M, Kording K, (2008) Estimating the sources of motor errors for adaptation and generalization. Nat Neurosci 11: 1454-1461.
5. Kording KP, Tenenbaum JB, Shadmehr R, (2007) The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat Neurosci 10: 779-786.
6. Sing GC, Joiner WM, Nanayakkara T, Brayanov JB, Smith MA, (2009) Primitives for motor adaptation reflect correlated neural tuning to position and velocity. Neuron 64: 575-589.
7. van Beers RJ, (2009) Motor learning is optimally tuned to the properties of motor noise. Neuron 63: 406-417.
8. Frank MJ, Doll BB, Oas-Terpstra J, Moreno F, (2009) Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci 12: 1062-1068.
9. Schonberg T, Daw ND, Joel D, O'Doherty JP, (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci 27: 12860-12867.
10. Trommershauser J, Maloney LT, Landy MS, (2008) Decision making, movement planning and statistical decision theory. Trends Cogn Sci 12: 291-297.
11. Kawato M, Gomi H, (1992) A computational model of four regions of the cerebellum based on feedback-error learning. Biol Cybern 68: 95-103.
12. Kawato M, (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9: 718-727.
13. Thoroughman KA, Shadmehr R, (2000) Learning of action through adaptive combination of motor primitives. Nature 407: 742-747.
14. Smith MA, Ghazizadeh A, Shadmehr R, (2006) Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol 4: e179.
15. Pearson TS, Krakauer JW, Mazzoni P, (2010) Learning not to generalize: modular adaptation of visuomotor gain. J Neurophysiol 103: 2938-2952.
16. Shadmehr R, (2004) Generalization as a behavioral window to the neural mechanisms of learning internal models. Hum Mov Sci 23: 543-568.
17. Haswell CC, Izawa J, Dowell LR, Mostofsky SH, Shadmehr R, (2009) Representation of internal models of action in the autistic brain. Nat Neurosci 12: 970-972.
18. Bedford FL, (1999) Keeping perception accurate. Trends Cogn Sci 3: 4-11.
19. Wolpert DM, Ghahramani Z, (2000) Computational principles of movement neuroscience. Nat Neurosci 3 (Suppl): 1212-1217.
20. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM, (2005) Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437: 1158-1161.
21. Trommershauser J, Gepshtein S, Maloney LT, Landy MS, Banks MS, (2005) Optimal compensation for changes in task-relevant movement variability. J Neurosci 25: 7169-7178.
22. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O, (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142: 284-291.
23. Doya K, (2000) Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr Opin Neurobiol 10: 732-739.
24. Izawa J, Rane T, Donchin O, Shadmehr R, (2008) Motor adaptation as a process of reoptimization. J Neurosci 28: 2883-2891.
25. Izawa J, Shadmehr R, (2008) On-line processing of uncertain information in visuomotor control. J Neurosci 28: 11360-11368.
26. Todorov E, Jordan MI, (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5: 1226-1235.
27. Shadmehr R, Krakauer JW, (2008) A computational neuroanatomy for motor control. Exp Brain Res 185: 359-381.
28. Izawa J, Kondo T, Ito K, (2004) Biological arm motion through reinforcement learning. Biol Cybern 91: 10-22.
29. Poggio T, Fahle M, Edelman S, (1992) Fast perceptual learning in visual hyperacuity. Science 256: 1018-1021.
30. Hwang EJ, Smith MA, Shadmehr R, (2006) Adaptation and generalization in acceleration-dependent force fields. Exp Brain Res 169: 496-506.
31. Tanaka H, Sejnowski TJ, Krakauer JW, (2009) Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas. J Neurophysiol 102: 2921-2932.
32. Tseng YW, Diedrichsen J, Krakauer JW, Shadmehr R, Bastian AJ, (2007) Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J Neurophysiol 98: 54-62.
33. Smith MA, Shadmehr R, (2005) Intact ability to learn internal models of arm dynamics in Huntington's disease but not cerebellar degeneration. J Neurophysiol 93: 2809-2821.
34. Yin HH, Knowlton BJ, (2006) The role of the basal ganglia in habit formation. Nat Rev Neurosci 7: 464-476.
35. Nakahara H, Doya K, Hikosaka O, (2001) Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuomotor sequences- a computational approach. J Cogn Neurosci 13: 626-647.
36. Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, et al. (2004) Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7: 887-893.
37. Samejima K, Ueda Y, Doya K, Kimura M, (2005) Representation of action-specific reward values in the striatum. Science 310: 1337-1340.
38. Packard MG, Knowlton BJ, (2002) Learning and memory functions of the Basal Ganglia. Annu Rev Neurosci 25: 563-593.
39. Wickens JR, Reynolds JN, Hyland BI, (2003) Neural mechanisms of reward-related motor learning. Curr Opin Neurobiol 13: 685-690.
40. Dickinson A, Balleine B, (1994) Motivational control of goal-directed action. Anima Learn Behave 22: 1-8.
41. Gabrieli JD, Stebbins GT, Singh J, Willingham DB, Goetz CG, (1997) Intact mirror-tracing and impaired rotary-pursuit skill learning in patients with Huntington's disease: evidence for dissociable memory systems in skill learning. Neuropsychology 11: 272-281.
42. Agostino R, Sanes JN, Hallett M, (1996) Motor skill learning in Parkinson's disease. J Neurol Sci 139: 218-226.
43. Marinelli L, Crupi D, Di Rocco A, Bove M, Eidelberg D, et al. (2009) Learning and consolidation of visuo-motor adaptation in Parkinson's disease. Parkinsonism Relat Disord 15: 6-11.
44. Criscimagna-Hemminger SE, Bastian AJ, Shadmehr R, (2010) Size of error affects cerebellar contributions to motor learning. J Neurophysiol 103: 2275-2284.
45. Harris CM, Wolpert DM, (1998) Signal-dependent noise determines motor planning. Nature 394: 780-784.
46. Jones KE, Hamilton AF, Wolpert DM, (2002) Sources of signal-dependent noise during isometric force production. J Neurophysiol 88: 1533-1544.
47. Faisal AA, Selen LP, Wolpert DM, (2008) Noise in the nervous system. Nat Rev Neurosci 9: 292-303.
48. Churchland MM, Afshar A, Shenoy KV, (2006) A central source of movement variability. Neuron 52: 1085-1096.
49. Burge J, Ernst MO, Banks MS, (2008) The statistical determinants of adaptation rate in human reaching. J Vis 8: 20 21-19.
50. Kording KP, Ku SP, Wolpert DM, (2004) Bayesian integration in force estimation. J Neurophysiol 92: 3161-3165.
51. Sutton R, Barto A, (1998) Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). MIT Press.