A computational model of use-dependent motor recovery following a stroke: Optimizing corticospinal activations via reinforcement learning can explain residual capacity and other strength recovery dynamics

Neural Networks

Volumn 29-30, Issue , 2012, Pages 60-69

  Reinkensmeyer, David J ^a,b   Guigon, Emmanuel ^c,d   Maier, Marc A ^e,f  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c SORBONNE UNIVERSITÉ   (France)

^d UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^e CNRS   (France)

^f UNIVERSITÉ PARIS DESCARTES   (France)

Author keywords

Motor control; Reinforcement learning; Stroke; Use dependent plasticity

Indexed keywords

BRAIN ACTIVATION; BRAIN-IMAGING DATA; CELL ACTIVATION; COMPUTATIONAL MODEL; CORTICOSPINAL; DIFFERENTIAL EFFECT; MOTOR CONTROL; MOTOR RECOVERY; MOTOR SYSTEMS; RAPID OPTIMIZATION; RESIDUAL CAPACITY; STRENGTH RECOVERY; STROKE; STRONGLY CONNECTED; SYNAPTIC WEIGHT;

CHEMICAL ACTIVATION; COMPUTATIONAL METHODS; LEARNING ALGORITHMS; OPTIMIZATION; PARTIAL DISCHARGES; RECOVERY; REINFORCEMENT LEARNING; SECONDARY BATTERIES; SECONDARY RECOVERY; STOCHASTIC MODELS;

PATIENT REHABILITATION;

ARTICLE; BODY MOVEMENT; CELL ACTIVATION; CEREBROVASCULAR ACCIDENT; CONTROLLED STUDY; CONVALESCENCE; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MOTOR PERFORMANCE; MUSCLE STRENGTH; NERVE CELL NETWORK; NEUROIMAGING; PHYSICAL CAPACITY; PREDICTION; PRIORITY JOURNAL; PROCESS MODEL; PROCESS OPTIMIZATION; PYRAMIDAL TRACT; REINFORCEMENT; SIGNAL TRANSDUCTION; STOCHASTIC MODEL;

COMPUTATIONAL BIOLOGY; HUMANS; LEARNING; MOTOR SKILLS; MUSCLE STRENGTH; PYRAMIDAL TRACTS; RECOVERY OF FUNCTION; REINFORCEMENT (PSYCHOLOGY); STROKE;

EID: 84860330660     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2012.02.002     Document Type: Article

References (64)

1. Ada L., Dorsch S., Canning C.G. Strengthening interventions increase strength and improve activity after stroke: a systematic review. The Australian Journal of Physiotherapy 2006, 52(4):241-248.
2. Anderson R.W., Omidvar O.M., Dayhoff J. Random-walk learning: a neurobiological correlate to trial-and-error. Progress in neural networks 1988, 221-244. Academic Press, Boston.
3. Ashe J. Force and the motor cortex. Behavioural Brain Research 1997, 87(2):255-269.
4. Barreca S., Wolf S.L., Fasoli S., Bohannon R. Treatment interventions for the paretic upper limb of stroke survivors: a critical review. Neurorehabilitation and Neural Repair 2003, 17(4):220-226.
5. Bohannon R.W., Warren M.E., Cogman K.A. Motor variables correlated with the hand-to-mouth maneuver in stroke patients. Archives of Physical Medicine and Rehabilitation 1991, 7:682-684.
6. Bolognini N., Pascual-Leone A., Fregni F. Using non-invasive brain stimulation to augment motor training-induced plasticity. Journal of NeuroEngineering and Rehabilitation 2009, 6:8.
7. Brochard S., Robertson J., Médée B., Rémy-Néris O. What's new in new technologies for upper extremity rehabilitation?. Current Opinion in Neurology 2010.
8. Broderick J., Brott T., Kothari R., Miller R., Khoury J., Pancioli A., et al. The greater Cincinnati/Northern Kentucky Stroke study: preliminary first-ever and total incidence rates of stroke among blacks. Stroke 1998, 415-421.
9. Brouwer B.J. Hand function and motor cortical output poststroke: are they related?. Archives of Physical Medicine and Rehabilitation 2006, 87(5):627-634.
10. Canning C.G., Ada L., Adams R., O'Dwyer N.J. Loss of strength contributes more to physical disability after stroke than loss of dexterity. Clinical Rehabilitation 2004, 18(3):300-308.
11. Carson R.G., Riek S., Shahbazpour N. Central and peripheral mediation of human force sensation following eccentric or concentric contractions. Journal of Physiology 2002, 539(3):913-925.
12. Conner J.M., Chiba A.A., Tuszynski M.H. The basal forebrain cholinergic system is essential for cortical plasticity and functional recovery following brain injury. Neuron 2005, 46(2):173-179.
13. Cramer S.C. Repairing the human brain after stroke: I. Mechanisms of spontaneous recovery. Annals of Neurology 2008, 63(3):272-287.
14. Dewald J.P., Beer R.F. Abnormal joint torque patterns in the paretic upper limb of subjects with hemiparesis. Muscle and Nerve 2001, 24(2):273-283.
15. Duncan P.W., Goldstein L.B., Horner R.D., Landsman P.B., Samsa G.P., Matchar D.B Similar motor recovery of upper and lower extremities after stroke. Stroke 1994, 25(6):1181-1188.
16. Fetz E.E., Cheney P.D. Postspike facilitation of forelimb muscle activity by primate corticomotoneuronal cells. Journal of Neurophysiology 1980, 44(4):751-772.
17. Fetz E.E., Cheney P.D., Mewes K., Palmer S. Control of forelimb muscle activity by populations of corticomotoneuronal and rubromotoneuronal cells. Progress in Brain Research 1989, 80:437-449.
18. Feydy A., Carlier R., Roby-Brami A., Bussel B., Cazalis F., Pierot L., et al. Longitudinal study of motor recovery after stroke: recruitment and focusing of brain activation. Stroke: A Journal of Cerebral Circulation 2002, 33(6):1610-1617.
19. French B., Thomas L.H., Leathley M.J., Sutton C.J., McAdam J., Forster A., et al. Repetitive task training for improving functional ability after stroke. Cochrane Database of Systematic Reviews (Online) 2007, (4). CD006073.
20. Fuglevand A.J., Winter D.A., Patla A.E. Models of recruitment and rate coding organization in motor-unit pools. Journal of Neurophysiology 1993, 70(6):2470-2488.
21. Goodall S., Reggia J.A., Chen Y., Ruppin E., Whitney C. A computational model of acute focal cortical lesions. Stroke 1997, 28(1):101-109.
22. Gresham G.E., Duncan P.W., Stason W.B. Post-stroke rehabilitation 1995, US Dept. Health and Human Services, Agency for Health Care Policy and Research, Rockville, MD.
23. Han C.E., Arbib M.A., Schweighofer N. Stroke rehabilitation reaches a threshold. PLoS Computational Biology 2008, 4:e1000133.
24. Harris J.E., Eng J.J. Paretic upper-limb strength best explains arm activity in people with stroke. Physical Therapy 2007, 87(1):88-97.
25. Heller A., Wade D.T., Wood V.A., Sunderland A., Hewer R.L., Ward E. Arm function after stroke: measurement and recovery over the first three months. Journal of Neurology, Neurosurgery and Psychiatry 1987, 50(6):714-719.
26. Kargo W.J., Nitz D.A. Improvements in the signal-to-noise ratio of motor cortex cells distinguish early versus late phases of motor skill learning. The Journal of Neuroscience 2004, 24(24):5560-5569.
27. Kasser R.J., Cheney P.D. Characteristics of corticomotoneuronal postspike facilitation and reciprocal suppression of EMG activity in the monkey. Journal of Neurophysiology 1985, 53(4):959-978.
28. Kwakkel G., Kollen B.J., Krebs H.I. Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabilitation and Neural Repair 2008, 22(2):111-121.
29. Kwakkel G., van Peppen R., Wagenaar R.C., Wood Dauphinee S., Richards C., Ashburn A., et al. Effects of augmented exercise therapy time after stroke: a meta-analysis. Stroke: A Journal of Cerebral Circulation 2004, 35(11):2529-2539.
30. Landau W.M., Sahrmann S.A. Preservation of directly stimulated muscle strength in hemiplegia due to stroke. Archives of Neurology 2002, 59(9):1453-1457.
31. Lang C.E., Wagner J.M., Edwards D.F., Dromerick A.W. Upper extremity use in people with hemiparesis in the first few weeks after stroke. Journal of Neurologic Physical Therapy: JNPT 2007, 31(2):56-63.
32. Langhorne P., Coupar F., Pollock A. Motor recovery after stroke: a systematic review. Lancet Neurology 2009, 8(8):741-754.
33. Lee D., Port N.L., Kruse W., Georgopoulos A.P. Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex. The Journal of Neuroscience 1998, 18:1161-1170.
34. Lemon R.N., Davis J.N., Yang H.W., Kirkwood P.A., Armand J., Maier M.A. Differences in the corticospinal projection from primary motor cortex and supplementary motor area to macaque upper limb motoneurons: an anatomical and electrophysiological study. Cerebral Cortex 2002, 12(3):281-296.
35. Lindberg P.G., Schmitz C., Engardt M., Forssberg H., Borg J. Use-dependent up- and down-regulation of sensorimotor brain circuits in stroke patients. Neurorehabilitation and Neural Repair 2007, 21(4):315-326.
36. Lindberg P., Schmitz C., Forssberg H., Engardt M., Borg J. Effects of passive-active movement training on upper limb motor function and cortical activation in chronic patients with stroke: a pilot study. Journal of Rehabilitation Medicine: Official Journal of the UEMS European Board of Physical and Rehabilitation Medicine 2004, 36(3):117-123.
37. Luft A.R., Schwarz S. Dopaminergic signals in primary motor cortex. International Journal of Developmental Neuroscience 2009, 27(5):415-421.
38. Lytton W.W., Stark J.M., Yamasaki D.S., Sober S.J. Computer models of stroke recovery: implications for neurorehabilitation. Neuroscientist 1999, 5:100-111.
39. Maier M.A., Perlmutter S.I., Fetz E.E. Response patterns and force relations of monkey spinal interneurons during active wrist movement. Journal of Neurophysiology 1998, 80(5):2495-2513.
40. Mazzoni P., Andersen R.A., Jordan M.I. A more biologically plausible learning rule for neural networks. Proceedings of the National Academy of Sciences of the United States of America 1991, 88(10):4433-4437.
41. Mewes K., Cheney P.D. Facilitation and suppression of wrist and digit muscles from single rubromotoneuronal cells in the awake monkey. Journal of Neurophysiology 1991, 66(6):1965-1977.
42. Mirbagheri M.M., Tsao C.C., Rymer W.Z. Changes of elbow kinematics and kinetics during 1 year after stroke. Muscle and Nerve 2008, 37(3):387-395.
43. Muellbacher W., Richards C., Ziemann U., Wittenberg G., Weltz D., Boroojerdi B., et al. Improving hand function in chronic stroke. Archives of Neurology 2002, 59(8):1278-1282.
44. Page S.J., Gater D.R., Bach-Y-Rita P. Reconsidering the motor recovery plateau in stroke rehabilitation. Archives of Physical Medicine and Rehabilitation 2004, 85(8):1377-1381.
45. Parry R.H., Lincoln N.B., Vass C.D. Effect of severity of arm impairment on response to additional physiotherapy early after stroke. Clinical Rehabilitation 1999, 13(3):187-198.
46. Perlmutter S.I., Maier M.A., Fetz E.E. Activity of spinal interneurons and their effects on forearm muscles during voluntary wrist movements in the monkey. Journal of Neurophysiology 1998, 80(5):2475-2494.
47. Powers R.K., Binder M.D. Input-output functions of mammalian motoneurons. Reviews of Physiology, Biochemistry and Pharmacology 2001, 143:137-263.
48. Reinkensmeyer D.J., Iobbi M.G., Kahn L.E., Kamper D.G., Takahashi C.D. Modeling reaching impairment after stroke using a population vector model of movement control that incorporates neural firing rate variability. Neural Computation 2003, 15(11):2619-2642.
49. Reinkensmeyer, D. J., Maier, M. A., Guigon, E., Chan, V., Akoner, O. M., & Wolbrecht, E. T. et al. (2009). Do robotic and non-robotic arm movement training drive motor recovery after stroke by a common neural mechanism? experimental evidence and a computational model. In Proceedings of the 2009 IEEE engineering in medicine and biology conference. 2009 (pp. 2439-2441).
50. Rijntjes M. Mechanisms of recovery in stroke patients with hemiparesis or aphasia: new insights, old questions and the meaning of therapies. Current Opinion in Neurology 2006, 19(1):76-83.
51. Rohrer B., Fasoli S., Krebs H.I., Hughes R., Volpe B., Frontera W.R., et al. Movement smoothness changes during stroke recovery. The Journal of Neuroscience 2002, 22(18):8297-8304.
52. Schweighofer N., Han C.E., Wolf S.L., Arbib M.A., Winstein C.J. A functional threshold for long-term use of hand and arm function can be determined: predictions from a computational model and supporting data from the extremity constraint-induced therapy evaluation (EXCITE) trial. Physical Therapy 2009, 89(12):1327-1336.
53. Sober S.J., Stark J.M., Yamasaki D.S., Lytton W.W. Receptive field changes after strokelike cortical ablation: a role for activation dynamics. Journal of Neurophysiology 1997, 78(6):3438-3443.
54. Stein R.B. Neuronal variability: noise or part of the signal?. Nature Reviews Neuroscience 2005, 6(5):389-397.
55. Stinear C.M. Functional potential in chronic stroke patients depends on corticospinal tract integrity. Brain 2007, 130(Pt. 1):170-180.
56. Sunderland A., Tinson D., Bradley L., Hewer R.L. Arm function after stroke. An evaluation of grip strength as a measure of recovery and a prognostic indicator. Journal of Neurology, Neurosurgery and Psychiatry 1989, 52(11):1267-1272.
57. Sutton R.S., Barto A.G. Reinforcement learning: an introduction 1998, MIT Press, Cambridge, MA.
58. Teasell R., Bitensky J., Salter K., Bayona N.A. The role of timing and intensity of rehabilitation therapies. Topics in Stroke Rehabilitation 2005, 12(3):46-57.
59. Thickbroom G.W. Motor outcome after subcortical stroke: MEPs correlate with hand strength but not dexterity. Clinical Neurophysiology 2002, 113(12):2025-2029.
60. van der Lee J., Snels I., Beckerman H., Lankhorst G., Wagenaar R., Bouter L. Exercise therapy for arm function in stroke patients: a systematic review of randomized controlled trial. Clinical Rehabilitation 2001, 15:20-31.
61. Ward N.S., Newton J.M., Swayne O.B., Lee L., Frackowiak R.S., Thompson A.J., et al. The relationship between brain activity and peak grip force is modulated by corticospinal system integrity after subcortical stroke. The European Journal of Neuroscience 2007, 25(6):1865-1873.
62. Werfel J., Xie X., Seung H.S. Learning curves for stochastic gradient descent in linear feedforward networks. Neural Computation 2005, 17(12):2699-2718.
63. Williams R.J. Simple statistical gradient following algorithms for connectionist reinforcement learning. Machine Learning 1992, 8:229-256.
64. Zhou P., Suresh N.L., Rymer W.Z. Model based sensitivity analysis of EMG-force relation with respect to motor unit properties: applications to muscle paresis in stroke. Annals of Biomedical Engineering 2007, 35(9):1521-1531.