Reinforcement learning of self-regulated β-oscillations for motor restoration in chronic stroke

Frontiers in Human Neuroscience

Volumn 9, Issue JULY, 2015, Pages

  Naros, Georgios ^a   Gharabaghi, Alireza ^a  

^a UNIVERSITY OF TÜBINGEN   (Germany)

Author keywords

Beta oscillations; Brain computer interface; Brain machine interface; Brain robot interface; Functional restoration; Hand function; Reinforcement learning; Stroke

Indexed keywords

BODY MASS; CLASSIFICATION; CLINICAL STUDY; FEASIBILITY STUDY; HUMAN; IMAGERY; INSTRUMENTAL CONDITIONING; LEARNING; MUSCLE; NEUROFEEDBACK; OSCILLATION; PHYSIOTHERAPY; REHABILITATION; REINFORCEMENT; REST; REWARD; STROKE PATIENT;

EID: 84947572857     PISSN: None     EISSN: 16625161     Source Type: Journal    
DOI: 10.3389/fnhum.2015.00391     Document Type: Article

References (69)

1. Allal, L., and Pelgrims Ducrey, G. (2000). Assessment of—or in—the zone of proximal development. Learn. Instr. 10, 137-152. doi: 10.1016/s0959-4752(99)00025-0
2. Ang, K. K., Chua, K. S. G., Phua, K. S., Wang, C., Chin, Z. Y., Kuah, C. W. K., et al. (2014). A Randomized Controlled Trial of EEG-Based Motor Imagery Brain- Computer Interface Robotic Rehabilitation for Stroke. Clin. EEG Neurosci. doi:10.1177/1550059414522229 [Epub ahead ofprint].
3. Ang, K. K., Guan, C., Chua, K. S. G., Ang, B. T., Kuah, C., Wang, C., et al. (2010). Clinical study of neurorehabilitation in stroke using EEG- based motor imagery brain-computer interface with robotic feedback. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2010, 5549-5552. doi: 10.1109/IEMBS.2010.5626782
4. Ang, K. K., Guan, C., Chua, K. S., Ang, B., Kuah, C., Wang, C., et al. (2011). A large clinical study on the ability of stroke patients to use an EEG-based motor imagery brain-computer interface. Clin. EEG Neurosci. 42, 253-258. doi:10.1177/155005941104200411
5. Aumann, T. D., and Prut, Y. (2014). Do sensorimotor ²-oscillations maintain muscle synergy representations in primary motor cortex? Trends Neurosci. 38, 77-85. doi: 10.1016/j.tins.2014.12.002
6. Bai, O., Huang, D., Fei, D.-Y., and Kunz, R. (2014). Effect of real­time cortical feedback in motor imagery-based mental practice training. NeuroRehabilitation 34, 355-363. doi: 10.3233/NRE-131039
7. Bai, O., Lin, P., Vorbach, S., Floeter, M. K., Hattori, N., and Hallett, M. (2008). A high performance sensorimotor beta rhythm-based brain-computer interface associated with human natural motor behavior. J. Neural Eng. 5, 24-35. doi: 10.1088/1741-2560/5/1/003
8. Bauer, R., Fels, M., Vukelic, M., Ziemann, U., and Gharabaghi, A. (2015). Bridging the gap between motor imagery and motor execution with a brain-robot interface. Neuroimage 108, 319-327. doi: 10.1016/j.neuroimage.2014.12.026
9. Bauer, R., and Gharabaghi, A. (2015a). Estimating cognitive load during self­regulation of brain activity and neurofeedback with therapeutic brain- computer interfaces. Front. Behav. Neurosci. 9:21. doi: 10.3389/fnbeh.2015.00021
10. Bauer, R., and Gharabaghi, A. (2015b). Reinforcement learning for adaptive threshold control of restorative brain-computer interfaces: a Bayesian simulation. Front. Neurosci. 9:36. doi: 10.3389/fnins.2015.00036
11. Boe, S., Gionfriddo, A., Kraeutner, S., Tremblay, A., Little, G., and Bardouille, T. (2014). Laterality of brain activity during motor imagery is modulated by the provision of source level neurofeedback. Neuroimage 101, 159-167. doi:10.1016/j.neuroimage.2014.06.066
12. Brinkman, L., Stolk, A., Dijkerman, H. C., de Lange, F. P., and Toni, I.. Distinct Roles for Alpha- and Beta-Band Oscillations during Mental Simulation of Goal-Directed Actions. J. Neurosci. 34, 14783-14792. doi: 10.1523/JNEUR0SCI.2039-14.2014
13. Brittain, J.-S., Sharott, A., and Brown, P. (2014). The highs and lows ofbeta activity in cortico-basalganglialoops. Eur.J.Neurosci. 39, 1951-1959. doi: 10.1111/ejn.12574
14. Buch, E. R., Modir Shanechi, A., Fourkas, A. D., Weber, C., Birbaumer, N., and Cohen, L. G. (2012). Parietofrontal integrity determines neural modulation associated with grasping imagery after stroke. Brain 135, 596-614. doi: 10.1093/brain/awr331
15. Buch, E., Weber, C., Cohen, L. G., Braun, C., Dimyan, M. A., Ard, T., et al. (2008). Think to move: a neuromagnetic brain-computer interface (BCI) systemfor chronic stroke. Stroke 39, 910-917. doi: 10.1161/STR0KEAHA.107.505313
16. Carmena, J. M. (2013). Advances in neuroprosthetic learning and control. PLoS Biol. 11:e1001561. doi: 10.1371/journal.pbio.1001561
17. Daly, J. J., and Wolpaw, J. R. (2008). Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 7, 1032-1043. doi: 10.1016/S1474-4422(08)70223-0
18. Davis, N. J., Tomlinson, S. P., and Morgan, H. M. (2012). The role of ²-frequency neural oscillations in motor control. J. Neurosci. 32, 403-404. doi: 10.1523/JNEUROSCI.5106-11.2012
19. Di Pino, G., Pellegrino, G., Assenza, G., Capone, F., Ferreri, F., Formica, D., et al. (2014). Modulation of brain plasticity in stroke: a novel model for neurorehabilitation. Nat. Rev. Neurol. 10, 597-608. doi: 10.1038/nrneurol.2014.162
20. Doyon, J., and Benali, H. (2005). Reorganization and plasticity in the adult brain during learning of motor skills. Curr. Opin. Neurobiol. 15, 161-167. doi:10.1109/ijcnn.2005.1556102
21. Fugl-Meyer, A. R., Jaasko, L., Leyman, I., Olsson, S., and Steglind, S. (1975). The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand. J. Rehabil. Med. 7, 13-31.
22. Gao, Q., Duan, X., and Chen, H. (2011). Evaluation of effective connectivity of motor areas during motor imagery and execution using conditional Granger causality. Neuroimage 54, 1280-1288. doi: 10.1016/j.neuroimage.2010.08.071
23. Gharabaghi, A., Kraus, D., Leao, M. T., Spuler, M., Walter, A., Bogdan, M., et al. (2014a). Coupling brain-machine interfaces with cortical stimulation for brain-state dependent stimulation: enhancing motor cortex excitability for neurorehabilitation. Front. Hum. Neurosci. 8:122. doi: 10.3389/fnhum.2014.00122
24. Gharabaghi, A., Naros, G., Khademi, F., Jesser, J., Spuler, M., Walter, A., et al. (2014b). Learned self-regulation of the lesioned brain with epidural electrocorticography. Front. Behav. Neurosci. 8:429. doi: 10.3389/fnbeh.2014.00429
25. Gharabaghi, A., Naros, G., Walter, A., Grimm, F., Schuermeyer, M., Roth, A., et al. (2014c). From assistance towards restoration with an implanted brain- computer interface based on epidural electrocorticography: A single case study. Restor. Neurol. Neurosci. 32, 517-525. doi: 10.3233/RNN-140387
26. Gomez-Rodriguez, M., Peters, J., Hill, J., Scholkopf, B., Gharabaghi, A., and Grosse-Wentrup, M. (2011). Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery. J. Neural Eng. 8:036005. doi: 10.1088/1741-2560/8/3/036005
27. Halsband, U., and Lange, R. K. (2006). Motor learning in man: a review of functional and clinical studies. J. Physiol. Paris 99, 414-424. doi: 10.1016/j.jphysparis.2006.03.007
28. Herrojo Ruiz, M., Brucke, C., Nikulin, V. V., Schneider, G.-H., and Kuhn, A. A. Beta-band amplitude oscillations in the human internal globus pallidus support the encoding of sequence boundaries during initial sensorimotor sequence learning. Neuroimage 85, 779-793. doi: 10.1016/j.neuroimage.2013.05.085
29. Jackson, A., and Fetz, E. E. (2011). Interfacing with the computational brain. IEEE Trans. NeuralSyst. Rehabil. Eng. 19, 534-541. doi: 10.1109/TNSRE.2011.2158586
30. Kilavik, B. E., Zaepffel, M., Brovelli, A., MacKay, W. A., and Riehle, A. (2013). The ups and downs of fi oscillations in sensorimotor cortex. Exp. Neurol. 245, 15-26. doi:10.1016/j.expneurol.2012.09.014
31. Klamroth-Marganska, V., Blanco, J., Campen, K., Curt, A., Dietz, V., Ettlin, T., et al. (2014). Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial. Lancet. Neurol. 13, 159-166. doi:10.1016/s1474-4422(13)70305-3
32. Kristeva, R., Patino, L., and Omlor, W. (2007). Beta-range cortical motor spectral power and corticomuscular coherence as a mechanism for effective corticospinal interaction during steady-state motor output. Neuroimage 36, 785-792. doi:10.1016/j.neuroimage.2007.03.025
33. Kwakkel, G., Kollen, B. J., and Krebs, H. I. (2008). Effects ofrobot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil. Neural Repair 22, 111-121. doi: 10.1177/1545968307305457
34. Lo, A. C., Guarino, P. D., Richards, L. G., Haselkorn, J. K., Wittenberg, G. F., Federman, D. G., et al. (2010). Robot-assisted therapy for long-term upper- limb impairment after stroke. N. Engl. J. Med. 362, 1772-1783. doi: 10.1056/NEJMoa0911341
35. Lohse, K. R., Lang, C. E., and Boyd, L. A. (2014). Is more better? Using metadata to explore dose-response relationships in stroke rehabilitation. Stroke 45, 2053-2058. doi:10.1161/STROKEAHA.114.004695
36. Mazaheri, A., and Jensen, O. (2010). Rhythmic pulsing: linking ongoing brain activity with evoked responses. Front. Hum. Neurosci. 4:177. doi: 10.3389/fnhum.2010.00177
37. McFarland, D. J., and Wolpaw, J. R. (2008). Sensorimotor rhythm-based brain- computer interface (BCI): model order selection for autoregressive spectral analysis. J. NeuralEng. 5, 155-162. doi: 10.1088/1741-2560/5/2/006
38. Mellinger, J., and Schalk, G. (2007). ‘‘BCI2000: A General-Purpose Software Platform for BCI Research,’’ in Brain-Computer Interfaces, (Cambridge: MIT Press).
39. Mima, T., Steger, J., Schulman, A. E., Gerloff, C., and Hallett, M. (2000). Electroencephalographic measurement of motor cortex control of muscle activity in humans. Clin. Neurophysiol. 111, 326-337. doi: 10.1016/s1388-2457(99)00229-1
40. Mrachacz-Kersting, N., Kristensen, S. R., Niazi, I. K., and Farina, D. (2012). Precise temporal association between cortical potentials evoked by motor imagination and afference induces cortical plasticity. J. Physiol. 590, 1669-1682. doi:10.1113/jphysiol.2011.222851
41. Niazi, I. K., Mrachacz-Kersting, N., Jiang, N., Dremstrup, K., and Farina, D. (2012). Peripheral electrical stimulation triggered by self-paced detection of motor intention enhances motor evoked potentials. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 595-604. doi: 10.1109/tnsre.2012.2194309
42. Pichiorri, F., Morone, G., Petti, M., Toppi, J., Pisotta, I., Molinari, M., et al. Brain-computer interface boosts motor imagery practice during stroke recovery. Ann. Neurol. 77, 851-865. doi: 10.1002/ana.24390
43. Plow, E. B., and Machado, A. (2014). Invasive neurostimulation in stroke rehabilitation. Neurotherapeutics 11, 572-582. doi: 10.1007/s13311-013-0245-y
44. Pollok, B., Latz, D., Krause, V., Butz, M., and Schnitzler, A. (2014). Changes of motor-cortical oscillations associated with motor learning. Neuroscience 275, 47-53. doi:10.1016/j.neuroscience.2014.06.008
45. Prasad, G., Herman, P., Coyle, D., McDonough, S., and Crosbie, J. (2010). Applying a brain-computer interface to support motor imagery practice in people with stroke for upper limb recovery: a feasibility study. J. Neuroeng. Rehabil. 7:60. doi: 10.1186/1743-0003-7-60
46. Ramos-Murguialday, A., Broetz, D., Rea, M., Laer, L., Yilmaz, O., Brasil, F. L., et al. (2013). Brain-machine interface in chronic stroke rehabilitation: a controlled study. Ann. Neurol. 74, 100-108. doi: 10.1002/ana.23879
47. Ramos-Murguialday, A., Schurholz, M., Caggiano, V., Wildgruber, M., Caria, A., Hammer, E. M., et al. (2012). Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses. PLoS One 7:e47048. doi:10.1371/journal.pone.0047048
48. Riddle, C. N., and Baker, S. N. (2005). Manipulation of peripheral neural feedback loops alters human corticomuscular coherence. J. Physiol. 566, 625-639. doi:10.1113/jphysiol.2005.089607
49. Rossiter, H. E., Boudrias, M.-H., and Ward, N. S. (2014). Do movement-related beta oscillations change after stroke? J. Neurophysiol. 112, 2053-2058. doi:10.1152/jn.00345.2014
50. Safrit, M. J., Cohen, A. S., and Costa, M. G. (1989). Item response theory and the measurement of motor behavior. Res. Q. Exerc. Sport 60, 325-335. doi:10.1080/02701367.1989.10607459
51. Schnotz, W., and Kurschner, C. (2007). A Reconsideration of Cognitive Load Theory. Educ. Psychol. Rev. 19, 469-508. doi: 10.1007/s10648-007-9053-4
52. Shindo, K., Kawashima, K., Ushiba, J., Ota, N., Ito, M., Ota, T., et al. (2011). Effects of neurofeedback training with an electroencephalogram- based brain-computer interface for hand paralysis in patients with chronic stroke: a preliminary case series study. J. Rehabil. Med. 43, 951-957. doi: 10.2340/16501977-0859
53. Stinear, C. M., Barber, P. A., Petoe, M., Anwar, S., and Byblow, W. D. (2012). The PREP algorithm predicts potential for upper limb recovery after stroke. Brain 135, 2527-2535. doi: 10.1093/brain/aws146
54. Stinear, C. M., Barber, P. A., Smale, P. R., Coxon, J. P., Fleming, M. K., and Byblow, W. D. (2007). Functional potential in chronic stroke patients depends on corticospinal tract integrity. Brain 130, 170-180. doi: 10.1093/brain/awl333
55. Sweller, J. (1994). Cognitive load theory, learning difficulty and instructional design. Learn. Instr. 4, 295-312. doi: 10.1016/0959-4752(94)90003-5
56. Szameitat, A. J., Shen, S., Conforto, A., and Sterr, A. (2012). Cortical activation during executed, imagined, observed and passive wrist movements in healthy volunteers and stroke patients. Neuroimage 62, 266-280. doi: 10.1016/j.neuroimage.2012.05.009
57. Takemi, M., Masakado, Y., Liu, M., and Ushiba, J. (2013a). Event-related desynchronization reflects downregulation ofintracortical inhibition in human primary motor cortex. J. Neurophysiol. 110, 1158-1166. doi: 10.1152/jn.01092.2012
58. Takemi, M., Masakado, Y., Liu, M., and Ushiba, J. (2013b). Is event-related desynchronization a biomarker representing corticospinal excitability? Conf. Proc.IEEE. Eng. Med. Biol. Soc. 2013, 281-284. doi: 10.1109/EMBC.2013.6609492
59. Theodoridis, S., and Koutroumbas, K. (2009). Pattern Recognition. Burlington, MA: Academic Press.
60. Thomas, E., Dyson, M., and Clerc, M. (2013). An analysis of performance evaluation for motor-imagery based BCI. J. Neural Eng. 10:031001. doi:10.1088/1741-2560/10/3/031001
61. Thompson, D. E., Blain-Moraes, S., and Huggins, J. E. (2013). Performance assessment in brain-computer interface-based augmentative and alternative communication. Biomed. Eng. Online 12:43. doi: 10.1186/1475-925x-12-43
62. van Wijk, B. C. M., Beek, P. J., and Daffertshofer, A. (2012). Neural synchrony within the motor system: what have we learned so far? Front. Hum. Neurosci. 6:252. doi: 10.3389/fnhum.2012.00252
63. Vidaurre, C., and Blankertz, B. (2010). Towards a cure for BCI illiteracy. Brain Topogr. 23, 194-198. doi: 10.1007/s10548-009-0121-6
64. Vukelic, M., Bauer, R., Naros, G., Naros, I., Braun, C., and Gharabaghi, A. (2014). Lateralized alpha-band cortical networks regulate volitional modulation of beta-band sensorimotor oscillations. Neuroimage 87, 147-153. doi:10.1016/j.neuroimage.2013.10.003
65. Vukelic, M., and Gharabaghi, A. (2015). Oscillatory entrainment of the motor cortical network during motor imagery is modulated by the feedback modality. Neuroimage 111, 1-11. doi: 10.1016/j.neuroimage.2015.01.058
66. Walter, A., Ramos Murguialday, A., Spuler, M., Naros, G., Leao, M. T., and Gharabaghi, A. (2012). Coupling BCI and cortical stimulation for brain-state-dependent stimulation: methods for spectral estimation in the presence of stimulation after-effects. Front. Neural Circuits 6:87. doi: 10.3389/fncir.2012.00087
67. Witham, C. L., Riddle, C. N., Baker, M. R., and Baker, S. N. (2011). Contributions of descending and ascending pathways to corticomuscular coherence in humans. J. Physiol. 589, 3789-3800. doi: 10.1113/jphysiol.2011.211045
68. Xu, R., Jiang, N., Mrachacz-Kersting, N., Lin, C., Asm Prieto, G., Moreno, J. C., et al. (2014).Aclosed-loop brain-computer interface triggering an active ankle- foot orthosis for inducing cortical neural plasticity. IEEE Trans. Biomed. Eng. 61,2092-2101. doi:10.1109/tbme.2014.2313867
69. Zoefel, B., Huster, R. J., and Herrmann, C. S. (2011). Neurofeedback training of the upper alpha frequency band in EEG improves cognitive performance. Neuroimage 54, 1427-1431. doi: 10.1016/j.neuroimage.2010.08.078