What is the optimal task difficulty for reinforcement learning of brain self-regulation?

Clinical Neurophysiology

Volumn 127, Issue 9, 2016, Pages 3033-3041

  Bauer, Robert ^a   Vukelić, Mathias ^a   Gharabaghi, Alireza ^a  

^a UNIVERSITY OF TÜBINGEN   (Germany)

Author keywords

Beta modulation; Brain computer interface; Brain machine interface; Brain robot interface; Volitional control

Indexed keywords

ADULT; ANALYSIS OF VARIANCE; ARTICLE; AUTOREGULATION; BRAIN; BRAIN COMPUTER INTERFACE; CLASSIFICATION ALGORITHM; ELECTROENCEPHALOGRAPHY PHASE SYNCHRONIZATION; HUMAN; HUMAN EXPERIMENT; IMAGERY; MEASUREMENT ACCURACY; NEGATIVE FEEDBACK; NEUROFEEDBACK; NEUROMODULATION; NORMAL HUMAN; PERCEPTIVE THRESHOLD; POSITIVE FEEDBACK; PRIORITY JOURNAL; REINFORCEMENT; ROBOTICS; FEMALE; INSTRUMENTAL CONDITIONING; LEARNING; MALE; PHYSIOLOGY; PROCEDURES; PSYCHOLOGY; PSYCHOMOTOR PERFORMANCE; SELF CONTROL; YOUNG ADULT;

ADULT; BRAIN; BRAIN-COMPUTER INTERFACES; CONDITIONING, OPERANT; FEMALE; HUMANS; LEARNING; MALE; NEUROFEEDBACK; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT (PSYCHOLOGY); SELF-CONTROL; YOUNG ADULT;

EID: 85007001554     PISSN: 13882457     EISSN: 18728952     Source Type: Journal    
DOI: 10.1016/j.clinph.2016.06.016     Document Type: Article

References (64)

1. Alegre, M., Alvarez-Gerriko, I., Valencia, M., Iriarte, J., Artieda, J., Oscillatory changes related to the forced termination of a movement. Clin Neurophysiol 119 (2008), 290–300.
2. Ang, K.K., Guan, C., Phua, K.S., Wang, C., Zhou, L., Tang, K.Y., et al. Brain–computer interface-based robotic end effector system for wrist and hand rehabilitation: results of a three-armed randomized controlled trial for chronic stroke. Front Neuroeng 7 (2014), 1–9.
3. Bauer, R., Fels, M., Vukelić, M., Ziemann, U., Gharabaghi, A., Bridging the gap between motor imagery and motor execution with a brain–robot interface. NeuroImage 108 (2015), 319–327.
4. Bauer, R., Gharabaghi, A., Estimating cognitive load during self-regulation of brain activity and neurofeedback with therapeutic brain–computer interfaces. Front Behav Neurosci 9 (2015), 1–9.
5. Bauer, R., Gharabaghi, A., Reinforcement learning for adaptive threshold control of restorative brain–computer interfaces: a Bayesian simulation. Front Neurosci 9 (2015), 1–10.
6. Bauer, R., Fels, M., Royter, V., Raco, V., Gharabaghi, A., Closed-loop adaptation of neurofeedback based on mental effort facilitates reinforcement learning of brain self-regulation. Clin Neurophysiol, 2016, 10.1016/j.clinph.2016.06.020.
7. Brauchle, D., Vukelić, M., Bauer, R., Gharabaghi, A., Brain state-dependent robotic reaching movement with a multi-joint arm exoskeleton: combining brain–machine interfacing and robotic rehabilitation. Front Hum Neurosci 9 (2015), 1–13.
8. Buch, E.R., Modir Shanechi, A., Fourkas, A.D., Weber, C., Birbaumer, N., Cohen, L.G., Parietofrontal integrity determines neural modulation associated with grasping imagery after stroke. Brain J Neurol 135 (2012), 596–614.
9. Clark, R.C., Efficiency in learning: evidence-based guidelines to manage cognitive load. 2006, Jossey-Bass, San Francisco.
10. Collinger, J.L., Wodlinger, B., Downey, J.E., Wang, W., Tyler-Kabara, E.C., Weber, D.J., et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381 (2013), 557–564.
11. Erbil, N., Ungan, P., Changes in the alpha and beta amplitudes of the central EEG during the onset, continuation, and offset of long-duration repetitive hand movements. Brain Res 1169 (2007), 44–56.
12. Fels, M., Bauer, R., Gharabaghi, A., Predicting workload profiles of brain–robot interface and electromygraphic neurofeedback with cortical resting-state networks: personal trait or task-specific challenge?. J Neural Eng, 12, 2015, 046029.
13. Gharabaghi, A., Kraus, D., Leão, M.T., Spüler, M., Walter, A., Bogdan, M., et al. Coupling brain–machine interfaces with cortical stimulation for brain-state dependent stimulation: enhancing motor cortex excitability for neurorehabilitation. Front Hum Neurosci, 8, 2014, 122, 10.3389/fnhum.2014.00122.
14. Gharabaghi, A., Naros, G., Khademi, F., Jesser, J., Spüler, M., Walter, A., et al. Learned self-regulation of the lesioned brain with epidural electrocorticography. Front Behav Neurosci, 8, 2014, 429, 10.3389/fnbeh.2014.00429.
15. Gharabaghi, A., Naros, G., Walter, A., Grimm, F., Schuermeyer, M., Roth, A., et al. From assistance towards restoration with an implanted brain–computer interface based on epidural electrocorticography: a single case study. Restor Neurol Neurosci 32 (2014), 517–525, 10.3233/RNN-140387.
16. Gharabaghi, A., Naros, G., Walter, A., Roth, A., Bogdan, M., Rosenstiel, W., et al. Epidural electrocorticography of phantom hand movement following long-term upper-limb amputation. Front Hum Neurosci, 8, 2014, 285, 10.3389/fnhum.2014.00285.
17. Georges, J.S., Determining instructional efficiency. Sch Rev 39 (1931), 64–66.
18. Gomez-Rodriguez, M., Peters, J., Hill, J., Schölkopf, B., Gharabaghi, A., Grosse-Wentrup, M., Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery. J Neural Eng, 8, 2011, 036005.
19. Hagura, N., Takei, T., Hirose, S., Aramaki, Y., Matsumura, M., Sadato, N., et al. Activity in the posterior parietal cortex mediates visual dominance over kinesthesia. J Neurosci 27 (2007), 7047–7053.
20. Hall, S.D., Prokic, E.J., McAllister, C.J., Ronnqvist, K.C., Williams, A.C., Yamawaki, N., et al. GABA-mediated changes in inter-hemispheric beta frequency activity in early-stage Parkinson's disease. Neuroscience 281C (2014), 68–76.
21. Hammer, E.M., Halder, S., Blankertz, B., Sannelli, C., Dickhaus, T., Kleih, S., et al. Psychological predictors of SMR-BCI performance. Biol Psychol 89 (2012), 80–86.
22. Hinterberger, T., Neumann, N., Pham, M., Kübler, A., Grether, A., Hofmayer, N., et al. A multimodal brain-based feedback and communication system. Exp Brain Res 154 (2004), 521–526.
23. Jaušovec, N., Jaušovec, K., Increasing working memory capacity with theta transcranial alternating current stimulation (tACS). Biol Psychol 96C (2013), 42–47.
24. Kaiser, V., Kreilinger, A., Müller-Putz, G.R., Neuper, C., First steps toward a motor imagery based stroke BCI: new strategy to set up a classifier. Front Neurosci 5 (2011), 1–10.
25. Kavounoudias, A., Roll, J.P., Anton, J.L., Nazarian, B., Roth, M., Roll, R., Proprio-tactile integration for kinesthetic perception: an fMRI study. Neuropsychologia 46 (2008), 567–575.
26. Keinrath, C., Wriessnegger, S., Müller-Putz, G.R., Pfurtscheller, G., Post-movement beta synchronization after kinesthetic illusion, active and passive movements. Int J Psychophysiol 62 (2006), 321–327.
27. Kraus, D., Naros, G., Bauer, R., Khademi, F., Leão, M.T., Ziemann, U., Gharabaghi, A., Brain state-dependent transcranial magnetic closed-loop stimulation controlled by sensorimotor desynchronization induces robust increase of corticospinal excitability. Brain Stimul, 2016, 10.1016/j.brs.2016.02.007.
28. Kraus, D., Naros, G., Bauer, R., Leão, M.T., Ziemann, U., Gharabaghi, A., Brain–robot interface driven plasticity: distributed modulation of corticospinal excitability. NeuroImage 125 (2016), 522–532, 10.1016/j.neuroimage.2015.09.074.
29. Lim, M., Kim, J.S., Kim, M., Chung, C.K., Ascending beta oscillation from finger muscle to sensorimotor cortex contributes to enhanced steady-state isometric contraction in humans. Clin Neurophysiol 125 (2014), 2036–2045.
30. Lorenz, R., Pascual, J., Blankertz, B., Vidaurre, C., Towards a holistic assessment of the user experience with hybrid BCIs. J Neural Eng, 11, 2014, 035007.
31. Lotte, F., Larrue, F., Mühl, C., Flaws in current human training protocols for spontaneous brain–computer interfaces: lessons learned from instructional design. Front Hum Neurosci 7 (2013), 1–11.
32. Lundqvist, M., Herman, P., Lansner, A., Theta and gamma power increases and alpha/beta power decreases with memory load in an attractor network model. J Cogn Neurosci 23 (2011), 3008–3020.
33. Mary, A., Bourguignon, M., Wens, V., Op de Beeck, M., Leproult, R., De Tiège, X., et al. Aging reduces experience-induced sensorimotor plasticity. A magnetoencephalographic study. NeuroImage 104 (2015), 59–68.
34. Matsumoto, M., Takada, M., Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 79 (2013), 1011–1024.
35. McFarland, D.J., Wolpaw, J.R., Sensorimotor rhythm-based brain–computer interface (BCI): model order selection for autoregressive spectral analysis. J Neural Eng 5 (2008), 155–162.
36. Naros, G., Gharabaghi, A., Reinforcement learning of self-regulated β-oscillations for motor restoration in chronic stroke. Front Hum Neurosci 9 (2015), 1–12.
37. Naros, G., Naros, I., Grimm, F., Ziemann, U., Gharabaghi, A., Reinforcement learning of self-regulated sensorimotor β-oscillations improves motor performance. NeuroImage 134 (2016), 142–152.
38. Nijboer, F., Birbaumer, N., Kübler, A., The influence of psychological state and motivation on brain–computer interface performance in patients with amyotrophic lateral sclerosis – a longitudinal study. Front Neurosci 4 (2010), 1–13.
39. Ninaus, M., Kober, S.E., Witte, M., Koschutnig, K., Stangl, M., Neuper, C., et al. Neural substrates of cognitive control under the belief of getting neurofeedback training. Front Hum Neurosci 7 (2013), 1–10.
40. Oldfield, R.C., The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9 (1971), 97–113.
41. Oostenveld, R., Fries, P., Maris, E., Schoffelen, J.-M., FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011 (2011), 1–9.
42. Parker, J., Mountain, G., Hammerton, J., A review of the evidence underpinning the use of visual and auditory feedback for computer technology in post-stroke upper-limb rehabilitation. Disabil Rehabil Assist Technol 6 (2011), 1–8.
43. Pichiorri, F., Morone, G., Petti, M., Toppi, J., Pisotta, I., Molinari, M., et al. Brain–computer interface boosts motor imagery practice during stroke recovery. Ann Neurol 77 (2015), 851–865.
44. Proske, U., Gandevia, S.C., The kinaesthetic senses. J Physiol 587 (2009), 4139–4146.
45. Rossiter, H.E., Boudrias, M.-H., Ward, N.S., Do movement-related beta oscillations change after stroke?. J Neurophysiol 112 (2014), 2053–2058, 10.1152/jn.00345.2014.
46. Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R., BCI2000: a general-purpose brain–computer interface (BCI) system. IEEE Trans Biomed Eng 51 (2004), 1034–1043.
47. Schnotz, W., Kürschner, C., A reconsideration of cognitive load theory. Educ Psychol Rev 19 (2007), 469–508.
48. Sharma, N., Motor imagery: a backdoor to the motor system after stroke?. Stroke 37 (2006), 1941–1952.
49. Sherlin, L.H., Arns, M., Lubar, J., Heinrich, H., Kerson, C., Strehl, U., et al. Neurofeedback and basic learning theory: implications for research and practice. J Neurother 15 (2011), 292–304.
50. Sokolova, M., Lapalme, G., A systematic analysis of performance measures for classification tasks. Inf Process Manag 45 (2009), 427–437.
51. Spüler, M., Walter, A., Ramos-Murguialday, A., Naros, G., Birbaumer, N., Gharabaghi, A., Rosenstiel, W., Bogdan, M., Decoding of motor intentions from epidural EcoG recordings in severely paralyzed chronic stroke patients. J Neural Eng, 11, 2014, 066008, 10.1088/1741-2560/11/6/066008.
52. Spüler, M., Grimm, F., Gharabaghi, A., Bogdan, M., Rosenstiel, W., Comparing methods for decoding movement trajectory from ECoG in chronic stroke patients. Advances in neurotechnology, electronics and informatics, 2016, Springer International Publishing, 125–139.
53. Stinear, C.M., Byblow, W.D., Steyvers, M., Levin, O., Swinnen, S.P., Kinesthetic, but not visual, motor imagery modulates corticomotor excitability. Exp Brain Res 168 (2005), 157–164.
54. Takemi, M., Masakado, Y., Liu, M., Ushiba, J., Event-related desynchronization reflects downregulation of intracortical inhibition in human primary motor cortex. J Neurophysiol 110 (2013), 1158–1166.
55. Theodoridis, S., Koutroumbas, K., Pattern recognition. 2009, Academic Press, Burlington, MA; London.
56. Thomas, E., Dyson, M., Clerc, M., An analysis of performance evaluation for motor-imagery based BCI. J Neural Eng, 10, 2013, 031001.
57. Thompson, D.E., Blain-Moraes, S., Huggins, J.E., Performance assessment in brain–computer interface-based augmentative and alternative communication. Biomed Eng Online, 12, 2013, 43.
58. Vitense, H.S., Jacko, J.A., Emery, V.K., Multimodal feedback: an assessment of performance and mental workload. Ergonomics 46 (2003), 68–87.
59. Vukelić, M., Bauer, R., Naros, G., Naros, I., Braun, C., Gharabaghi, A., Lateralized alpha-band cortical networks regulate volitional modulation of beta-band sensorimotor oscillations. NeuroImage 87 (2014), 147–153.
60. Vukelić, M., Gharabaghi, A., Oscillatory entrainment of the motor cortical network during motor imagery is modulated by the feedback modality. NeuroImage 111 (2015), 1–11.
61. Vukelić, M., Gharabaghi, A., Self-regulation of circumscribed brain activity modulates spatially selective and frequency specific connectivity of distributed resting state networks. Front Behav Neurosci, 9, 2015, 10.3389/fnbeh.2015.00181.
62. Walter, A., Ramos Murguialday, A., Spüler, M., Naros, G., Leao, M.T., Gharabaghi, A., Rosenstiel, W., Birbaumer, N., Bogdan, M., Coupling BCI and cortical stimulation for brain-state-dependent stimulation: methods for spectral estimation in the presence of stimulation after-effects. Front Neural Circuits, 6, 2012, 87, 10.3389/fncir.2012.00087.
63. Wigfield, A., Eccles, J., Expectancy-value theory of achievement motivation. Contemp Educ Psychol 25 (2000), 68–81.
64. Witte, M., Kober, S.E., Ninaus, M., Neuper, C., Wood, G., Control beliefs can predict the ability to up-regulate sensorimotor rhythm during neurofeedback training. Front Hum Neurosci 7 (2013), 1–8.