A haemodynamic brain-computer interface based on real-time classification of near infrared spectroscopy signals during motor imagery and mental arithmetic

Journal of Near Infrared Spectroscopy

Volumn 21, Issue 3, 2013, Pages 157-171

  Stangl, Matthias ^a,b   Bauernfeind, Günther ^c   Kurzmann, Jürgen ^a   Scherer, Reinhold ^c   Neuper, Christa ^a,c  

^a UNIVERSITY OF GRAZ   (Austria)

^b GERMAN CENTER FOR NEURODEGENERATIVE DISEASES DZNE   (Germany)

^c GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Brain computer interface (BCI); Mental arithmetic; Motor imagery; Near infrared (NIR) spectroscopy; Real time classification

Indexed keywords

BIOMEDICAL SIGNAL PROCESSING; CALCULATIONS; IMAGE CLASSIFICATION; INFRARED DEVICES; NEAR INFRARED SPECTROSCOPY;

CLASSIFICATION ACCURACY; HAEMODYNAMIC RESPONSE; MENTAL ARITHMETIC; MOTOR IMAGERY; REAL TIME; SENSORIMOTOR CORTEX; SIGNAL ACQUISITIONS; SIGNAL CLASSIFICATION;

BRAIN COMPUTER INTERFACE;

EID: 84880283611     PISSN: 09670335     EISSN: 17516552     Source Type: Journal    
DOI: 10.1255/jnirs.1048     Document Type: Article

References (71)

1. J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller and T. M. Vaughan, "Brain-computer interfaces for communication and control", Clin. Neurophysiol. 113(6), 767 (2002). doi: 10.1016/S1388-2457 (02) 00057-3
2. N. Birbaumer, "Breaking the silence: brain-computer interfaces (BCI) for communication and motor control", Psychophysiology 43(6), 517 (2006). doi: 10.1111/1.1469-8986.2006.00456.x
3. R. Sitaram, Y. Hoshi and C. Guan, "Near infrared spectroscopy based brain-computer interface", Proc. SPIE 5852, 434 (2005).
4. A. Villringer and U. Dirnagl, "Coupling of brain activity and cerebral blood flow: basis of functional neuroimaging", Cerebrovasc. Brain Metab. 7(3), 240 (1995).
5. P. T. Fox and M. E. Raichle, "Focal physiological uncoupling of cerebral blood flow and oxidative metabolism during somatosensory stimulation in human subjects", Proc. Natl. Acad. Sci. USA 83(4), 1140 (1986). doi: 10.1073/pnas.83.4.1140
6. H. Obrig, C. Hirth, J. G. Junge-Hülsing, C. Doge, T. Wolf, U. Dirnagl and A. Villringer, "Cerebral oxygenation changes in response to motor stimulation", J. Appl. Physiol. 81(3), 1174 (1996).
7. A. Villringer and B. Chance, "Non-invasive optical spectroscopy and imaging of human brain function", Trends Neurosci. 20(10), 435 (1997). doi: 10.1016/S0166-2236 (97) 01132-6
8. G. Strangman, J. P. Culver, J. H. Thompson and D. A. Boas, "A quantitative comparison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation", NeuroImage 17(2), 719 (2002). doi: 10.1006/nimg.2002.1227
9. R. B. Buxton, K. Uludag, D. J. Dubowitz and T. T. Liu, "Modeling the hemodynamic response to brain activation", NeuroImage 23, 220 (2004). doi: 10.1016/j.neuroimage.2004.07.013
10. R. Sitaram, A. Caria and N. Birbaumer, "Hemodynamic brain-computer interfaces for communication and rehabilitation", Neural Netw. 22(9), 1320 (2009). doi: 10.1016/j.neu net.2009.05.009
11. S. Lloyd-Fox, A. Blasi and C. E. Elwell, "Illuminating the developing brain: the past, present and future of functional near infrared spectroscopy", Neurosci. Biobehav. Rev. 34(3), 269 (2010). doi: 10.1016/j.neubiorev. 2009.07.008.
12. T. Muehlemann, D. Haensse and M. Wolf, "Wireless miniaturized in-vivo near infrared imaging", Opt. Express 16(14), 10323 (2008). doi: 10.1364/0E.16.010323
13. L. Holper and M. Wolf, "Single-trial classification of motor imagery differing in task complexity: a functional near-infrared spectroscopy study", J. Neuro. Eng. Rehabil. 8, 34 (2011). doi: 10.1186/1743-0003-8-34.
14. G. Pfurtscheller, B. Z. Allison, C. Brunner, G. Bauernfeind, T. Solis-Escalante, R. Scherer, T. O. Zander, G. Mueller-Putz, C. Neuper and N. Birbaumer, "The hybrid BCI", Front. Neurosci. 4, 42 (2010).
15. S. Fazli, J. Mehnert, J. Steinbrink, G. Curio, A. Villringer, K.-R. Müller and B. Blankertz, "Enhanced performance by a hybrid NIRS-EEG brain computer interface", NeuroImage 59(1), 519 (2012). doi: 10.1016/j.neuroimage.2011.07.084
16. S. M. Coyle, T. E. Ward and C. M. Markham, "Braincomputer interface using a simplified functional nearinfrared spectroscopy system", J. Neural Eng. 4(3), 219 (2007). doi: 10.1088/1741-2560/4/3/007
17. A. F. Abdelnour and T. Huppert, "Real-time imaging of human brain function by near-infrared spectroscopy using an adaptive general linear model", NeuroImage 46(1), 133 (2009). doi: 10.1016/j.neuroimage.2009.01.033
18. H. Ayaz, P. A. Shewokis, S. Bunce, M. Schultheis and B. Onaral, "Assessment of cognitive neural correlates for a functional near infrared-based brain computer interface system", in Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience, Ed by D. D. Schmorrow, I. V. Estabrooke and M. Grootjen. Springer Verlag, Berlin, Germany, p. 699 (2009).
19. E. V. Friedrich, R. Scherer and C. Neuper, "The effect of distinct mental strategies on classification performance for brain-computer interfaces", Int. J. Psychophysiol. 84(1), 86 (2012). doi: 10.1016/j.ijpsycho.2012.01.014
20. J. Decety and J. Grèzes, "Neural mechanisms subserving the perception of human actions", Trends Cogn. Sci. 3(5), 172 (1999). doi: 10.1016/S1364-6613 (99) 01312-1
21. M. Jeannerod, "The representing brain: neural correlates of motor intention and imagery", Behav. Brain Sci. 17, 187 (1994). doi: 10.1017/S0140525X00034026
22. J. Decety, "Do imagined and executed actions share the same neural substrate?", Cogn. Brain Res. 3, 87 (1996). doi: 10.1016/0926-6410 (95) 00033-X
23. J. Decety, "The neurophysiological basis of motor imagery", Behav. Brain Res. 77(1-2), 45 (1996). doi: 10.1016/0166-4328 (95) 00225-1
24. S. M. Coyle, T. E. Ward, C. M. Markham and G. McDarby, "On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces", Physiol. Meas. 25(4), 815 (2004). doi: 10.1088/0967-3334/25/4/ 003
25. R. Sitaram, H. Zhang, C. Guan, M. Thulasidas, Y. Hoshi, A. Ishikawa, K. Shimizu and N. Birbaumer, "Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a braincomputer interface", NeuroImage 34(4), 1416 (2007). doi: 10.1016/j.neuroimage.2006.11.005
26. S. C. Wriessnegger, J. Kurzmann and C. Neuper, "Spatiotemporal differences in brain oxygenation between movement execution and imagery: A multichannel nearinfrared spectroscopy study", Int. J. Psychophysiol. 67(1), 54 (2008). doi: 10.1016/j.ijpsycho.2007.10.004
27. L. Holper and M. Wolf, "Motor imagery in response to fake feedback measured by functional near-infrared spectroscopy", NeuroImage 50(1), 190 (2010). doi: 10.1016/j.neuroimage.2009.12.055
28. G. Pfurtscheller, C. Neuper, A. Schlögl and K. Lugger, "Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters", IEEE Trans. Rehabil. Eng. 6(3), 316 (1998).
29. G. Pfurtscheller, C. Neuper, C. Guger, W. Harkam, H. Ramoser, A. Schlögl, B. Obermaier and M. Pregenzer, "Current trends in Graz brain-computer interface (BCI) research", IEEE Trans. Rehabil. Eng. 8(2), 216 (2000).
30. G. Pfurtscheller and C. Neuper, "Motor imagery and direct brain-computer communication", Proc. IEEE 89, 1123 (2001).
31. C. Neuper, G. R. Müller-Putz, R. Scherer and G. Pfurtscheller, "Motor imagery and EEG-based control of spelling devices and neuroprostheses", Prog. Brain Res. 159, 393 (2006). doi: 10.1016/S0079-6123 (06) 59025-9
32. M. Arsalidou and M. J. Taylor, "Is 2+2=4? Meta-analyses of brain areas needed for numbers and calculations", NeuroImage 54(3), 2382 (2011). doi: 10.1016/j.neuroimage.2010.10.009
33. G. Bauernfeind, R. Leeb, S. C. Wriessnegger and G. Pfurtscheller, "Development, set-up and first results for a one-channel near-infrared spectroscopy system", Biomed. Eng. 53(1), 36 (2008). doi: 10.1515/BMT.2008.005
34. G. Bauernfeind, R. Scherer, G. Pfurtscheller and C. Neuper, "Single-trial classification of antagonistic oxyhemoglobin responses during mental arithmetic", Med. Biol. Eng. Comput. 49(9), 979 (2011). doi: 10.1007/s11517-011-0792-5
35. G. Pfurtscheller, G. Bauernfeind, S. C. Wriessnegger and C. Neuper, "Focal frontal (de) oxyhemoglobin responses during simple arithmetic", Int. J. Psychophysiol. 76(3), 186 (2010). doi: 10.1016/j.i jpsycho.2010.03.013
36. S. D. Power, T. H. Falk and T. Chau, "Classification of prefrontal activity due to mental arithmetic and music imagery using hidden Markov models and frequency domain near-infrared spectroscopy", J. Neural Eng. 7(2), 26002 (2010). doi: 10.1088/1741-2560/7/2/026002
37. S. D. Power, A. Kushki and T. Chau, "Towards a system-paced near-infrared spectroscopy brain-computer interface: differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state", J. Neural Eng. 8, 66004 (2011). doi: 10.1088/1741-2560/8/6/066004
38. S. D. Power, A. Kushki and T. Chau, "Automatic singletrial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI", BMC Res. Notes 5, 141 (2012). doi: 10.1186/1756-0500-5-141
39. C. Neuper, R. Scherer, M. Reiner and G. Pfurtscheller, "Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG", Cogn. Brain Res. 25, 668 (2005). doi: 10.1016/j.cogbrainres.2005.08.014
40. K. Uludag, J. Steinbrink, A. Villringer and H. Obrig, "Separability and cross talk: optimizing dual wavelength combinations for near-infrared spectroscopy of the adult head", NeuroImage 22(2), 583 (2004). doi: 10.1016/j.neuroimage.2004.02.023
41. R. W. Homan, J. Herman and P. Purdy, "Cerebral location of international 10-20 system electrode placement", Electroenceph. Clin. Neurophysiol. 66(4), 376 (1987). doi: 10.1016/0013-4694 (87) 90206-9
42. Y. Hoshi, N. Kobayashi and M. Tamura, "Interpretation of near-infrared spectroscopy signals: a study with a newly developed perfused rat brain model", J. Appl. Physiol. 90(5), 1657 (2001).
43. L. Holper, N. Kobashi, D. Kiper, F. Scholkmann, M. Wolf and K. Eng, "Trial-to-trial variability differentiates motor imagery during observation between low versus high responders: A functional near-infrared spectroscopy study", Behav. Brain Res. 229(1), 29 (2012). doi: 10.1016/j. bbr. 2011.12.038
44. R. O. Duda, P. E. Hart and D. G. Stork, Pattern Classification. John Wiley & Sons, Inc., New York, USA (2001).
45. D. A. Benaron, S. R. Hintz, A. Villringer, D. Boas, A. Kleinschmidt, J. Frahm, C. Hirth, H. Obrig, J. C. van Houten, E. L. Kermit, W. F. Cheong and D. K. Stevenson, "Noninvasive functional imaging of human brain using light", J. Cereb. Blood Flow Metab. 20(3), 469 (2000).
46. F. E. Grubbs, "Procedures for detecting outlying observations in samples", Technometrics 11, 1 (1969). doi: 10.1080/00401706.1969.10490657
47. B. Shoelson, "deleteoutliers", MATLAB Central. Available: www.mathworks.com/matlabcentral/fileexchange/3961. Accessed: 9 April (2010).
48. A. W. Whitney, "A direct method of nonparametric measurement selection", IEEE, Trans. Comput. C-20, 1100 (1971).
49. J. Reunanen, "Search strategies", in Feature Extraction: Foundations and Applications. Ed by I. Guyon, S. Gunn, M. Nikravesh and L. A. Zadeh. Springer Verlag, Berlin, Germany, p. 119 (2006).
50. P. A. Lachenbruch and M. R. Mickey, "Estimation of error rates in discriminant analysis", Technometrics 10, 1 (1968). doi: 10.1080/00401706.1968.10490530
51. G. R. Müller-Putz, R. Scherer, C. Brunner, R. Leeb and G. Pfurtscheller, "Better than random? A closer look on BCI results", Int. J. Bioelectromagn. 10, 52 (2008).
52. M. Naito, Y. Michioka, K. Ozawa, Y. Ito, M. Kiguchi and T. Kanazawa, "A communication means for totally lockedin ALS patients based on changes in cerebral blood volume measured with near-infrared light", IEICETrans. Inf. Syst. 90, 1028 (2007).
53. B. Blankertz, C. Sannelli, S. Haider, E. M. Hammer, A. Kübler, K.-R. Müller, G. Curio and T. Dickhaus, "Neurophysiological predictor of SMR-based BCI performance", NeuroImage 51(4), 1303 (2010). doi: 10.1016/j. neuroimage.2010.03.022
54. S. D. Power, A. Kushki and T. Chau, "Intersession consistency of single-trial classification of the prefrontal response to mental arithmetic and the no-control state by NIRS", PLoS ONE 7(7), e37791 (2012). doi: 10.1371/journal.pone.0037791
55. S. Luu and T. Chau, "Decoding subjective preference from single-trial near-infrared spectroscopy signals", J. Neural Eng. 6(1), 16003 (2009). doi: 10.1088/1741-2560/6/1/016003
56. B. Abibullaev and J. An and J.-I. Moon, "Neural network classification of brain hemodynamic responses from four mental tasks", Int. J. Optomechatronics. 5, 340 (2011). doi: 10.1080/15599612.2011.633209
57. N. De Pisapia, J. A. Slomski and T. S. Braver, "Functional specializations in lateral prefrontal cortex associated with the integration and segregation of information in working memory", Cereb. Cortex 17(5), 993 (2007). doi: 1007-1019 doi:10.1093/cercor/bhl011
58. O. Gruber, P. Indefrey, H. Steinmetz and A. Kleinschmidt, "Dissociating neural correlates of cognitive components in mental calculation", Cereb. Cortex 11(4), 350 (2001). doi: 10.1093/cercor/11.4.350
59. L. Zago, M. Pesenti, E. Mellet, F. Crivello, B. Mazoyer and N. Tzourio-Mazoyer, "Neural correlates of simple and complex mental calculation", NeuroImage 13(2), 314 (2001). doi: 10.1016/S1053-8119 (01) 91657-X
60. M. Delazer, F. Domahs, L. Bartha, C. Brenneis, A. Lochy, T. Trieb and T. Benke, "Learning complex arithmetic-an fMRI study", Cogn. Brain Res. 18, 76 (2003). doi: 10.1016/j. cogbrainres.2003.09.005
61. M. Tanida, M. Katsuyama and K. Sakatani, "Relation between mental stress-induced prefrontal cortex activity and skin conditions: a near-infrared spectroscopy study", Brain Res. 1184, 210 (2007). doi: 10.1016/j.brainres. 2007.09.058
62. M. Tanida, M. Katsuyama and K. Sakatani, "Effects of fragrance administration on stress-induced prefrontal cortex activity and sebum secretion in the facial skin", Neurosci. Lett. 432(2), 157 (2008). doi: 10.1016/j.neulet.2007.12.014
63. J. Kong, C. Wang, K. Kwong, M. Vangel, E. Chua and R. Gollub, "The neural substrate of arithmetic operations and procedure complexity", Cogn. Brain Res. 22(3), 397 (2005). doi: 10.1016/j.cogbrainres.2004.09.011
64. C. Hock, F. Müller-Spahn, S. Schuh-Hofer, M. Hofmann, U. Dirnagl and A. Villringer, "Age dependency of changes in cerebral hemoglobin oxygenation during brain activation: a near-infrared spectroscopy study", J. Cereb. Blood Flow Metab. 15(6), 1103 (1995). doi: 10.1038/jcbfm.1995.137
65. Y. Hoshi, H. Onoe, Y. Watanabe, J. Andersson, M. Bergström, A. Lilja, B. Langström and M. Tamura, "Non-synchronous behavior of neuronal activity, oxidative metabolism and blood supply during mental tasks in man", Neurosci. Lett. 172(1-2), 129 (1994). doi: 10.1016/0304-3940 (94) 90679-3
66. Y. Hoshi and M. Tamura, "Detection of dynamic changes in cerebral oxygenation coupled to neuronal function during mental work in man", Neurosci. Lett. 150(1), 5 (1993). doi: 10.1016/0304-3940 (93) 90094-2
67. M. Tanida, K. Sakatani, R. Takano and K. Tagai, "Relation between asymmetry of prefrontal cortex activities and the autonomic nervous system during a mental arithmetic task: near infrared spectroscopy study", Neurosci. Lett. 369(1), 69 (2004). doi: 10.1016/j.neulet.2004.07.076
68. T. Fehr, C. Code and M. Herrmann, "Common brain regions underlying different arithmetic operations as revealed by conjunct fMRI-BOLD activation", Brain Res. 1172, 93 (2007). doi: 10.1016/l.brainres.2007.07. 043
69. O. Houdé and N. Tzourio-Mazoyer, "Neural foundations of logical and mathematical cognition", Nat. Rev. Neurosci. 4(6), 507 (2003). doi: 10.1038/nrn1117
70. F. Matthews, B. A. Pearlmutter, T. E. Ward, C. Soraghan and C. Markham, "Hemodynamics for brain-computer interfaces", IEEE. Signal Process. Mag. 25, B7 (2008). doi: 10.1109/MSP.200B.440B445
71. A. K. Singh, M. Okamoto, H. Dan, V. Jurcak and I. Dan, "Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI", NeuroImage 27(4), 842 (2005). doi: 10.1016/l.neuroimage.2005.05.019