Controlling an avatar by thought using real-time fMRI

Journal of Neural Engineering

Volumn 11, Issue 3, 2014, Pages

  Cohen, Ori ^a,b   Koppel, Moshe ^b   Malach, Rafael ^c   Friedman, Doron ^a  

^a INTERDISCIPLINARY CENTER   (Israel)

^b BAR ILAN UNIVERSITY   (Israel)

^c WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

Avatar; BCI; fMRI

Indexed keywords

BRAIN; MAGNETIC RESONANCE IMAGING; VIRTUAL REALITY;

AVATAR; BCI; FMRI; HEMODYNAMIC RESPONSE; HIGH SPATIAL RESOLUTION; REAL-TIME FUNCTIONAL MAGNETIC RESONANCE IMAGING; REGIONS OF INTEREST; TEMPORAL RESOLUTION;

BRAIN COMPUTER INTERFACE;

ADULT; ARTICLE; ASSOCIATION; BOLD SIGNAL; BRAIN COMPUTER INTERFACE; CLINICAL ARTICLE; COMPUTER SYSTEM; CUE BASED TASK; FEEDBACK SYSTEM; FEMALE; FINGER; FREE CHOICE TASK; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HAND; HEMODYNAMICS; HUMAN; IMAGERY; LEG MOVEMENT; MALE; MENTAL TASK; NUCLEAR MAGNETIC RESONANCE SCANNER; PRIORITY JOURNAL; TASK PERFORMANCE; THINKING; TOE; VIRTUAL REALITY; ALGORITHM; BRAIN MAPPING; COMPUTER ASSISTED DIAGNOSIS; COMPUTER INTERFACE; EVOKED MUSCLE RESPONSE; IMAGINATION; MOVEMENT (PHYSIOLOGY); NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; PROCEDURES; ROBOTICS; SPATIAL ORIENTATION;

ADULT; ALGORITHMS; BRAIN MAPPING; BRAIN-COMPUTER INTERFACES; COMPUTER SYSTEMS; EVOKED POTENTIALS, MOTOR; FEMALE; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGINATION; MAGNETIC RESONANCE IMAGING; MALE; MOVEMENT; ROBOTICS; SPATIAL NAVIGATION; THINKING; USER-COMPUTER INTERFACE;

EID: 84901255024     PISSN: 17412560     EISSN: 17412552     Source Type: Journal    
DOI: 10.1088/1741-2560/11/3/035006     Document Type: Article

References (43)

1. Leuthardt E C, Schalk G, Wolpaw J R, Ojemann J G and Moran D W 2004 A brain-computer interface using electrocorticographic signals in humans IEEE Trans. Systems Man Cybern. Part A 1 63
2. Donoghue J P, Nurmikko A, Black M and Hochberg L R 2007 Assistive technology and robotic control using motor cortex ensemble-based neural interface systems in humans with tetraplegia J. Physiol. 579 603-11 (Pubitemid 46404344)
3. Kim S, Simeral J D, Hochberg L R, Donoghue J P, Friehs G M and Black M J 2011 A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair IEEE Trans. Neural Syst. Rehabil. Eng. 19 193-203
4. Hochberg L R et al 2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Nature 485 372-5
5. Faller J, Müller-Putz G, Schmalstieg D and Pfurtscheller G 2010 An application framework for controlling an avatar in a desktop-based virtual environment via a software ssvep brain-computer interface Presence: Teleoperators Virtual Environ. 19 25-34
6. Touyama H and Hirose M 2007 Steady-state VEPs in CAVE for walking around the virtual world Universal Access in Human-Computer Interaction. Ambient Interaction (Lecture Notes in Computer Science vol 4555) pp 715-7
7. Prueckl R and Guger C 2009 A brain-computer interface based on steady state visual evoked potentials for controlling a robot Bio-Inspired Systems: Computational and Ambient Intelligence (Lecture Notes in Computer Science vol 5517) ed J Cabestany, F Sandoval, A Prieto and J Corchado (Berlin: Springer) pp 690-7
8. Ortner R, Guger C, Prueckl R, Graenbacher E and Edlinger G 2010 SSVEP based brain-computer interface for robot control Computers Helping People with Special Needs (Lecture Notes in Computer Science vol 6180) ed K Miesenberger, J Klaus, W Zagler and A Karshmer (Berlin: Springer) pp 85-90
9. Shen H, Zhao L, Bian Y and Xiao L 2009 Research on SSVEP-based controlling system of multi-DoF manipulator Advances in Neural Networks - ISNN 2009 (Lecture Notes in Computer Science vol 5553) ed Y Wen, H Haibo and N Zhang (Berlin: Springer) pp 171-7
10. Long J, Li Y, Wang H, Yu T, Pan J and Li F 2012 A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair IEEE Trans. Neural Syst. Rehabil. Eng. 20 720-9
11. Gergondet P, Druon S, Kheddar A, Hintermuller C, Guger C and Slater M 2011 Using brain-computer interface to steer a humanoid robot ROBIO '11: IEEE Int. Conf. on Robotics and Biomimetics pp 192-7
12. Gergnodet P, Kheddar A, Hintermuller C, Guger C and Slater M 2013 Multitask humanoid control with a brain-computer interface: user experiment with HRP-2 ISER: 13th Int. Symp. on Experimental Robotics (Berlin: Springer) pp 215-29
13. Bayliss J D 2003 Use of the evoked potential P3 component for control in a virtual apartment IEEE Trans. Rehabil. Eng. 11 113-6 (Pubitemid 36935034)
14. Piccione F, Priftis K, Silvoni S, Piron L, Merico A, Tonin P, Vidale D and Furlan R 2007 Integration of a p300 brain computer interface into virtual environment Virtual Rehabilitation, 2007 (Piscataway, NJ: IEEE) p 88
15. Neuper C and Pfurtscheller G 2001 Event-related dynamics of cortical rhythms: Frequency-specific features and functional correlates Int. J. Psychophysiol. 43 41-58 (Pubitemid 33115236)
16. Friedman D, Leeb R, Pfurtscheller G and Slater M 2010 Human-computer interface issues in controlling virtual reality with brain-computer interface Hum. Comput. Interact. 25 67-93
17. Pfurtscheller G, Leeb R, Keinrath C, Friedman D, Neuper C, Guger C and Slater M 2006 Walking from thought Brain Res. 1071 145-52 (Pubitemid 43255485)
18. Leeb R et al 2006 Walking by thinking: The brainwaves are crucial, not the muscles! Presence: Teleoperators Virtual Environments 15 500-14 (Pubitemid 44728702)
19. Royer A S, Doud A J, Rose M L and He B 2010 EEG control of a virtual helicopter in 3-dimensional space using intelligent control strategies IEEE Trans. Neural Syst. Rehabil. Eng. 18 581-9
20. Leeb R, Friedman D, Muller-Putz G, Scherer R, Slater M and Pfurtscheller G 2007 Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: A case study with a tetraplegic Comput. Intell. Neurosci. 2007 79642
21. Bakardjian H, Tanaka T and Cichocki A 2010 Optimization of SSVEP brain responses with application to eight-command brain-computer interface Neurosci. Letters. 469 34-8
22. Coyle S M, Ward T E and Markham C M 2007 Brain-computer interface using a simplified functional near-infrared spectroscopy system J. Neural Eng. 4 219
23. Sitaram R, Caria A and Birbaumer N 2009 Hemodynamic brain-computer interfaces for communication and rehabilitation Neural Netw. 22 1320-8
24. Pfurtscheller G, Allison B Z, Brunner C, Bauernfeind G, Solis-Escalante T, Scherer R, Zander T O, Mueller-Putz G, Neuper C and Birbaumer N 2010 The hybrid BCI Front. Neurosci. 4 3
25. Wienbruch C, Candia V, Svensson J, Kleiser R and Kollias S S 2006 A portable and low-cost fMRI compatible pneumatic system for the investigation of the somatosensensory system in clinical and research environments Neurosci. Lett. 398 183-8 (Pubitemid 43673716)
26. deCharms R C 2008 Applications of real-time fMRI Nature Rev. Neurosci. 9 720-9
27. Berman B D, Horovitz S G, Venkataraman G and Hallett M 2012 Self-modulation of primary motor cortex activity with motor and motor imagery tasks using real-time fMRI-based neurofeedback Neuroimage 59 917-25
28. deCharms R C, Christoff K, Glover G H, Pauly J M, Whiteld S and Gabrieli J D 2004 Learned regulation of spatially localized brain activation using real-time fmri Neuroimage 21 436-43 (Pubitemid 38115033)
29. Rota G, Sitaram R, Veit R, Erb M, Weiskopf N, Dogil G and Birbaumer N 2009 Self regulation of regional cortical activity using real-time fmri: the right inferior frontal gyrus and linguistic processing Hum. Brain Mapp. 30 1605-14
30. Caria A, Sitaram R and Birbaumer N 2012 Real-time fmri: a tool for local brain regulation Neuroscientist 18 487-501
31. Johnson K A, Hartwell K, LeMatty T, Borckardt J, Morgan P S, Govindarajan K, Brady K and George M S Intermittent real-time fmri feedback is superior to continuous presentation for a motor imagery task: a pilot study J. Neuroimaging 22 58-66
32. Yoo S S, Fairneny T, Chen N K, Choo S E, Panych L P, Park H W, Lee S Y and Jolesz F A 2004 Brain-computer interface using fMRI: Spatial navigation by thoughts Neuroreport 15 1591
33. LaConte S M, Peltier S J and Hu X P 2007 Real-time fMRI using brain-state classification Human Brain Mapp. 28 1033-1044 (Pubitemid 47549707)
34. Eklund A, Ohlsson H, Andersson M, Rydell J, Ynnerman A and Knutsson H 2009 Using real-time fMRI to control a dynamical system by brain activity classification Medical Image Computing and Computer-Assisted Intervention-MICCAI 2009 (Berlin: Springer) pp 1000-8
35. Hornyak T 2006 Thinking of child's play Sci. Am. 295 30
36. Lee J H, Ryu J, Jolesz F A, Cho Z H and Yoo S S 2009 Brain-machine interface via real-time fMRI: preliminary study on thought-controlled robotic arm Neurosci. Lett. 450 1-6
37. Minati L, Nigri A, Rosazza C and Bruzzone M G 2012 Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals Med. Eng. Phys. 34 650-8
38. Andersson J P 2012 Visuospatial attention based brain-computer interfacing using real-time fMRI PhD Thesis Utrecht University, The Netherlands
39. Cohen O, Druon S, Lengagne S, Mendelsohn A, Malach R, Kheddar A and Friedman D 2012 fMRI based robotic embodiment: a pilot study IEEE/RAS-EMBS Int. Conf. on Biomedical Robotics and Biomechatronics (BioRob) (Roma, Italy, 24-28 June 2012) pp 314-9
40. O'Hara K, Sellen A and Harper R 2011 Embodiment in brain-computer interaction ACM: Proc. SIGCHI Conf. on Human Factors in Computing Systems pp 353-62
41. Friedman D, Leeb R, Guger C, Steed A, Pfertscheller G and Slater M 2007 Navigating virtual reality by thought: What is it like? Presence: Teleoperators Virtual Environments 16 100-10
42. van der Kouwe A, Gicquel S, Chen G, Schmitt F, Harder M, Salat D, Sorensen A G, Fischl B and Dale A 2003 On-line automatic slice positioning and between-scan correction for brain mr protocols Proc. 11th Annu. Meeting of ISMRM, vol 11 (Toronto, Canada) p 797
43. Mueller E, Thesen S, Heid O and Schad L R 2000 Prospective acquisition correction for head motion with image-based tracking for real-time fMRI Magn. Reson. Med. 44 457-65