Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals

Medical Engineering and Physics

Volumn 34, Issue 5, 2012, Pages 650-658

  Minati, Ludovico ^a   Nigri, Anna ^a   Rosazza, Cristina ^a   Bruzzone, Maria Grazia ^a  

^a DEPARTMENT OF NEUROSURGERY   (Italy)

Author keywords

Brain machine interface (BMI); Computer vision; Functional magnetic resonance imaging (fMRI); Robotics

Indexed keywords

ABSTRACT LEVELS; ACTION RECOGNITION; BRAIN AREAS; BRAIN MACHINE INTERFACE; BRAIN-MACHINE INTERFACING; EXECUTION TIME; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL MRI; FUNCTIONAL MRI (FMRI); INPUT NODE; MACHINE VISION SYSTEMS; MATLAB LANGUAGES; PRE-SELECTED; PROOF OF CONCEPT; REALTIME PROCESSING; ROBOT ARMS; SOURCE CODES; STATE MACHINE; VISION-GUIDED ROBOTS;

BRAIN; COUPLINGS; FUNCTIONAL NEUROIMAGING; MAGNETIC RESONANCE IMAGING; ROBOTIC ARMS; ROBOTICS; SIGNAL PROCESSING;

COMPUTER VISION;

ACCURACY; ADULT; ARTICLE; BOLD SIGNAL; BRAIN COMPUTER INTERFACE; CLASSIFIER; COMPUTER LANGUAGE; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GAME; HUMAN; HUMAN EXPERIMENT; IMAGERY; MALE; MOVEMENT (PHYSIOLOGY); NORMAL HUMAN; PRIORITY JOURNAL; RECOGNITION; ROBOTICS; TASK PERFORMANCE; VISUAL SYSTEM;

ADULT; BRAIN; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; OXYGEN; ROBOTICS; SOFTWARE; THINKING; VISION, OCULAR;

EID: 84861333081     PISSN: 13504533     EISSN: 18734030     Source Type: Journal    
DOI: 10.1016/j.medengphy.2012.02.004     Document Type: Article

References (45)

1. Vidal J.J. Toward direct brain-computer communication. Ann Rev Biophys Bioeng 1973, 2:157-180.
2. Nicolelis M.A. Brain-machine interfaces to restore motor function and probe neural circuits. Nat Rev Neurosci 2003, 4(5):417-422.
3. Kotchetkov I.S., Hwang B.Y., Appelboom G., Kellner C.P., Connolly E.S. Brain-computer interfaces: military, neurosurgical, and ethical perspective. Neurosurg Focus 2010, 28(5):E25.
4. Konrad P., Shanks T. Implantable brain computer interface: challenges to neurotechnology translation. Neurobiol Dis 2010, 38(3):369-375.
5. Moore M.M. Real-world applications for brain-computer interface technology. IEEE Trans Neural Syst Rehabil Eng 2003, 11:162-165.
6. Pérez-Marcos D., Buitrago J.A., Velásquez F.D. Writing through a robot: a proof of concept for a brain-machine interface. Med Eng Phys 2011, 33(10):1314-1317.
7. Vidaurre C., Blankertz B. Towards a cure for BCI illiteracy. Brain Topogr 2010, 23(2):194-198.
8. Logothetis N.K., Pfeuffer J. On the nature of the BOLD fMRI contrast mechanism. Magn Reson Imaging 2004, 22(10):1517-1531.
9. Naselaris T., Prenger R.J., Kay K.N., Oliver M., Gallant J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63(6):902-915.
10. Sitaram R., Caria A., Birbaumer N. Hemodynamic brain-computer interfaces for communication and rehabilitation. Neural Networks 2009, 22(9):1320-1328.
11. Weiskopf N., Sitaram R., Josephs O., Veit R., Scharnowski F., Goebel R., et al. Real-time functional magnetic resonance imaging: methods and applications. Magn Reson Imaging 2007, 25(6):989-1003.
12. Yoo S.S., Fairneny T., Chen N.K., Choo S.E., Panych L.P., Park H., et al. Brain-computer interface using fMRI: spatial navigation by thoughts. Neuroreport 2004, 15(10):1591-1595.
13. LaConte S.M., Peltier S.J., Hu X.P. Real-time fMRI using brain-state classification. Hum Brain Mapp 2007, 28(10):1033-1044.
14. Ferrari M., Mottola L., Quaresima V. Principles, techniques, and limitations of near infrared spectroscopy. Can J Appl Physiol 2004, 29(4):463-487.
15. Coyle S.M., Ward T.E., Markham C.M. Brain-computer interface using a simplified functional near-infrared spectroscopy system. J Neural Eng 2007, 4:219.
16. Goodrich M.A., Schultz A.C. Human-robot interaction: a survey. Found Trends Hum Comput Interact (USA) 2007, 1:203-275.
17. Kansaku K., Hata N., Takano K. My thoughts through a robot's eyes: an augmented reality-brain-machine interface. Neurosci Res 2010, 66(2):219-222.
18. Lee J.H., Ryu J., Jolesz F.A., Cho Z.H., Yoo S.S. Brain-machine interface via real-time fMRI: preliminary study on thought-controlled robot arm. Neurosci Lett 2009, 450(1):1-6.
19. Tsubone T., Muroga T., Wada Y. Application to robot control using brain function measurement by near-infrared spectroscopy. Conf Proc IEEE Eng Med Biol Soc 2007, 2007:5342-5345.
20. Eklund A., Ohlsson H., Andersson M., Rydell J., Ynnerman A., Knutsson H. Using real-time fMRI to control a dynamical system by brain activity classification. Med Image Comput Comput Assist Interv 2009, 12(1):1000-1008.
21. Minati L., Zorat A. A tree architecture with hierarchical data processing on a sensor-rich hexapod robot. Adv Robotics 2002, 16(7):595-608.
22. Hashimoto K. A review on vision-based control of robot manipulators. Adv Robotics 2003, 17(10):969-991.
23. Weissenbacher A., Kasess C., Gerstl F., Lanzenberger R., Moser E., Windischberger C. Correlations and anticorrelations in resting-state functional connectivity MRI: a quantitative comparison of preprocessing strategies. Neuroimage 2009, 47(4):1408-1416.
24. Worsley K.J., Friston K.J. Analysis of fMRI time-series - revisited again. Neuroimage 1995, 2:173-181.
25. Picton P.D. Neural networks 2000, Palgrave Macmillan, Basingstoke, UK.
26. Møller M.F. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 1993, 6:525-533.
27. Horn B.K.P. Robot vision 1996, MIT Press, Cambridge, MA, USA.
28. Spong M.W., Lewis F.L., Abdallah C.T. Robot control: dynamics, motion planning and analysis 1994, IEEE Press, NY, USA.
29. Gürkök H., Poel M., Zwiers J. Classifying motor imagery in presence of speech. Intl Joint Conf Neural Networks (IJCNN), vol. 1 2010, 8.
30. Mellet E., Tzourio-Mazoyer N., Bricogne S., Mazoyer B., Kosslyn S.M., Denis M. Functional anatomy of high-resolution visual mental imagery. J Cogn Neurosci 2000, 12(1):98-109.
31. Bunzeck N., Wuestenberg T., Lutz K., Heinze H.J., Jancke L. Scanning silence: mental imagery of complex sounds. Neuroimage 2005, 26(4):1119-1127.
32. Arsalidou M., Taylor M.J. Is 2+2=4? Meta-analyses of brain areas needed for numbers and calculations. Neuroimage 2011, 54(3):2382-2393.
33. Henson R. A mini-review of fMRI studies of human medial temporal lobe activity associated with recognition memory. Q J Exp Psychol B 2005, 58(3-4):340-360.
34. Norman K.A., Polyn S.M., Detre G.J., Haxby J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn Sci 2006, 10(9):424-430.
35. Seghier M.L., Lazeyras F., Pegna A.J., Annoni J.M., Zimine I., Mayer E., et al. Variability of fMRI activation during a phonological and semantic language task in healthy subjects. Hum Brain Mapp 2004, 23(3):140-155.
36. Xu L., Johnson T.D., Nichols T.E., Nee D.E. Modeling inter-subject variability in FMRI activation location: a Bayesian hierarchical spatial model. Biometrics 2009, 65(4):1041-1051.
37. Penny W., Kiebel S., Friston K. Variational Bayesian inference for fMRI time series. Neuroimage 2003, 19(3):727-741.
38. Donchin E., Arbel Y. P300 based brain computer interfaces: a progress report. Foundations of augmented cognition. Proc Neuroergonom Oper Neurosci 2009, 5638:724-731.
39. Guger C., Daban S., Sellers E., Holzner C., Krausz G., Carabalona R., et al. How many people are able to control a P300-based brain-computer interface (BCI)?. Neurosci Lett 2009, 462:94-98.
40. Opelt A., Pinz A., Fussenegger M., Auer P. Generic object recognition with boosting. IEEE Trans Pattern Anal Machine Intell 2006, 28(3):416-431.
41. Hae Jong S., Milanfar P. Training-free, generic object detection using locally adaptive regression kernels. IEEE Trans Pattern Anal Machine Intell 2010, 32(9):1688-1704.
42. Soto Martell J.W., Gini G. Robotic hands: design review and proposal of new design process. Int J Appl Math Comput Sci 2007, 4(2):587-592.
43. Berman B.D., Horovitz S.G., Venkataraman G., Hallett M. Self-modulation of primary motor cortex activity with motor and motor imagery tasks using real-time fMRI-based neurofeedback. Neuroimage 2012, 59(2):917-925.
44. Sitaram R., Veit R., Stevens B., Caria A., Gerloff C., Birbaumer N., et al. Acquired control of ventral premotor cortex activity by feedback training: an exploratory real-time fMRI and TMS study. Neurorehabil Neural Repair 2012, 26(3):256-265.
45. Hermes D., Vansteensel M.J., Albers A.M., Bleichner M.G., Benedictus M.R., Mendez Orellana C., et al. Functional MRI-based identification of brain areas involved in motor imagery for implantable brain-computer interfaces. J Neural Eng 2011, 8(2):025007.