Decoding semantics across fMRI sessions with different stimulus modalities: A practical MVPA study

Frontiers in Neuroinformatics

Volumn 6, Issue JULY 2012, 2012, Pages

  Akama, Hiroyuki ^a   Murphy, Brian ^b,c   Na, Li ^a   Shimizu, Yumiko ^d   Poesio, Massimo ^c,e  

^a Tokyo Institute of Technology ^*  (Japan)

^b Carnegie Mellon University   (United States)

^c UNIVERSITY OF TRENTO   (Italy)

^d TOKYO CITY UNIVERSITY   (Japan)

^e UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

Computational neurolinguistics; Embodiment; fMRI; GLM; Individual variability; Machine learning; MVPA

Indexed keywords

ADULT; ARTICLE; BRAIN REGION; CLASSIFICATION; COHORT ANALYSIS; CONTROLLED STUDY; FEMALE; FRONTAL LOBE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; JAPANESE; LEFT INFERIOR PARIETAL LOBE; MALE; MULTIVARIATE ANALYSIS; MULTIVARIATE PATTERN ANALYSIS; NORMAL HUMAN; OCCIPITAL LOBE; PREDICTION; SEMANTICS; SUPRAMARGINAL GYRUS; TASK PERFORMANCE; TEMPORAL LOBE;

EID: 84887270678     PISSN: 16625196     EISSN: None     Source Type: Journal    
DOI: 10.3389/fninf.2012.00024     Document Type: Article

References (64)

1. Aguirre, G. K., Zarahn, E., D'Esposito, M. (1998). The variability of human BOLD haemodynamic responses. NeuroImage 8, 360-369.
2. Aron, A. R., Gluck, M. A., Poldrack, R. A. (2006). Long-term test-retest reliability of functional MRI in a classification learning task, NeuroImage 29, 1000-1006.
3. Barsalou, L. W. (1999). Perceptual symbol systems. Behavioural and Brain Sciences 22, 577-660.
4. Barsalou, L. W. (2003). Situated simulation in the human conceptual system. Language and Cognitive Processes 18, 513-562.
5. Barsalou, L. W., Solomon, K. O., Wu, L.-L. (1999). Perceptual Simulation in Conceptual Tasks. Cultural, typological, and psychological perspectives in cognitive linguistics: The proceedings of the 4th conference of the International Cognitive Linguistics Association. Vol. 3, 209-228.
6. Bergen, B. (2005). Mental simulation in literal and figurative language. The Literal and Non-Literal in Language and Thought, Peter Lang. 255-278.
7. Binder, J. R., Desai, R. H., Graves, W. W., Conant, L. L. (2009). Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral Cortex 19, 2767-2796.
8. Boulenger, V., Hauk, O., Pulvermüller, F. (2009). Grasping Ideas with the Motor System: Semantic Somatotopy in Idiom Comprehension. Cerebral Cortex 19, 1905-1914.
9. Burgess. C. (2000). Invited Reply, Theory and Operational Definitions in Computational Memory Models: A Response to Glenberg and Robertson. Journal of Memory and Language 43, 402-408.
10. Chan, A. M., Halgren E., Marinkovic K., Cash S. S. (2011). Decoding word and category-specific spatiotemporal representations from MEG and EEG. NeuroImage 54(4):3028-39.
11. Chao, L., Haxby, J., & Martin, A. (1999). Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nature Neuroscience.
12. Clithero, J. A., Smith, D. V., Carter, R. M., Huettel, S. A. (2011). Within- and cross-participant classifiers reveal different neural coding of information. NeuroImage 56, 699-708
13. Cox, D. D., Savoy, R. L. (2003). Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. NeuroImage 19, 261-270.
14. Devereux, B., Kelly, C., Korhonen, A. (2010). Using fMRI activation to conceptual stimuli to evaluate methods for extracting conceptual representations from corpora. Proceedings of First Workshop on Computational Neurolinguistics, NAACL HLT 2010, 70-78.
15. Duann, J.-R., Jung, T.-P., Kuo, W.-J., Yeh, T.-C., Makeig, S., Hsieh, J.-C., Sejnowski, T. J. (2002). Single-Trial Variability in Event-Related BOLD Signals. NeuroImage 15, 823-835.
16. Feldman, J., Narayanan, S. (2004). Embodied Meaning in a Neural Theory of Language. Brain and Language 89, 385-392.
17. Friston, K. J., Holmes, A. P., Worsely, K. J., Poline, J.-P., Frith, C. D., Frackowiak, R. S. J. (1995). Statistical parametric maps in functional imaging: A linear approach. Human Brain Mapping 2, 189-210.
18. Glenberg, A. M., Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin and Review 9, 558-565.
19. Handwerker, D. A., Ollinger, J. M., D'Esposito, M. (2004). Variation of BOLD haemodynamic responses across subjects and brain regions and their effects on statistical analyses, NeuroImage 21, 1639-1651.
20. Hanke, M., Halchenko, Y. O., Sederberg, P. B., Olivetti, E., Fründ, I., Rieger, J. W., Herrmann, C. S., Haxby, J. V., Hanson, S. J., Pollmann, S. (2009). PyMVPA: a unifying approach to the analysis of neuroscientific data. Frontiers in Neuroinformatics, 3:3.
21. Haynes, J.-D., Rees, G. (2006). Decoding mental states from brain activity in humans. Nature Reviews Neuroscience 7, 523-534.
22. Haxby, J. V., Gobbini, M. I., Maura L. I. F., Schouten, J. L., Pietrini, P. (2001). Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex. Science 293, 2425-2430.
23. Haxby, J. V., Guntupalli, J. S., Connolly, A. C., Halchenko, Y. O., Conroy, B. R., Gobbini, M. I., Hanke, M. and Ramadge, P. J. (2011). A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex. Neuron, 72, 404-416.
24. Hugdahl, K., Law, I., Kyllingsbaek, S., Bronnick, K., Gade, A., Paulson, O. B. (2000). Effects of attention on dichotic listening: an 15O-PET study. Human Brain Mapping 10 (2), 87-97.
25. Jimura, K., Poldrack, R.A. (2011). Analyses of regional-average activation and multivoxel pattern information tell complementary stories. Neuropsychologia 50(4), 544-52.
26. Kamitani, Y., Tong, F. (2005). Decoding the visual and subjective contents of the human brain. Nature Neuroscience 8 (5), 679-685.
27. Kriegeskorte, N., Bandettini, P. (2007a). Analyzing for information, not activation, to exploit high-resolution fMRI. NeuroImage, Vol. 38, 649-662.
28. Kriegeskorte, N., Bandettini, P. (2007b). Combining the tools: Activation- and information-based fMRI analysis. NeuroImage, Volume 38, 666-668.
29. Kriegeskorte, N., Simmons, W. K., Bellgowan, P., and Baker, C. (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nature Neuroscience, 12: 535-540.
30. LaConte, S., Strother, S., Cherkassky, V., Anderson, J., Hua, X. (2005). Support vector machines for temporal classification of block design fMRI data. NeuroImage 26, 317-329.
31. Lee, J.-H., Marzelli, M., Jolesz, F. A., Yoo, S.-S. (2009). Automated classification of fMRI data employing trial-based imagery tasks. Medical Image Analysis 13(3):392-404.
32. Mahon, B. Z., Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology 102, 59-70.
33. Mahon, B.Z., Anzellotti, S., Schwarzbach, J., Zampini, M., Caramazza, A. (2009). Category-Specific Organization in the Human Brain Does Not Require Visual Experience, Neuron, 63, 397-405.
34. Mahon, B. Z., Caramazza, A. (2010). Judging semantic similarity: An event-related fMRI study with auditory word stimuli, Neuroscience 169, 279-286.
35. McGonigle, D. J., Howseman, A. M., Athwal, B. S., Friston, K. J., Frackowiak, R. S. J., Holmes, A. P. (2000). Variability in fMRI: An Examination of Intersession Differences. NeuroImage 11, 708-734.
36. Mitchell, T. M., Hutchinson, R., Niculescu, R. S., Pereira, F., Wang, X., Just, M., Newman, S. (2004). Learning to decode cognitive states from brain images. Machine Learning 57, 145-175.
37. Mitchell, T., Shinkareva, S., Carlson, A., Chang, K., Malave, V., Mason, R., Just, M. (2008). Predicting human brain activity associated with the meanings of nouns. Science 320, 1191-1195.
38. Mourão-Miranda, J., Bokde, A. L. W., Born, C., Hampel, H., Stetter, M. (2008). Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data. NeuroImage, Vol. 28, Issue 4, 980-995.
39. Mur, M., Bandettini, P. A., Kriegeskorte, N. (2009). Revealing representational content with pattern-information fMRI-an introductory guide. SCAN 4,101-109.
40. Murphy, B., Baroni, M., Poesio, M. (2009). EEG responds to conceptual stimuli and corpus semantics. Proceedings of ACL/EMNLP 2009, 619-627.
41. Murphy, B., Poesio, M., Bovolo, F., Bruzzone, L., Dalponte, M., & Lakany, H. (2011). EEG decoding of semantic category reveals distributed representations for single concepts. Brain and Language, 117(1), 12-22.
42. Murphy, B., Talukdar, P., Mitchell, T. (2012). Selecting Corpus-Semantic Models for Neurolinguistic Decoding. First Joint Conference on Lexical and Computational Semantics (*SEM), 114-123
43. Norman, K. A., Polyn, S. M., Detre, G. J., Haxby, J. V. (2006). Beyond mind-reading: multi-variate pattern analysis of fMRI data. TRENDS in Cognitive Sciences, Vol.10, No.9, 424-430.
44. O'Toole, A. J., Jiang, F., Abdi, H., Penard, N., Dunlop, J. P., Parent, M. A. (2007). Theoretical, statistical, and practical perspectives on pattern-based classification approaches to the analysis of functional neuroimaging data. Journal of Cognitive Neuroscience 19, 1735-1752.
45. Pereira, F., Mitchell T., Botvinick, M. (2009). Machine learning classifiers and fMRI: A tutorial overview, NeuroImage 45, 199-S209.
46. Pereira, F., Detre, G., Botvinick, M. (2011). Generating text from functional brain images. Frontiers in Human Neuroscience 5:72.
47. Pereira, F., Botvinick, M., Detre, G. (2010). Learning semantic features for fMRI data from definitional text. Proceedings of the Proceedings of First Workshop on Computational Neurolinguistics, NAACL HLT 2010, 1-9.
48. Pulvermüller, F. (2001). Brain reflections of words and their meaning. TRENDS in Cognitive Sciences, Vol.5, No.12, 517-524.
49. Pulvermüller, F. (2005). Brain mechanisms linking language and action, Nature Reviews Neuroscience, 6, 576-58.
50. Raizada, R. D., Tsao, F. M., Liu, H. M., Kuhl, P. K. (2010). Quantifying the adequacy of neural representations for a cross-language phonetic discrimination task: prediction of individual differences. Cerebral Cortex 20 (1), 1-12.
51. Rohrer, T. (2007). The Body in Space. Body, Language and Mind, v.1, 339-378.
52. Solomon, K. O., Barsalou, L. W. (2004). Perceptual simulation in property verification. Memory & Cognition 32 (2), 244-259.
53. Spiridon, M., Kanwisher, N. (2002). How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron 35, 1157-1165.
54. Shimizu, Y., Akama, H. (2008). Simulation Semantics Based on the Subdivision of the Figures of Speech. The Journal of Environmental and Information Studies Musashi Institute of Technology 2008, 73-85.
55. Shinkareva, S. V., Mason, R. A., Malave, V. L., Wang, W., Mitchell, T. M., Just, M. A. (2008). Using fMRI Brain Activation to Identify Cognitive States Associated with Perception of Tools and Dwellings. PLoS One, 1-9.
56. Smith, S. M., Beckmann, C. F., Ramnani, N., Woolrich, M. W., Bannister, P. R., Jenkinson, M., Matthews, P. M., McGonigle, D. J. (2005). Variability in fMRI: A Re-Examination of Intersession Differences. Human Brain Mapping 24 (3):248-57.
57. Vingerhoets, G., de Lange, F. P., Vandamaele, P., Karel Deblaere and Erik Achten (2002). Motor Imagery in Mental Rotation: An fMRI study. Neuroimage 17, 1623-1633.
58. Wallentin, M., Lund, T. E., Stergaard, S., Stergaard, L., Roepstor, A. (2005). Motion verb sentences activate left posterior middle temporal cortex despite static context. Cognitive Neuroscience and Neuropsychology, Vol. 16, No. 6, 649-652.
59. Wang, X., Hutchinson, R., Mitchell, T. M. (2004). Training fMRI Classifiers to Discriminate Cognitive States across Multiple Subjects. Advances in Neural Information Processing Systems 16, 709-716.
60. Weekes, B.S., Zubicaray, G. de., McMahon, K., Eastburn, M., Bryant, M., Wang, D. (2005). Orthographic effects on picture naming in Chinese: A 4T fMRI study. Brain and Language, 95, 14-15.
61. Weil, R. S., Reesa, G. (2010). Decoding the neural correlates of consciousness. Current Opinion in Neurology 23, 649-655.
62. Willems, R. M., Casasanto, D. (2010). Flexibility in embodied language understanding. Front. Psychology 2:116.
63. Willms, J. L., Shapiro, K. A., Peelen, M. V., Pajtas, P. E., Costa, A., Moo, L. R., Caramazza, A. (2011). Language-invariant verb processing regions in Spanish-English bilinguals, NeuroImage 57, 251-261.
64. Wu, L.-L., Barsalou, L. W. (2009). Perceptual simulation in conceptual combination: Evidence from property generation. Acta Psychologica 132, 173-189.