Commonality of neural representations of words and pictures

NeuroImage

Volumn 54, Issue 3, 2011, Pages 2418-2425

  Shinkareva, Svetlana V ^a   Malave, Vicente L ^b   Mason, Robert A ^c   Mitchell, Tom M ^d   Just, Marcel Adam ^c  

^a UNIVERSITY OF SOUTH CAROLINA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

^d Carnegie Mellon University   (United States)

Author keywords

Brain state; Classification; FMRI; Multi voxel pattern analysis; Pictures; Words

Indexed keywords

ADULT; ARTICLE; BRAIN REGION; CLASSIFICATION ALGORITHM; COGNITION; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; LOGISTIC REGRESSION ANALYSIS; MALE; NERVE CONDUCTION; NEUROIMAGING; NORMAL HUMAN; PRIORITY JOURNAL; STIMULUS RESPONSE; TASK PERFORMANCE; WORD RECOGNITION;

ADULT; ALGORITHMS; BRAIN; BRAIN MAPPING; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; PHOTIC STIMULATION; READING; THINKING; VISUAL PERCEPTION;

EID: 78650215949     PISSN: 10538119     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2010.10.042     Document Type: Article

References (49)

1. Aziz-Zadeh L., Damasio A. Embodied semantics for actions: findings from functional brain imaging. J. Physiol. Paris 2008, 102:35-39.
2. Barsalou L.W. Perceptual symbols systems. Behav. Brain Sci. 1999, 22:577-660.
3. Boronat C.B., Buxbaum L.J., Coslett H.B., Tang K., Saffran E.M., Kimberg D.Y., Detre G.J. Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging. Cogn. Brain Res. 2005, 23:361-373.
4. Bright P., Moss H., Tyler L.K. Unitary vs multiple semantics: PET studies of word and picture processing. Brain Lang. 2004, 89(3):417-432.
5. Carlson T.A., Schrater P., He S. Patterns of activity in the categorical representations of objects. J. Cogn. Neurosci. 2003, 15(5):704-717.
6. Chao L.L., Haxby J.V., Martin A. Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nat. Neurosci. 1999, 2(10):913-919.
7. Chee M.W., Weekes B., Lee K.M., Soon C.S., Schreiber A., Hoon J.J., Chee M. Overlap and dissociation of semantic processing of Chinese characters, English words, and pictures: evidence from fMRI. Neuroimage 2000, 12(4):392-403.
8. Cox D.D., Savoy R.L. Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 2003, 19:261-270.
9. Cusack R., Veldsman M., Naci L., Mitchell D. Using Dynamically Adaptive Imaging with fMRI to rapidly characterize neural representations. In Proceedings of the ISMRM, 18th Scientific Meeting (Stockholm) 2010, p. 2346.
10. Davatzikos C., Ruparel K., Fan Y., Shen D.G., Acharyya M., Loughead J.W., Gur R.C., Langleben D.D. Classifying spatial patterns of brain activity with machine learning methods: application to lie detection. Neuroimage 2005, 28(3):663-668.
11. Devlin J.T., Rushworth M.F.S., Matthews P.M. Category-related activation for written words in the posterior fusiform is task specific. Neuropsychologia 2005, 43:69-74.
12. Eger E., Michel V., Thirion B., Amadon A., Dehaene S., Kleinschmidt A. Deciphering cortical number coding from human brain activity patterns. Curr. Biol. 2009, 19:1608-1615.
13. Etzel J.A., Gazzola V., Keysers C. Testing simulation theory with cross-modal multivariate classification of fMRI data. PLoS ONE 2008, 3(11):e3690.
14. Formisano E., Martino F.D., Bonte M., Goebel R. "Who" is saying "what"? Brain-based decoding of human voice and speech. Science 2008, 322:970-973.
15. Gallagher H.L., Happe F.G., Brunswick N., Fletcher P.C., Frith U., Frith C.D. Reading the mind in cartoons and stories: an fMRI study of "theory of mind" in verbal and nonverbal tasks. Neuropsychologia 2000, 38:11-21.
16. Gates L., Yoon M.G. Distinct and shared cortical regions of the human brain activated by pictorial depictions versus verbal descriptions: an fMRI study. Neuroimage 2005, 24(2):473-486.
17. Glenberg A.M. What memory is for. Behav. Brain Sci. 1997, 20:1-55.
18. Grill-Spector K., Kushnir T., Edelman S., Avidan G., Itzchak Y. Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 1999, 24:187-203.
19. Hanson S.J., Halchenko Y.O. Brain reading using full brain support vector machines for object recognition: there is no "face" identification area. Neural Comput. 2007, 20(2):486-503.
20. Hanson S.J., Matsuka T., Haxby J.V. Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a "face" area?. Neuroimage 2004, 23:156-166.
21. Hasson U., Nir Y., Levy I., Fuhrmann G., Malach R. Intersubject synchronization of cortical activity during natural vision. Science 2004, 303:1634-1640.
22. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 2001, 293(5539):2425-2430.
23. Haynes J.D., Rees G. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 2006, 7(7):523-534.
24. Hodges J.R., Patterson K. Semantic memory disorders. Trends Cogn. Sci. 1997, 1:68-72.
25. Hoenig K., Sim E., Bochev V., Herrnberger B., Kiefer M. Conceptual flexibility in the human brain: dynamic recruitment of semantic maps from visual, motor, and motion-related areas. J. Cogn. Neurosci. 2008, 20(10):1799-1814.
26. James T.W., Humphrey G.K., Gati J.S., Menon R.S., Goodale M.A. Differential effects of viewpoint on object-driven activation in dorsal and ventral streams. Neuron 2002, 35:793-801.
27. Just M.A., Cherkassky V.L., Aryal S., Mitchell T.M. A neurosemantic theory of concrete noun representation based on the underlying brain codes. PLoS ONE 2010, 5(1):e8622.
28. Knops A., Thirion B., Hubbard E.M., Michel V., Dehaene S. Recruitment of an area involved in eye movements during mental arithmetic. Science 2009, 324:1583-1585.
29. Mechelli A., Price C., Friston K.J., Ishai A. Where bottom-up meets top-down: neuronal interactions during perception and imagery. Cereb. Cortex 2004, 14:1256-1265.
30. Mitchell T.M. Machine learning 1997, McGraw-Hill, Boston. C.L. Liu (Ed.).
31. Mitchell T.M., Shinkareva S.V., Carlson A., Chang K.M., Malave V.L., Mason R.A., Just M.A. Predicting human brain activity associated with the meanings of nouns. Science 2008, 320:1191-1195.
32. Mourao-Miranda J., Bokde A.L., Born C., Hampel H., Stetter M. Classifying brain states and determining the discriminating activation patterns: support vector machine on functional MRI data. Neuroimage 2005, 28(4):980-995.
33. Norman K.A., Polyn S.M., Detre G.J., Haxby J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 2006, 10(9):424-430.
34. O'Toole A., Jiang F., Abdi H., Haxby J.V. Partially distributed representations of objects and faces in ventral temporal cortex. J. Cogn. Neurosci. 2005, 17(4):580-590.
35. O'Toole A.J., Jiang F., Abdi H., Penard N., Dunlop J.P., Parent M.A. Theoretical, statistical, and practical perspectives on pattern-based classification approaches to the analysis of functional neuroimaging data. J. Cogn. Neurosci. 2007, 19(11):1735-1752.
36. Perani D., Schnur T., Tettamanti M., Gorno-Tempini M., Cappa S.F., Fazio F. Word and picture matching: a PET study of semantic category effects. Neuropsychologia 1999, 37(3):293-306.
37. Pereira F., Mitchell T., Botvinick M. Machine learning classifiers and fMRI: a tutorial overview. Neuroimage 2009, 45:S199-S209.
38. Pietrini P., Furey M.L., Ricciardi E., Gobbini M.I., Wu W.H., Cohen L., Guazzelli M., Haxby J.V. Beyond sensory images: object-based representation in the human ventral pathway. Proc. Natl Acad. Sci. USA 2004, 101(15):5658-5663.
39. Poldrack R.A., Halchenko Y.O., Hanson S.J. Decoding the large-scale structure of brain function by classifying mental states across individuals. Psychol. Sci. 2009, 20(11):1364-1372.
40. Polyn S.M., Natu V.S., Cohen J.D., Norman K.A. Category-specific cortical activity precedes retrieval during memory search. Science 2005, 310:1963-1966.
41. Pulvermuller F. Brain mechanisms linking language and action. Nat. Rev. Neurosci. 2005, 6:1-7.
42. Pulvermuller F., Kherif F., Hauk O., Mohr B., Nimmo-Smith I. Distributed cell assemblies for general lexical and category-specific semantic processing as revealed by fMRI cluster analysis. Hum. Brain Mapp. 2009, 12:3837-3850.
43. Rademacher J., Galaburda A.M., Kennedy D.N., Filipek P.A., Caviness V.S. Human cerebral cortex: localization, parcellation and morphometry with magnetic resonance imaging. J. Cogn. Neurosci. 1992, 4:352-374.
44. Raizada R.D.S., Tsao F., Liu H., Kuhl P. Quantifying the adequacy of neural representations for cross-language phonetic discrimination task: prediction of individual differences. Cereb. Cortex 2010, 20:1-12.
45. Shinkareva S.V., Mason R.A., Malave V.L., Wang W., Mitchell T.M., Just M.A. Using FMRI brain activation to identify cognitive states associated with perception of tools and dwellings. PLoS ONE 2008, 3(1). e1394-e1394.
46. Tzourio-Mazoyer N., Landeau B., Papathanassiou D., Crivello F., Etard O., Delcroix N., Mazoyer B., Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 2002, 15:273-289.
47. Vandenberghe R., Price C., Wise R., Josephs O., Frackowiak R.S. Functional anatomy of a common semantic system for words and pictures. Nature 1996, 383(6597):254-256.
48. Vuilleumier P., Henson R.N., Driver J., Dolan R.J. Multiple levels of visual object constancy revealed by event-related fMRI of repetition priming. Nat. Neurosci. 2002, 5:491-499.
49. Willems R.M., Ozyurek A., Hagoort P. Seeing and hearing meaning: ERP and fMRI evidence of word versus picture integration into a sentence context. J. Cogn. Neurosci. 2008, 20(7):1235-1249.