Representational similarity encoding for fMRI: Pattern-based synthesis to predict brain activity using stimulus-model-similarities

NeuroImage

Volumn 128, Issue , 2016, Pages 44-53

  Anderson, Andrew James ^a   Zinszer, Benjamin D ^a   Raizada, Rajeev D S ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

Author keywords

Decoding; Encoding; fMRI; Representational similarity analysis; Semantic memory; Semantic model

Indexed keywords

ADULT; ARTICLE; BRAIN FUNCTION; CONTROLLED STUDY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NERVOUS SYSTEM PARAMETERS; NORMAL HUMAN; NUCLEAR MAGNETIC RESONANCE SCANNER; PREDICTION; PRIORITY JOURNAL; REPRESENTATIONAL SIMILARITY ANALYSIS; SEMANTIC MEMORY; STIMULUS RESPONSE; VISUAL STIMULATION; BIOLOGICAL MODEL; BRAIN; BRAIN MAPPING; IMAGE PROCESSING; MEMORY; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; PROCEDURES; THEORETICAL MODEL;

BRAIN; BRAIN MAPPING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MEMORY; MODELS, NEUROLOGICAL; MODELS, THEORETICAL;

EID: 84954289879     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2015.12.035     Document Type: Article

References (43)

1. Akama H., Miyake M., Jung J., Murphy B. Using graph components derived from an associative concept dictionary to predict fMRI neural activation patterns that represent the meaning of nouns. PLoS ONE 2015, 10.1371/journal.pone.0125725.
2. Anderson A.J., Bruni E., Lopopolo A., Poesio M., Baroni M. Reading visually embodied meaning from the brain: visually grounded computational models decode visual-object mental imagery induced by written text. NeuroImage 2015, 120:309-322.
3. Anderson A.J., Murphy B., Poesio M. Discriminating taxonomic categories and domains in mental simulations of concepts of varying concreteness. J. Cogn. Neurosci. 2014, 26(3):658-681.
4. Anderson A.J., Bruni E., Bordignon U., Poesio M., Baroni M. Of words, eyes and brains: correlating image-based distributional semantic models with neural representations of concepts. Proceedings of EMNLP 2013 1960-1970 Association for Computational Linguistics, Seattle, WA 2013.
5. Austin J., Stonham T.J. Distributed associative memory for use in scene analysis. Image Vis. Comput. 1987, 5(4):251-260.
6. Baroni M., Dinu G., Kruszewski G. Don't count, predict! A systematic comparison of context-counting vs.context-predicting semantic vectors. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, Maryland, USA 2014.
7. Bruffaerts R., Dupont P., Peeters R., De Deyne S., Storms G., Vandenberghe R. Similarity of fMRI activity patterns in left perirhinal cortex reflects similarity between words. J. Neurosci. 2013, 33(47):18597-18607.
8. Carlson T.A., Simmons R.A., Kriegeskorte N., Slevc L.,.R. The emergence of semantic meaning in the ventral temporal pathway. J. Cogn. Neurosci. 2014, 26(1):120-131.
9. Connolly A.C., Guntupalli J.S., Gors J., Hanke M., Halchenko Y.O., et al. The representation of biological classes in the human brain. J. Neurosci. 2012, 32:2608-2618.
10. deCharms R.C. Applications of real-time fMRI. Nat. Rev. Neurosci. 2008, 9(9):720-729.
11. Devereux B., Kelly C., Korhonen A. Using fMRI activation to conceptual stimuli to evaluate methods for extracting conceptual representations from corpora. Proceedings of the NAACL HLT 2010 First Workshop on Computational Neurolinguistics 2010, 70-78. Association for Computational Linguistics, Los Angeles, USA. B. Murphy, K.K. Chang, A. Korhonen (Eds.).
12. Edelman S. Representation is representation of similarities. Behav. Brain Sci. 1998, 21:449-467.
13. Edelman S., Grill-Spector K., Kushnir T., Malach R. Towards direct visualization of the internal shape space by fMRI. Psychobiology 1998, 26:309-321.
14. Fernandino L., Humphries C.J., Seidenberg M.S., Gross W.L., Conant L.L., Binder J.R. Predicting brain activation patterns associated with individual lexical concepts based on five sensory-motor attributes. Neuropsycholigia 2015, 10.1016/j.neuropsychologia.2015.04.009.
15. Haxby J.V., Connolly A.C., Guntupalli J.S. Decoding neural representational spaces using multivariate pattern analysis. Annu. Rev. Neurosci. 2014, 37:435-436. 10.1146/annurev-neuro-062012-170325.
16. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 2001, 293:2425-2430.
17. Jackson R.L., Hoffman P., Pobric G., Lambon Ralph M.A. The nature and neural correlates of semantic association versus conceptual similarity. Cereb. Cortex 2015, 25(11):4319-4333.
18. Jelodar A.B., Alizadeh M., Khadivi S. Wordnet based features for predicting brain activity associated with meanings of nouns. Proceedings of the NAACL HLT 2010 First Workshop on Computational Neurolinguistics, W10 2010, 0603.
19. Just M.A., Cherkassky V.L., Buchweitz A., Keller T.A., Mitchell T.M. Identifying autism from neural representations of social interactions: neurocognitive markers of autism. PLoS ONE 2014, 9.
20. Kalénine S., Peyrin C., Pichat C., Segebarth C., Bonthoux F., Baciu M. The sensory-motor specificity of taxonomic and thematic conceptual relations: a behavioral and fMRI study. NeuroImage 2009, 44:1152-1162.
21. Kay K.N., Naselaris T., Prenger R.J., Gallant J.L. Identifying natural images from human brain activity. Nature 2008, 452:352-355.
22. Kohonen T. Correlation matrix memories. IEEE Trans. Comput. 1972, 21:353-359.
23. Kohonen T. Self-Organizing Maps 1997, Springer, New York.
24. Kriegeskorte N., Kievit R.A. Representational geometry: integrating cognition, computation, and the brain. Trends Cogn. Sci. 2013, 17:401-412. 10.1016/j.tics.2013.06.007.
25. Kriegeskorte N., Mur M., Bandettini P. Representational similarity analysis-connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2008, 2(4):1-28.
26. Kriegeskorte N., Mur M., Ruff D.A., Kiani R., Bodurka J., Esteky H., Tanaka K., Bandettini P.A. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 2008, 60:1126-1141.
27. Kriegeskorte N., Goebel R., Bandettini P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. U. S. A. 2006, 103:3863-3868.
28. Levy J.P., Bullinaria J.A. Using enriched semantic representations in predictions of human brain activity. Connectionist Models of Neurocognition and Emergent Behavior: From Theory to Applications 2012, 292-308. World Scientific, Singapore. E.J. Davelaar (Ed.).
29. Lin E.L., Murphy G.L. Thematic relations in adults' concepts. J. Exp. Psychol. Gen. 2001, 130:3-28.
30. Matthews P.M., Honey G.D., Bullmore E.T. Nat. Rev. Neurosci. 2006, 7:732-744.
31. Mitchell T.M., Shinkareva S.V., Carlson A., Chang K.-M., Malave V.L., Mason R.A., Just M.A. Predicting human brain activity associated with the meaning of nouns. Science 2008, 320:1191-1195.
32. Murphy B., Talukdar P., Mitchell T. Selecting corpus-semantic models for neurolinguistic decoding. Proceedings of First Joint Conference on Lexical and Computational Semantics (SEM) 2012, 114.
33. Naselaris T., Prenger R.J., Kay K.,.N., Oliver M., Gallant J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63:902-915.
34. Naselaris T., Kay K.N., Nishimoto S., Gallant J.L. Encoding and decoding in fMRI. NeuroImage 2011, 56:400-410.
35. Nili H., Wingfield C., Walther A., Su L., Marslen-Wilson W., Kriegeskorte N. A toolbox for representational similarity analysis. PLoS Comput. Biol. 2014, 10(4). 10.1371/journal.pcbi.1003553.
36. Palatucci M., Pomerleau D., Hinton G., Mitchell T. Zero-shot learning with semantic output codes. Neural Inf. Process. Syst. 2009, 22:1410-1418.
37. Peelen M.V., Fei-fei L., Kastner S. Neural mechanisms of rapid natural scene categorization in human visual cortex. Nature 2009, 460:94-97.
38. Pereira F., Botvinick M., Detre G. Using Wikipedia to learn semantic feature representations of concrete concepts in neuroimaging experiments. Artif. Intell. 2013, 194:240-252.
39. Raizada R.D.S., Connolly A.C. What makes different people's representations alike: neural similarity-space solves the problem of across-subject fMRI decoding. J. Cogn. Neurosci. 2012, 24(4):868-877.
40. Schwartz M.F., Kimberg D.Y., Walker G.M., Brecher A., Faseyitan O.K., Dell G.S., Mirman D., Coslett H.B. Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain. Proc. Natl. Acad. Sci. U. S. A. 2011, 108:8520-8524.
41. Sulzer J., Haller S., Scharnowski F., Weiskopf N., Birbaumer N., Blefari M.L., Bruehl A.B., Cohen L.G., deCharms R.C., Gassert R., Goebel R., Herwig U., LaConte S., Linden D., Luft A., Seifritz E., Sitaram R. Real-time fMRI neurofeedback: progress and challenges. NeuroImage 2013, 76:386-399.
42. Willshaw D.J., Buneman O.P., Longuet-Higgins H.C. Non-holographic associative memory. Nature 1969, 222:960-962. 10.1038/222960a0.
43. Zinszer B.D., Anderson A.J., Kang O., Wheatley T., Raizada R.D.S. You say potato, I say tudou: how speakers of different languages can share the same concept. Proceedings of the 37th Annual Conference of the Cognitive Science Society 2015.