Sight and sound converge to form modality-invariant representations in temporoparietal cortex

Journal of Neuroscience

Volumn 32, Issue 47, 2012, Pages 16629-16636

  Man, Kingson ^a   Kaplan, Jonas T ^a   Damasio, Antonio ^a   Meyer, Kaspar ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION POTENTIAL; ARTICLE; AUDITORY STIMULATION; EVENT RELATED POTENTIAL; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NERVE CELL NETWORK; NORMAL HUMAN; PARIETAL CORTEX; PRIORITY JOURNAL; RECOGNITION; SUPERIOR TEMPORAL SULCUS; TEMPORAL CORTEX; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ANALYSIS OF VARIANCE; AUDITORY PERCEPTION; BRAIN MAPPING; CEREBRAL CORTEX; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; IMAGINATION; MAGNETIC RESONANCE IMAGING; MALE; PARIETAL LOBE; PATTERN RECOGNITION, AUTOMATED; PHOTIC STIMULATION; TEMPORAL LOBE; VISUAL PERCEPTION;

EID: 84870044761     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.2342-12.2012     Document Type: Article

References (39)

1. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559-571. CrossRef Medline
2. Barnes CL, Pandya DN (1992) Efferent cortical connections of multimodal cortex of the superior temporal sulcus in the rhesus monkey. J Comp Neurol 318:222-244. Medline
3. Beauchamp MS (2005) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93-113. CrossRef Medline
4. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190-1192. CrossRef Medline
5. Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57:849-872. Medline
6. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433-436. CrossRef Medline
7. Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369-384. Medline
8. Bunzeck N, Wuestenberg T, Lutz K, Heinze HJ, Jancke L (2005) Scanning silence: mental imagery of complex sounds. Neuroimage 26:1119-1127. CrossRef Medline
9. Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110-1123. CrossRef Medline
10. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:27:1-27:27. CrossRef
11. Dahl CD, Logothetis NK, Kayser C (2009) Spatial organization of multisensory responses in temporal association cortex. J Neurosci 29:11924-11932. CrossRef Medline
12. Damasio AR (1989) Time-locked multiregional retroactivation: a systemslevel proposal for the neural substrates of recall and recognition. Cognition 33:25-62. CrossRef Medline
13. Doehrmann O, Naumer MJ (2008) Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242:136-150. CrossRef Medline
14. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 57:11-23. CrossRef Medline
15. Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1-47. CrossRef Medline
16. Hanke M, Halchenko YO, Sederberg PB, Hanson SJ, Haxby JV, Pollmann S (2009) PyMVPA: A python toolbox for multivariate pattern analysis of fMRI data. Neuroinformatics 7:37-53. CrossRef Medline
17. Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615-1637. Medline
18. Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5:143-156. CrossRef Medline
19. Jenkinson M, Bannister P, Brady M, Smith S (2002) Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17:825-841. CrossRef Medline
20. Kriegeskorte N, Goebel R, Bandettini P (2006) Information-based functional brain mapping. Proc Natl Acad Sci U S A 103:3863-3868. CrossRef Medline
21. Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:289-297. CrossRef Medline
22. Lee SH, Kravitz DJ, Baker CI (2012) Disentangling visual imagery and perception of real-world objects. Neuroimage 59:4064-4073. CrossRef Medline
23. Meyer K, Damasio A (2009) Convergence and divergence in a neural architecture for recognition and memory. Trends Neurosci 32:376-382. CrossRef Medline
24. Meyer K, Kaplan JT, Essex R, Webber C, Damasio H, Damasio A (2010) Predicting visual stimuli on the basis of activity in auditory cortices. Nat Neurosci 13:667-668. CrossRef Medline
25. Meyer K, Kaplan JT, Essex R, Damasio H, Damasio A (2011) Seeing touch is correlated with content-specific activity in primary somatosensory cortex. Cereb Cortex 21:2113-2121. CrossRef Medline
26. Peelen MV, Atkinson AP, Vuilleumier P (2010) Supramodal representations of perceived emotions in the human brain. J Neurosci 30:10127-10134. CrossRef Medline
27. Pereira F, Botvinick M (2011) Information mapping with pattern classifiers: A comparative study. Neuroimage 56:476-496. CrossRef Medline
28. Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu WH, Cohen L, Guazzelli M, Haxby JV (2004) Beyond sensory images: Object-based representation in the human ventral pathway. Proc Natl Acad Sci U S A 101:5658-5663. CrossRef Medline
29. Reddy L, Tsuchiya N, Serre T (2010) Reading the mind's eye: decoding category information during mental imagery. Neuroimage 50:818-825. CrossRef Medline
30. Seltzer B, Pandya DN (1978) Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res 149:1-24. CrossRef Medline
31. Seltzer B, Pandya DN (1991) Post-rolandic cortical projections of the superior temporal sulcus in the rhesus monkey. J Comp Neurol 312:625-640. Medline
32. Seltzer B, Pandya DN (1994) Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J Comp Neurol 343:445-463. Medline
33. Shinkareva SV, Malave VL, Mason RA, Mitchell TM, Just MA (2011) Commonality of neural representations of words and pictures. Neuroimage 54:2418-2425. CrossRef Medline
34. Smith SM (2002) Fast robust automated brain extraction. Hum Brain Mapp 17:143-155. CrossRef Medline
35. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TEJ, Johansen-Berg H, Bannister PR, De Luca M, Drobnjak I, Flitney DE, Niazy RK, Saunders J, Vickers J, Zhang Y, De Stefano N, Brady JM, Matthews PM (2004) Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 23 [Suppl 1]:S208-S219. CrossRef Medline
36. Stokes M, Thompson R, Cusack R, Duncan J (2009) Top-down activation of shape-specific population codes in visual cortex during mental imagery. J Neurosci 29:1565-1572. CrossRef Medline
37. Vetter P, Smith FW, Muckli L (2011) Decoding natural sounds in early visual cortex. J Vis 11:779. CrossRef
38. Woolrich MW, Ripley BD, Brady M, Smith SM (2001) Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14: 1370-1386. CrossRef Medline
39. Zatorre RJ, Halpern AR (2005) Mental concerts: musical imagery and auditory cortex. Neuron 47:9-12. CrossRef Medline