Coding of multisensory temporal patterns in human superior temporal sulcus

Frontiers in Integrative Neuroscience

Volumn , Issue AUGUST 2012, 2012, Pages

  Noesselt, Tömme ^a   Bergmann, Daniel ^a,b   Heinze, Hans Jochen ^a   Münte, Thomas ^c   Spence, Charles ^d  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

^b Asklepios Westklinikum Hamburg   (Germany)

^c UNIVERSITY OF LÜBECK   (Germany)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Audiovisual; fMRI; Human; Speech; Temporal perception

Indexed keywords

ARTICLE; AUDITORY STIMULATION; BOLD SIGNAL; BRAIN FUNCTION; EPISODIC MEMORY; EYE MOVEMENT; EYELID REFLEX; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEMODYNAMICS; HUMAN; NEUROIMAGING; PREFRONTAL CORTEX; SPEECH ANALYSIS; STIMULUS RESPONSE; SUPERIOR TEMPORAL SULCUS; SUPPLEMENTARY MOTOR AREA; TEMPORAL GYRUS; VOXEL BASED MORPHOMETRY;

EID: 84865040276     PISSN: 16625145     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnint.2012.00064     Document Type: Article

References (52)

1. Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI. (2005): Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J CognNeurosci 17(3):377-91.
2. Beauchamp MS. (2005a): See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. CurrOpinNeurobiol 15(2):145-53.
3. Beauchamp MS. (2005b): Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3(2):93-113.
4. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A. (2004a): Unraveling multisensory integration: patchy organization within human STS-c multisensory cortex. Nat Neurosci 7(11):1190-2.
5. Beauchamp MS, Lee KE, Argall BD, Martin A. (2004b): Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41(5):809-23.
6. Benevento LA, Fallon J, Davis BJ, Rezak M. (1977): Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. ExpNeurol 57(3):849-72.
7. Bruce C, Desimone R, Gross CG. (1981): Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46(2):369-84.
8. Bushara KO, Grafman J, Hallett M. (2001): Neural correlates of auditory-visual stimulus onset asynchrony detection. J Neurosci 21(1):300-4.
9. Calvert GA. (2001): Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11(12):1110-23.
10. Calvert GA, Campbell R. (2003): Reading speech from still and moving faces: the neural substrates of visible speech. J CognNeurosci 15(1):57-70.
11. Calvert GA, Campbell R, Brammer MJ. (2000): Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10(11):649-57.
12. Calvert GA, Hansen PC, Iversen SD, Brammer MJ. (2001): Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14(2):427-38.
13. Cusick CG, Seltzer B, Cola M, Griggs E. (1995): Chemoarchitectonics and corticocortical terminations within the superior temporal sulcus of the rhesus monkey: evidence for subdivisions of superior temporal polysensory cortex. J Comp Neurol 360(3):513-35.
14. Dennett D. (1991). Consciousness explained. London: Penguin Press.
15. Desimone R, Gross CG. (1979): Visual areas in the temporal cortex of the macaque. Brain Res 178(2-3):363-80.
16. Dhamala M, Assisi CG, Jirsa VK, Steinberg FL, Kelso JA. (2007): Multisensory integration for timing engages different brain networks. Neuroimage 34(2):764-73.
17. Dixon NF, Spitz L. (1980): The detection of auditory visual desynchrony. Perception 9(6):719-21.
18. Dosenbach NU, Visscher KM, Palmer ED, Miezin FM, Wenger KK, Kang HC, Burgund ED, Grimes AL, Schlaggar BL, Petersen SE. (2006): A core system for the implementation of task sets. Neuron 50(5):799-812.
19. Driver J, Noesselt T. (2008): Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57(1):11-23.
20. Fairhall S, Macaluso E. (2009): Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. European J Neurosci 29(6):1247-57.
21. Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ. (1997): Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6(3):218-29.
22. Ghazanfar AA, Chandrasekaran C, Logothetis NK. (2008): Interactions between the superior 8 temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J Neurosci 28(17):4457-69.
23. Gil-da-Costa R, Braun A, Lopes M, Hauser MD, Carson RE, Herscovitch P, Martin A. (2004): Toward an evolutionary perspective on conceptual representation: speciesspecific calls activate visual and affective processing systems in the macaque. ProcNatlAcadSci U S A 101(50):17516-21.
24. Heekeren HR, Marrett S, Ungerleider LG. (2008): The neural systems that mediate human perceptual decision making. Nat Rev Neurosci 9(6):467-79.
25. Hikosaka K, Iwai E, Saito H, Tanaka K. (1988): Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60(5):1615-37.
26. Kanowski M, Rieger JW, Noesselt T, Tempelmann C, Hinrichs H. (2007): Endoscopic eye tracking system for fMRI. J Neurosci Methods 160(1):10-5.
27. Kelly SD. 2005. The puzzle of temporal experience. In: Akins ABK, editor. Cognition and the brain: The philosophy and neuroscience movement Cambridge: Cambridge University Press. Pp. 208-40.
28. Kleinschmidt A, Buchel C, Zeki S, Frackowiak RS. (1998): Human brain activity during spontaneously reversing perception of ambiguous figures. ProcBiolSci 265(1413):2427-33.
29. Köhler W. 1947. Gestalt psychology: An introduction to new concepts in modern psychology. New York: Liveright Publication Corporation.
30. Macaluso E, Driver J. (2001): Spatial attention and crossmodal interactions between vision and touch. Neuropsychologia 39(12):1304-16.
31. Macaluso E, George N, Dolan R, Spence C, Driver J. (2004): Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 21(2):725-32.
32. Marchant, J. L., Ruff, C. C., & Driver, J. (2012). Audiovisual synchrony enhances BOLD responses in a brain network including multisensory STS while also enhancing targetdetection performance for both modalities. Hum Brain Mapp, 33(5):1212-24.
33. McGrath M, Summerfield Q. (1985): Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. J AcoustSoc Am 77(2):678-85.
34. Miller LM, D'Esposito M. (2005): Perceptual fusion and stimulus coincidence in the crossmodal integration of speech. J Neurosci 25(25):5884-93.
35. Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J. (2007): Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci 27(42):11431-41.
36. Ochiai T, Grimault S, Scavarda D, Roch G, Hori T, Riviere D, Mangin JF, Regis J. (2004): Sulcal pattern and morphology of the superior temporal sulcus. Neuroimage 22(2):706-19.
37. Padberg J, Seltzer B, Cusick CG. (2003): Architectonics and cortical connections of the upper bank of the superior temporal sulcus in the rhesus monkey: an analysis in the tangential plane. J Comp Neurol 467(3):418-34.
38. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A. (2008): Neuronal oscillations and visual amplification of speech. Trends CognSci 12(3):106-13.
39. Seltzer B, Pandya DN. (1991): Post-rolandic cortical projections of the superior temporal sulcus in the rhesus monkey. J Comp Neurol 312(4):625-40.
40. Slutsky DA, Recanzone GH. (2001): Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12(1):7-10.
41. Spence C, Squire S. (2003): Multisensory integration: maintaining the perception of synchrony. CurrBiol 13(13):R519-21.
42. Stein BE, Meredith MA. 1993. The merging of the senses. Cambridge: MIT Press.
43. Stevenson RA, Altieri NA, Kim S, Pisoni DB, James TW (2010): Neural processing of asynchronous audiovisual speech perception. Neuroimage 49(4):33-38.
44. Szycik GR, Tausche P, Munte TF. (2008): A novel approach to study audiovisual integration in speech perception: Localizer fMRI and sparse sampling. Brain Res 1220:142-9.
45. Tsao DY, Freiwald WA, Tootell RB, Livingstone MS. (2006): A cortical region consisting entirely of face-selective cells. Science 311(5761):670-4.
46. van Atteveldt N, Formisano E, Goebel R, Blomert L. (2004): Integration of letters and speech sounds in the human brain. Neuron 43(2):271-82.
47. van Atteveldt NM, Formisano E, Blomert L, Goebel R. (2007): The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17(4):962-74.
48. Vatakis A, Spence C. (2006a): Audiovisual synchrony perception for music, speech, and object actions. Brain Res 1111(1):134-42.
49. Vatakis A, Spence C. (2006b): Audiovisual synchrony perception for speech and music assessed using a temporal order judgment task. NeurosciLett 393(1):40-4.
50. Vroomen, J., & Keetels, M. (2010). Perception of intersensory synchrony: A tutorial review. Attention, Perception, & Psychophysics, (72): 871-884.
51. Wright TM, Pelphrey KA, Allison T, McKeown MJ, McCarthy G. (2003): Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb Cortex 13(10):1034-43.
52. Yeterian EH, Pandya, DN., Tomaiuolo, F., Petrides, M (2012). The cortical connectivity of the prefrontal cortex in themonkey brain. Cortex (48):58-81.