Natural asynchronies in audiovisual communication signals regulate neuronal multisensory interactions in voice-sensitive cortex

Proceedings of the National Academy of Sciences of the United States of America

Volumn 112, Issue 1, 2015, Pages 273-278

  Perrodin, Catherine ^a   Kayser, Christoph ^b   Logothetis, Nikos K ^a,c   Petkov, Christopher I ^d  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d NEWCASTLE UNIVERSITY   (United Kingdom)

Author keywords

Communication; Multisensory; Neurons; Oscillations; Voice

Indexed keywords

ADULT; ANIMAL EXPERIMENT; ARTICLE; AUDIOVISUAL AID; AUDITORY CORTEX; AUDITORY RESPONSE; AUDITORY STIMULATION; BRAIN ELECTROPHYSIOLOGY; BRAIN REGION; ELECTROENCEPHALOGRAM; EVOKED AUDITORY RESPONSE; EVOKED RESPONSE; EVOKED VISUAL RESPONSE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; IMITATION; INTERPERSONAL COMMUNICATION; MALE; NERVE CELL; NERVE CELL EXCITABILITY; NERVE EXCITABILITY; NERVE STIMULATION; NONHUMAN; OSCILLATION; RHESUS MONKEY; SENSORY STIMULATION; SPIKE; STIMULUS; SUPERIOR TEMPORAL SULCUS; TEMPORAL CORTEX; VISUAL STIMULATION; VOCALIZATION; VOICE ONSET TIME; ANIMAL; ANIMAL COMMUNICATION; BRAIN; HEARING; HUMAN; PHYSIOLOGY; SENSATION; VISION; VOICE;

ANIMALIA; MACACA; PRIMATES;

ANIMAL COMMUNICATION; ANIMALS; AUDITORY PERCEPTION; BRAIN; EVOKED POTENTIALS, VISUAL; HUMANS; MACACA MULATTA; MALE; NEURONS; SENSATION; VISUAL PERCEPTION; VOICE;

EID: 84920385595     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1412817112     Document Type: Article

References (57)

1. Shams L, Kamitani Y, Shimojo S (2000) Illusions. What you see is what you hear. Nature 408(6814):788.
2. Howard IP, Templeton WB (1966) Human Spatial Orientation (Wiley, London), p 533.
3. Slutsky DA, Recanzone GH (2001) Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12(1):7-10.
4. McGrath M, Summerfield Q (1985) Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. J Acoust Soc Am 77(2):678-685.
5. Stevenson RA, Wallace MT (2013) Multisensory temporal integration: Task and stimulus dependencies. Exp Brain Res 227(2):249-261.
6. van Wassenhove V, Grant KW, Poeppel D (2007) Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45(3):598-607.
7. Vatakis A, Spence C (2006) Audiovisual synchrony perception for music, speech, and object actions. Brain Res 1111(1):134-142.
8. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53(2):279-292.
9. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25(20):5004-5012.
10. Chandrasekaran C, Ghazanfar AA (2009) Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. J Neurophysiol 101(2):773-788.
11. Rauschecker JP, Scott SK (2009) Maps and streams in the auditory cortex: Nonhuman primates illuminate human speech processing. Nat Neurosci 12(6):718-724.
12. Perrodin C, Kayser C, Logothetis NK, Petkov CI (2014) Auditory and visual modulation of temporal lobe neurons in voice-sensitive and association cortices. J Neurosci 34(7):2524-2537.
13. Perrodin C, Kayser C, Logothetis NK, Petkov CI (2011) Voice cells in the primate temporal lobe. Curr Biol 21(16):1408-1415.
14. Kikuchi Y, Horwitz B, Mishkin M (2010) Hierarchical auditory processing directed rostrally along the monkey's supratemporal plane. J Neurosci 30(39):13021-13030.
15. Bendor D, Wang X (2007) Differential neural coding of acoustic flutter within primate auditory cortex. Nat Neurosci 10(6):763-771.
16. Creutzfeldt O, Hellweg FC, Schreiner C (1980) Thalamocortical transformation of responses to complex auditory stimuli. Exp Brain Res 39(1):87-104.
17. Ghitza O (2011) Linking speech perception and neurophysiology: Speech decoding guided by cascaded oscillators locked to the input rhythm. Front Psychol 2:130.
18. Giraud AL, Poeppel D (2012) Cortical oscillations and speech processing: Emerging computational principles and operations. Nat Neurosci 15(4):511-517.
19. Ding N, Chatterjee M, Simon JZ (2013) Robust cortical entrainment to the speech envelope relies on the spectro-temporal fine structure. Neuroimage 88C:41-46.
20. Ng BS, Logothetis NK, Kayser C (2013) EEG phase patterns reflect the selectivity of neural firing. Cereb Cortex 23(2):389-398.
21. Engel AK, Senkowski D, Schneider TR (2012) Multisensory integration through neural coherence. The Neural Bases of Multisensory Processes, Frontiers in Neuroscience, eds Murray MM, Wallace MT (CRC, Boca Raton, FL).
22. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A (2008) Neuronal oscillations and visual amplification of speech. Trends Cogn Sci 12(3):106-113.
23. Thut G, Miniussi C, Gross J (2012) The functional importance of rhythmic activity in the brain. Curr Biol 22(16):R658-R663.
24. Ng BS, Schroeder T, Kayser C (2012) A precluding but not ensuring role of entrained low-frequency oscillations for auditory perception. J Neurosci 32(35):12268-12276.
25. Kayser C, Petkov CI, Logothetis NK (2008) Visual modulation of neurons in auditory cortex. Cereb Cortex 18(7):1560-1574.
26. Thorne JD, De Vos M, Viola FC, Debener S (2011) Cross-modal phase reset predicts auditory task performance in humans. J Neurosci 31(10):3853-3861.
27. van Atteveldt N, Murray MM, Thut G, Schroeder CE (2014) Multisensory integration: Flexible use of general operations. Neuron 81(6):1240-1253.
28. Lakatos P, et al. (2005) An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J Neurophysiol 94(3):1904-1911.
29. Sugihara T, Diltz MD, Averbeck BB, Romanski LM (2006) Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J Neurosci 26(43):11138-11147.
30. Bizley JK, Nodal FR, Bajo VM, Nelken I, King AJ (2007) Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cereb Cortex 17(9):2172-2189.
31. Petkov CI, et al. (2008) A voice region in the monkey brain. Nat Neurosci 11(3):367-374.
32. Belin P, Zatorre RJ, Lafaille P, Ahad P, Pike B (2000) Voice-selective areas in human auditory cortex. Nature 403(6767):309-312.
33. Werner S, Noppeney U (2010) Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J Neurosci 30(7):2662-2675.
34. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10(6):278-285.
35. Tsao DY, Freiwald WA, Tootell RB, Livingstone MS (2006) A cortical region consisting entirely of face-selective cells. Science 311(5761):670-674.
36. Leopold DA, Bondar IV, Giese MA (2006) Norm-based face encoding by single neurons in the monkey inferotemporal cortex. Nature 442(7102):572-575.
37. Perrett DI, Rolls ET, Caan W (1982) Visual neurones responsive to faces in the monkey temporal cortex. Exp Brain Res 47(3):329-342.
38. Schall S, Kiebel SJ, Maess B, von Kriegstein K (2013) Early auditory sensory processing of voices is facilitated by visual mechanisms. Neuroimage 77:237-245.
39. Dahl CD, Logothetis NK, Kayser C (2010) Modulation of visual responses in the superior temporal sulcus by audio-visual congruency. Front Integr Neurosci 4:10.
40. Romei V, Gross J, Thut G (2012) Sounds reset rhythms of visual cortex and corresponding human visual perception. Curr Biol 22(9):807-813.
41. Zampini M, Guest S, Shore DI, Spence C (2005) Audio-visual simultaneity judgments. Percept Psychophys 67(3):531-544.
42. Chandrasekaran C, Trubanova A, Stillittano S, Caplier A, Ghazanfar AA (2009) The natural statistics of audiovisual speech. PLOS Comput Biol 5(7):e1000436.
43. Kayser C, Logothetis NK, Panzeri S (2010) Visual enhancement of the information representation in auditory cortex. Curr Biol 20(1):19-24.
44. Ohshiro T, Angelaki DE, DeAngelis GC (2011) A normalization model of multisensory integration. Nat Neurosci 14(6):775-782.
45. Henry MJ, Obleser J (2012) Frequency modulation entrains slow neural oscillations and optimizes human listening behavior. Proc Natl Acad Sci USA 109(49):20095-20100.
46. Henry MJ, Herrmann B, Obleser J (2014) Entrained neural oscillations in multiple frequency bands comodulate behavior. Proc Natl Acad Sci USA 111(41):14935-14940.
47. Busch NA, Dubois J, VanRullen R (2009) The phase of ongoing EEG oscillations predicts visual perception. J Neurosci 29(24):7869-7876.
48. Fiebelkorn IC, et al. (2011) Ready, set, reset: Stimulus-locked periodicity in behavioral performance demonstrates the consequences of cross-sensory phase reset. J Neurosci 31(27):9971-9981.
49. Gleiss S, Kayser C (2013) Eccentricity dependent auditory enhancement of visual stimulus detection but not discrimination. Front Integr Neurosci 7:52.
50. Powers AR, 3rd, Hillock AR, Wallace MT (2009) Perceptual training narrows the temporal window of multisensory binding. J Neurosci 29(39):12265-12274.
51. Landau AN, Fries P (2012) Attention samples stimuli rhythmically. Curr Biol 22(11):1000-1004.
52. Lakatos P, et al. (2013) The spectrotemporal filter mechanism of auditory selective attention. Neuron 77(4):750-761.
53. Ghazanfar AA, Nielsen K, Logothetis NK (2006) Eye movements of monkey observers viewing vocalizing conspecifics. Cognition 101(3):515-529.
54. Lansing CR, McConkie GW (2003) Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Percept Psychophys 65(4):536-552.
55. Lakatos P, et al. (2009) The leading sense: Supramodal control of neurophysiological context by attention. Neuron 64(3):419-430.
56. Stanford TR, Quessy S, Stein BE (2005) Evaluating the operations underlying multisensory integration in the cat superior colliculus. J Neurosci 25(28):6499-6508.
57. Berens P (2009) CircStat: A MATLAB toolbox for circular statistics. J Stat Softw 31(10):1-21.