Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions

PLoS ONE

Volumn 8, Issue 9, 2013, Pages

  Smith, Elliot ^a,b   Duede, Scott ^c   Hanrahan, Sara ^b   Davis, Tyler ^a,b   House, Paul ^a   Greger, Bradley ^a,b  

^a UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

^b Department of Electrical and Computer Engineering   (United States)

^c UNIVERSITY OF UTAH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUDITORY CORTEX; AUDITORY STIMULATION; BRAIN ELECTROPHYSIOLOGY; CASE REPORT; COGNITION; HEARING; HUMAN; MENTAL FUNCTION; NEUROTRANSMISSION; OCCIPITAL CORTEX; PHONEME; STIMULUS RESPONSE; TEMPORAL GYRUS; VISUAL STIMULATION;

AUDITORY CORTEX; AUDITORY PERCEPTION; HUMANS; ILLUSIONS; MALE; PHOTIC STIMULATION;

EID: 84883339369     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0073148     Document Type: Article

References (29)

1. McGurk H, Macdonald J, (1976) Hearing lips and seeing voices. Nature 264: 746-748.
2. Jiang J, Bernstein LE, (2011) Psychophysics of the McGurk and other audiovisual speech integration effects. Journal of Experimental Psychology Human Perception and Performance 37: 1193-1209.
3. Hackett TA, (2011) Information flow in the auditory cortical network. Hearing Research 271: 133-146.
4. Rauschecker JP, Scott SK, (2009) Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nature Neuroscience 12: 718-724.
5. Romanski LM, (2007) Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cerebral Cortex (New York, NY: 1991) 17Suppl 1: i61-i69.
6. Ungerleider LG, Haxby JV, (1994) 'What' and 'where' in the human brain. Current Opinions in Neurobiology 4: 157-165.
7. Chang EF, Rieger JW, Johnson K, Berger MS, Barbaro NM, et al. (2010) Categorical speech representation in human superior temporal gyrus. Nature Neuroscience 13: 1428-1432.
8. Pasley BN, David SV, Mesgarani N, Flinker A, Shamma SA, et al. (2012) Reconstructing speech from human auditory cortex. PLoS Biology 10: e1001251.
9. Blakely T, Miller KJ, Rao RP, Holmes MD, Ojemann JG, (2008) Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Conference 2008: 4964-4967.
10. Ghazanfar AA, Chandrasekaran C, Logothetis NK, (2008) Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. Journal of Neuroscience 28: 4457-4469.
11. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK, (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience 25: 5004-5012.
12. Besle J, Bertrand O, Giard M-H, (2009) Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex. Hearing Research 258: 143-151.
13. Besle J, Fischer C, Bidet-Caulet A, Lecaignard F, Bertrand O, et al. (2008) Visual activation and audiovisual interactions in the auditory cortex during speech perception: intracranial recordings in humans. Journal of Neuroscience 28: 14301-14310.
14. Senkowski D, Gomez-Ramirez M, Lakatos P, Wylie GR, Molholm S, et al. (2007) Multisensory processing and oscillatory activity: analyzing non-linear electrophysiological measures in humans and simians. Experimental Brain Research 177: 184-195.
15. Sams M, Aulanko R, Hämäläinen M, Hari R, Lounasmaa OV, et al. (1991) Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters 127: 141-145.
16. van Wassenhove V, Grant KW, Poeppel D, (2005) Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America 102: 1181-1186.
17. Buzsáki G, Anastassiou CA, Koch C, (2012) The origin of extracellular fields and currents-EEG, ECoG, LFP and spikes. Nature Reviews Neuroscience 13: 407-420.
18. Miller KJ, Sorensen LB, Ojemann JG, den Nijs M, (2009) Power-law scaling in the brain surface electric potential. PLoS Computational Biology 5: e1000609.
19. Manning JR, Jacobs J, Fried I, Kahana MJ, (2009) Broadband shifts in local field potential power spectra are correlated with single-neuron spiking in humans. Journal of Neuroscience 29: 13613-13620.
20. Hermes D, Miller KJ, Noordmans HJ, Vansteensel MJ, Ramsey NF, (2010) Automated electrocorticographic electrode localization on individually rendered brain surfaces. Journal of Neuroscience Methods 185: 293-298.
21. Kellis S, Miller K, Thomson K, Brown R, House P, et al. (2010) Decoding spoken words using local field potentials recorded from the cortical surface. Journal of Neural Engineering 7: 056007.
22. Smith E, Kellis S, House P, Greger B, (2013) Decoding stimulus identity from multi-unit activity and local field potentials along the ventral auditory stream in the awake primate: implications for cortical neural prostheses. Journal of Neural Engineering 10: 016010.
23. Li X, Ouyang G, (2010) Estimating coupling direction between neuronal populations with permutation conditional mutual information. NeuroImage 52: 497-507.
24. Smith EH, Kellis SS, House P, Greger B, (2012) Information transfer along the ventral auditory stream in the awake macaque. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Conference 2012: 5178-5181.
25. Mesgarani N, Chang EF, (2012) Selective cortical representation of attended speaker in multi-talker speech perception. Nature 485: 233-236.
26. Ghazanfar AA, Schroeder CE, (2006) Is neocortex essentially multisensory? Trends in Cognitive Sciences 10: 278-285.
27. Rosenblum LD, Schmuckler MA, Johnson JA, (1997) The McGurk effect in infants. Perception & Psychophysics 59: 347-357.
28. Bokil H, Andrews P, Kulkarni JE, Mehta S, Mitra PP, (2010) Chronux: a platform for analyzing neural signals. Journal of Neuroscience Methods 192: 146-151.
29. Krzanowski W (1990) Principles of Multivariate Analysis. Oxford: Oxford University Press.