Detection of Audiovisual Speech Correspondences Without Visual Awareness

Psychological Science

Volumn 24, Issue 4, 2013, Pages 423-431

  Alsius, Agnès ^a   Munhall, Kevin G ^a  

^a QUEEN S UNIVERSITY   (Canada)

Author keywords

consciousness; speech perception

Indexed keywords

EID: 84876187287     PISSN: 09567976     EISSN: 14679280     Source Type: Journal    
DOI: 10.1177/0956797612457378     Document Type: Article

References (52)

1. Alais D.,van Boxtel J. J.,Parker A.,van Ee R.Attending to auditory signals slows visual alternations in binocular rivalry.Vision Research. 2010;50:929-935.
2. ArrighiR.MariniF.BurrD. (2009). Meaningful auditory information enhances perception of visual biological motion. Journal of Vision, 9(4), Article 25. Retrieved from http://www.journalofvision.org/content/9/4/25.full.
3. Bernstein L. E.,Auer E. T.,Takayanagi S.Auditory speech detection in noise is enhanced by lipreading.Speech Communication. 2004;44:5-18.
4. Bertelson P.,Francesco P.,Ladavas E.,Vroomen J.,de Gelder B.Ventriloquism in patients with unilateral visual neglect.Neuropsychologia. 2000;38:1634-1642.
5. Bolognini N.,Frassinetti F.,Serino A.,Ladavas E."Acoustical vision" of below threshold stimuli: Interaction among spatially converging audiovisual inputs.Experimental Brain Research. 2005;160:273-282.
6. ChandrasekaranC.TrubanovaA.StillittanoS.CaplierA.GhazanfarA. A. (2009). The natural statistics of audiovisual speech. PLoS Computational Biology, 5, e1000436. Retrieved from http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000436.
7. ChenY.-C.YehS.-L.SpenceC. (2011). Crossmodal constraints on human perceptual awareness: Auditory semantic modulation of binocular rivalry. Frontiers in Psychology, 2, 1-12. Retrieved from http://www.frontiersin.org/Perception_Science/10.3389/fpsyg.2011.00212/full.
8. ConradV.BartelsA.KleinerM.NoppeneyU. (2010). Audiovisual interactions in binocular rivalry. Journal of Vision, 10(10), Article 27. Retrieved from http://www.journalofvision.org/content/10/10/27/F1.expansion.html.
9. Driver J.,Noesselt T.Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments.Neuron. 2008;57:11-23.
10. Dufour A.,Touzalin P.,Moessinger M.,Brochard R.,Després O.Visual motion disambiguation by a subliminal sound.Consciousness and Cognition. 2008;17:790-797.
11. Eskelund K.,Tuomainen J.,Andersen T. S.Multistage audiovisual integration of speech: Dissociating identification and detection.Experimental Brain Research. 2011;208:447-457.
12. Falchier A.,Clavagnier S.,Barone P.,Kennedy H.Anatomical evidence of multimodal integration in primate striate cortex.Journal of Neuroscience. 2002;22:5749-5759.
13. Fowler C. A.The handbook of multisensory processes. Calvert G.Spence C.Stein B. E., ed. Cambridge, MA: MIT Press; 2004:189-202.
14. Frassinetti F.,Bolognini N.,Bottari D.,Bonora A.,Ladavas E.Audio-visual integration in patients with visual deficit.Journal of Cognitive Neuroscience. 2005;17:1442-1452.
15. Frassinetti F.,Bolognini N.,Ladavas E.Enhancement of visual perception by crossmodal visuo-auditory interaction.Experimental Brain Research. 2002;147:332-343.
16. Grant K. W.,Seitz P.The use of visible speech cues for improving auditory detection of spoken sentences.Journal of the Acoustical Society of America. 2000;108:1197-1208.
17. Hertrich I.,Mathiak K.,Lutzenberger W.,Menning H.,Ackermann H.Sequential audiovisual interactions during speech perception: A whole-head MEG study.Neuropsychologia. 2007;45:1342-1354.
18. Jiang Y.,Costello P.,He S.Processing of invisible stimuli advantage of upright faces and recognizable words in overcoming interocular suppression.Psychological Science. 2007;18:349-355.
19. Jiang Y.,He S.Cortical responses to invisible faces: Dissociating subsystems for facial-information processing.Current Biology. 2006;16:2023-2029.
20. Kim J.,Davis C.Hearing foreign voices: Does knowing what is said affect masked visual speech detection?.Perception. 2003;32:111-120.
21. Kim J.,Davis C.Investigating the audio-visual speech detection advantage.Speech Communication. 2004;44:19-30.
22. Kim J.,Kroos C.,Davis C.Hearing a point-light talker: An auditory influence on a visual detection task.Perception. 2010;39:407-416.
23. Klucharev V.,Möttönen R.,Sams M.Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception.Cognitive Brain Research. 2003;18:65-75.
24. Kouider S.,Dehaene S.Levels of processing during non-conscious perception: A critical review of visual masking.Philosophical Transactions of the Royal Society B: Biological Sciences. 2007;362:857-875.
25. Leo F.,Bolognini N.,Passamonti C.,Stein B. E.,Ladavas E.Cross-modal localization in hemianopia: New insights on multisensory integration.Brain. 2008;131:855-865.
26. McDonald J. J.,Ward L. M.Involuntary listening aids seeing: Evidence from human electrophysiology.Psychological Science. 2000;11:167-171.
27. McGurk H.,MacDonald J.Hearing lips and seeing voices.Nature. 1976;265:746-748.
28. Miller L. M.,D'Esposito M.Perceptual fusion and stimulus coincidence in the cross-modal integration of speech.Journal of Neuroscience. 2005;25:5884-5893.
29. Mitroff S. R.,Scholl B. J.Forming and updating object representations without awareness: Evidence from motion-induced blindness.Vision Research. 2005;45:961-967.
30. Morris S. B.,DeShon R. P.Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs.Psychological Methods. 2002;7:105-125.
31. Mudrik L.,Breska A.,Lamy D.,Deouell L.Integration without awareness: Expanding the limits of unconscious processing.Psychological Science. 2011;22:764-770.
32. Munhall K.,ten Hove M.,Brammer M.,Pare M.Audiovisual integration of speech in a bistable illusion.Current Biology. 2009;19:735-739.
33. Munhall K. G.,Vatikiotis-Bateson E.The handbook of multisensory processing. Calvert G. A.Spence C.Stein B. E., ed. Cambridge, MA: MIT Press; 2004:177-188.
34. Noesselt T.,Bergmann D.,Hake M.,Heinze H. J.,Fendrich R.Sound increases the saliency of visual events.Brain Research. 2008;1220:157-163.
35. Pearson J.,Clifford C. W.,Tong F.The functional impact of mental imagery on conscious perception.Current Biology. 2008;18:982-986.
36. Rubin E.Synsoplevede Figurer. Copenhagen, Denmark: Gyldenhal; 1915:.
37. Sánchez-GarcíaC.AlsiusA.EnnsJ. T.Soto-FaracoS. (2011). Cross-modal prediction in speech perception. PLoS ONE, 6(10), e25198. Retrieved from http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0025198.
38. Schroeder C.,Foxe J.The timing and laminar profi{ligature}le of converging inputs to multisensory areas of the macaque neocortex.Cognitive Brain Research. 2002;14:187-198.
39. Schwartz J. L.,Grimault N.,Hupé J. M.,Moore B. C.,Pressnitzer D.Multistability in perception: Binding sensory modalities, an overview.Philosophical Transactions of the Royal Society B: Biological Sciences. 2012;367:896-905.
40. Sekuler R.,Sekuler A. B.,Lau R.Sound alters visual motion perception.Nature. 1997;385:308.
41. Shams L.,Iwaki S.,Chawala A.,Bhattacharya J.Early modulation of visual cortex by sound: An MEG study.Neuroscience Letters. 2005;378:76-81.
42. Sheth B. R.,Shimojo S.Sound-aided recovery from and persistence against visual filling-in.Vision Research. 2004;44:1907-1917.
43. Stein B. E.,London N.,Wilkinson L.,Price D.Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis.Journal of Cognitive Neuroscience. 1996;8:497-506.
44. Stevenson R. A.,Altieri N. A.,Kim S.,Pisoni D. B.,James T. W.Neural processing of asynchronous audiovisual speech perception.NeuroImage. 2010;49:3308-3318.
45. Tsuchiya N.,Koch C.Continuous flash suppression reduces negative afterimages.Nature Neuroscience. 2005;8:1096-1101.
46. Tuomainen J.,Andersen T.,Tiippana K.,Sams M.Audio-visual speech perception is special.Cognition. 2005;96:B13-B22.
47. van Ee R.,van Boxtel J. J.,Parker A. L.,Alais D.Multisensory congruency as a mechanism for attentional control over perceptual selection.Journal of Neuroscience. 2009;29:11641-11649.
48. Vroomen J.,de Gelder B.Sound enhances visual perception: Cross-modal effects of auditory organization on vision.Journal of Experimental Psychology: Human Perception and Performance. 2000;26:1583-1590.
49. Watkins S.,Shams L.,Tanaka S.,Haynes J.-D.,Rees G.Sound alters activity in human V1 in association with illusory visual perception.NeuroImage. 2006;31:1247-1256.
50. Yamada Y.,Kawabe T.Illusory line motion and transformational apparent motion during continuous flash suppression.Japanese Psychological Research. 2012;54:348-359.
51. Yang E.,Zald D. H.,Blake R.Fearful expressions gain preferential access to awareness during continuous flash suppression.Emotion. 2007;7:882-886.
52. Yehia H.,Rubin P.,Vatikiotis-Bateson E.Quantitative association of vocal-tract and facial behaviour.Speech Communication. 1998;26:23-43.