Cross-modal prediction in speech perception

PLoS ONE

Volumn 6, Issue 10, 2011, Pages

  Sánchez García, Carolina ^a   Alsius, Agnès ^b   Enns, James T ^c   Soto Faraco, Salvador ^a,d  

^a UNIVERSITAT POMPEU FABRA   (Spain)

^b QUEEN S UNIVERSITY   (Canada)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^d INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; CONTROLLED STUDY; CROSS MODAL PREDICTION; DECISION MAKING; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; ONLINE ANALYTICAL PROCESSING; PREDICTION; SPEECH ANALYSIS; SPEECH PERCEPTION; VISUAL INFORMATION;

FACE; FEMALE; HUMANS; MALE; MODELS, BIOLOGICAL; PHOTIC STIMULATION; SOUND; SPEECH PERCEPTION; VISION, OCULAR; YOUNG ADULT;

EID: 80053581920     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0025198     Document Type: Article

References (40)

1. Stein BE, Meredith M, (1993) The merging of the senses Cambridge, MA MIT Press.
2. Calvert GA, Spence C, Stein BE, (2004) The Handbook of Multisensory Processing Cambridge, MA MIT Press.
3. McGurk H, MacDonald J, (1976) Hearing lips and seeing voices. Nature 264: 746-748.
4. Sumby WH, Pollack I, (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26: 212-215.
5. Ross LA, Saint-Amour D, Leavitt VM, Javitt DC, Foxe JJ, (2007) Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb Cortex 17: 1147-1153.
6. MacLeod A, Summerfield Q, (1987) Quantifying the contribution of vision to speech perception in noise. Br J Audiol 21: 131-141.
7. Arnold P, Hill F, (2001) Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact. Br J Psychol 92 Part 2: 339-355.
8. Navarra J, Soto-Faraco S, (2007) Hearing lips in a second language: visual articulatory information enables the perception of second language sounds. Psychol Res 71: 4-12.
9. Angelucci A, Bullier J, (2003) Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons? J Physiol Paris 97: 141-154.
10. Bubic A, von Cramon DY, Schubotz RI, (2010) Prediction, cognition and the brain. Front Hum Neurosci 4: 25.
11. Friston K, (2005) A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci 360: 815-836.
12. Rao RP, Ballard DH, (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2: 79-87.
13. Wolpert DM, (1997) Computational approaches to motor control. Trends Cogn Sci 1: 209-216.
14. Bar M, (2004) Visual objects in context. Nat Rev Neurosci 5: 617-629.
15. Lamme VA, (1995) The neurophysiology of figure-ground segregation in primary visual cortex. J Neurosci 15: 1605-1615.
16. Murray SO, Kersten D, Olshausen BA, Schrater P, Woods DL, (2002) Shape perception reduces activity in human primary visual cortex. Proc Natl Acad Sci U S A 99: 15164-15169.
17. Keller PE, Koch I, (2008) Action planning in sequential skills: relations to music performance. Q J Exp Psychol (Colchester) 61: 275-291.
18. Di Lollo V, Enns JT, Rensink RA, (2000) Competition for consciousness among visual events: the psychophysics of reentrant visual processes. J Exp Psychol Gen 129: 481-507.
19. Enns JT, Lleras A, (2008) What's next? New evidence for prediction in human vision. TRENDS Cogn Sci 12: 327-333.
20. Spratling MW, (2008) Predictive coding as a model of biased competition in visual attention. Vision Res 48: 1391-1408.
21. Summerfield C, Egner T, (2009) Expectation (and attention) in visual cognition. Trends Cogn Sci 13: 403-409.
22. Pickering MJ, Garrod S, (2007) Do people use language production to make predictions during comprehension? TRENDS Cogn Sci 11: 105-110.
23. Poeppel D, Idsardi WJ, van Wassenhove V, (2008) Speech perception at the interface of neurobiology and linguistics. Philos Trans R Soc Lond B Biol Sci 363: 1071-1086.
24. Skipper JI, van Wassenhove V, Nusbaum HC, Small SL, (2007) Hearing lips and seeing voices: how cortical areas supporting speech production mediate audiovisual speech perception. Cereb Cortex 17: 2387-2399.
25. Stevens KN, Halle M, (1967) Remarks on analysis by synthesis and distinctive features. In: Wathen-Dunn W, editors. Models for the perception of speech and visual form: proceedings of a symposium Cambridge, MA MIT Press pp. 88-102.
26. Van Wassenhove V, Grant KW, Poeppel D, (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci U S A 102: 1181-1186.
27. Van Berkum JJ, Brown CM, Zwitserlood P, Kooijman V, Hagoort P, (2005) Anticipating upcoming words in discourse: evidence from ERPs and reading times. J Exp Psychol Learn Mem Cogn 31: 443-467.
28. Dambacher M, Rolfs M, Gollner K, Kliegl R, Jacobs AM, (2009) Event-related potentials reveal rapid verification of predicted visual input. PLoS One 4: e5047.
29. DeLong KA, Urbach TP, Kutas M, (2005) Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nat Neurosci 8: 1117-1121.
30. Fowler CA, (2004) Speech as a supramodal or amodal phenomenon. In: Calvert G, Spence C, Stein BE, editors. The Handbook of Multisensory Processing Cambridge, MA MIT Press pp. 189-202.
31. Rosenblum LD, Miller RM, Sanchez K, (2007) Lip-read me now, hear me better later: cross-modal transfer of talker-familiarity effects. Psychol Sci 18: 392-396.
32. Skipper JI, Nusbaum HC, Small SL, (2005) Listening to talking faces: motor cortical activation during speech perception. Neuroimage 25: 76-89.
33. Kamachi M, Hill H, Lander K, Vatikiotis-Bateson E, (2003) "Putting the face to the voice": matching identity across modality. Curr Biol 13: 1709-1714.
34. Chandrasekaran C, Trubanova A, Stillittano S, Caplier A, Ghazanfar AA, (2009) The natural statistics of audiovisual speech. PLoS Comput Biol 5: e1000436.
35. Stekelenburg JJ, Vroomen J, (2007) Neural correlates of multisensory integration of ecologically valid audiovisual events. J Cogn Neurosci 19: 1964-1973.
36. Soto-Faraco S, Navarra J, Weikum WM, Vouloumanos A, Sebastian-Galles N, et al. (2007) Discriminating languages by speech-reading. Percept Psychophys 69: 218-231.
37. Summerfield Q, (1987) Some preliminaries to a comprehensive account of audio-visual speech perception. In: Dodd B, Campbell R, editors. Hearing by Eye: The Psychology of Lipreading New York (NY) Lawrence Erlbaum Associates pp. 3-51.
38. Schwartz JL, Robert-Ribes J, Escudier P, (1998) Ten years after Summerfield...a taxonomy of models for audiovisual fusion in speech perception. In: Campbell R, Dodd B, Burnham D, editors. Hearing by Eye, II. Perspectives and directions in research on audiovisual aspects of language processing Hove (UK) Psychology Press pp. 85-108.
39. Arnal LH, Morillon B, Kell CA, Giraud AL, (2009) Dual neural routing of visual facilitation in speech processing. J Neurosci 29: 13445-13453.
40. Forster KI, Forster JC, (2003) DMDX: a windows display program with millisecond accuracy. Behav Res Methods Instrum Comput 35: 116-124.