Data and simulations about audiovisual asynchrony and predictability in speech perception

Auditory-Visual Speech Processing 2013, AVSP 2013

Volumn , Issue , 2013, Pages 147-152

  Jean Luc, Schwartz ^a   Christophe, Savariaux ^a  

^a UNIV GRENOBLE ALPES   (France)

Author keywords

audiovisual asynchrony; predictive coding; temporal integration window; visual lead lag; visual prediction

Indexed keywords

ASYNCHRONY; AUDIOVISUAL ASYNCHRONY; INTEGRATION WINDOW; LEAD-LAG; PREDICTIVE CODING; SPEECH PERCEPTION; TEMPORAL INTEGRATION; TEMPORAL INTEGRATION WINDOW; VISUAL LEAD/LAG; VISUAL PREDICTION;

SPEECH PROCESSING;

EID: 84905485735     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. Abry, C., & Lallouache, T. M. (1995). Modeling lip constriction anticipatory behaviour for rounding in French with the MEM. Proc. ICPhS'95, 152-155.
2. Abry C., & Lallouache T. (1996). Le MEM: Un modèle d'anticipation paramétrable par locuteur. Données sur l'arrondissement en français (MEM: A speaker parameterized anticipation model). Bul. de la Comm. Parlée 3, 85-99.
3. Abry C., Lallouache M. T., & Cathiard M. A. (1996). How can coarticulation models account for speech sensitivity to audiovisual desynchronization? In Stork D. and Hennecke M. (Eds.) Speechreading by Humans and Machines, NATO ASI Series F (vol. 150, pp. 247-255). Berlin: Springer-Verlag.
4. Arnal, L.H., Morillon, B., Kell, C.A., & Giraud, A.L. (2009). Dual Neural Routing of Visual Facilitation in Speech Processing. Journal of Neuroscience, 29, 13445-53.
5. Arnal, L.H., Wyart, V., & Giraud, A.L. (2011). Transitions in neural oscillations reflect prediction errors generated in audiovisual speech. Nature Neuroscience, doi:10.1038/nn.2810.
6. Barker, J. P., & Berthommier, F. (1999). Evidence of correlation between acoustic and visual features of speech. In Proceedings ICPhS'99 (pp. 199-202.) San Francisco: USA.
7. Bell-Berti, F., & Harris, K. S. (1982). Temporal patterns of coarticulation: Lip rounding. J. Acoust. Soc. Am. 71, 449-459.
8. Benguerel, A. P., & Cowan, H. A. (1974). Coarticulation of upper lip protrusion in French. Phonetica 30, 41-55.
9. Bernstein, L. E., Takayanagi, S., & Auer, E. T., Jr. (2004). Auditory speech detection in noise enhanced by lipreading. Speech Communication 44, 5-18.
10. Berthommier, F. (2004). A phonetically neutral model of the low-level audiovisual interaction. Speech Communication 44, 31-41.
11. Besle, J., Fort, A., Delpuech, C. & Giard, M-H. (2004). Bimodal Speech: Early Visual Effect in the Human Auditory Cortex. European Journal of Neuroscience, 20, 2225-2234.
12. Bessière, P., Laugier, C., & Siegwart, R. (Eds.) (2008). Probabilistic Reasoning and Decision Making in Sensory-Motor Systems. Springer Tracts in Advanced Robotics. Berlin: Springer-Verlag Series, vol. 46.
13. Boersma, P., & Weenink, D. (2012). Praat: doing phonetics by computer (Version 5.3.04) [Computer program]. Retrieved May 2012, from http://www.praat.org.
14. Brungart, D.S., Iyer, N., Simpson, B.D., & van Wassenhove, V. (2008). The effects of temporal asynchrony on the intelligibility of accelerated speech. Proc. AVSP' 2008, 19-24.
15. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A.A. (2009). The natural statistics of audiovisual speech. PLoS Computational Biology, 5: e1000436.
16. Czap, László (2011). On the audiovisual asynchrony of speech, Proc AVSP'2011, 137-140.
17. Feldhoffer, G., Bárdi, T., Takács, Gy. & Tihanyi, A. (2007). Temporal asymmetry in relations of acoustic and visual features of speech. Proc. 15th European Signal Processing Conf., Poznan.
18. Friston, K. (2005). A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci, 360, 815-836.
19. Grant, K. W., & Seitz, P. (2000). The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am. 108, 1197-1208.
20. Henke, W.L. (1966). Dynamic articulatory model of speech production using computer simulation. Dissertation Massachusetts Institute of Technology
21. Jesse, A., & Massaro, D. W. (2010). The temporal distribution of information in audiovisual spoken-word identification. Attention, Perception & Psychophysics 72, 209-225.
22. Jiang, J., Alwan, A., Keating, P.A., Auer, E.T. Jr, & Bernstein, L.E. (2002). On the Relationship between Face Movements, Tongue Movements, and Speech Acoustics. Eurasip Journal on Advances in Signal Processing 11, 1174-1188.
23. Kim, J., & Davis, C. (2004). Investigating the audio-visual speeh detection advantage. Speech Commun, 44, 19-30.
24. Lallouache, M.T. (1990). Un poste 'visage-parole'. Acquisition et traitement de contours labiaux. Proc. XVIII Journées d'Études sur la Parole, 282-286.
25. Massaro, D. W., Cohen, M. M., & Smeele, P. M. (1996). Perception of asynchronous and conflicting visual and auditory speech. J. Acoust. Soc. Am. 100, 1777-1786.
26. Munhall, K., Gribble, P., Sacco, L., & Ward, M. (1996). Temporal constraints on the McGurk effect. Perception and Psychophysics 58, 351-362.
27. Munhall, K.G., & Tohkura, Y. (1998). Audiovisual gating and the time course of speech perception. J. Acoust. Soc. Am. 104, 530-539.
28. Musacchia, C., & Schroeder, C.E. (2009). Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hearing Research 258, 72-79.
29. Nahorna, O., Berthommier, F., & Schwartz, J.L. (2012). Binding and unbinding the auditory and visual streams in the McGurk effect. J. Acoust. Soc. Am. 132, 1061-1077.
30. Noiray, A., Cathiard, M.-A., Abry, C., Ménard, L. & Savariaux, C. (2008). Emergence of a vowel gesture control: Attunement of the anticipatory rounding temporal pattern in French children. In Kern S., Gayraud F. & Marsico E. (Eds.) Emergence of Language Abilities (pp. 100-117). Newcastle: Cambridge Scholars Pub.
31. Perkell, J. S., & Chiang, C. (1986). Preliminary support for a 'hybrid model' of anticipatory coarticulation. In Proc. 12th International Conference of Acoustics, A3-A6.
32. Perkell, J. S., & Matthies, L. M. (1992). Temporal measures of anticipatory labial coarticulation for the vowel/u/: Within- and cross-subject variability. J. Acoust. Soc. Am. 91, 2911-2925.
33. Roy, J.-R., Sock, R., Vaxelaire, B., & Hirsch, F. (2003). Auditory effects of anticipatory and carryover coarticulation. In Proc. 6th Int. Sem. Speech Production, Macquarie Series for Cognitive Sciences, 243-248.
34. Schwartz, J.L., Berthommier, F., & Savariaux, C. (2004). Seeing to hear better: Evidence for early audio-visual interactions in speech identification. Cognition 93, B69-B78.
35. Schwartz J.-L., Boë L.-J., Badin P., & Sawallis R. T. (2012). Grounding stop place systems in the perceptuo-motor substance of speech: On the universality of the labial-coronal-velar stop series. Journal of Phonetics 40, 20-36.
36. Smeele, P. M. T. (1994). Perceiving speech: Integrating auditory and visual speech. Unpublished doctoral dissertation, Delft University of Technology.
37. Troille, E., Cathiard, M.A., & Abry, C. (2010). Speech face perception is locked to anticipation in speech production. Speech Communication 52, 513-524.
38. de la Vaux, S. K., & Massaro, D.W. (2004). Audiovisual Speech Gating: Examining Information and Information Processing. Cognitive Processing 5, 106-112.
39. van Wassenhove, V., Grant, K.W., & Poeppel, D. (2005) Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Science, 102, 1181-1186.
40. van Wassenhove, V., Grant, K.W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45, 598-607.
41. Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998) Quantitative association of vocal tract and facial behavior. Speech Communication 26, 23-43.
42. Zion Golumbic, E., Cogan, G.B., Schroeder, C.E., & Poeppel, D. (2013). Visual input enhances selective speech envelope tracking in auditory cortex at a "cocktail party. J Neurosci. 33, 1417-26.