The effect of lip-reading on primary stream segregation

Journal of the Acoustical Society of America

Volumn 130, Issue 1, 2011, Pages 283-291

  Devergie, Aymeric ^a   Grimault, Nicolas ^a   Gaudrain, Etienne ^b   Healy, Eric W ^c   Berthommier, Frédéric ^d  

^a UNIVERSITÉ LYON 1   (France)

^b MRC COGNITION AND BRAIN SCIENCES UNIT   (United Kingdom)

^c OHIO STATE UNIVERSITY   (United States)

^d GIPSA LAB   (France)

Author keywords

[No Author keywords available]

Indexed keywords

AUDIO-VISUAL INTERACTIONS; AUDITORY STREAMING; BEHAVIORAL EXPERIMENT; FUNDAMENTAL FREQUENCIES; STREAM SEGREGATION;

EXPERIMENTS; SPEECH INTELLIGIBILITY;

LINGUISTICS;

ADOLESCENT; ADULT; ANALYSIS OF VARIANCE; ARTICLE; ASSOCIATION; AUDITORY STIMULATION; AUDITORY THRESHOLD; HUMAN; LIP READING; NOISE; PERCEPTION; PHOTOSTIMULATION; PURE TONE AUDIOMETRY; SPEECH; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; TIME;

ACOUSTIC STIMULATION; ADOLESCENT; ANALYSIS OF VARIANCE; AUDIOMETRY, PURE-TONE; AUDITORY THRESHOLD; CUES; HUMANS; LIPREADING; NOISE; PERCEPTUAL MASKING; PHOTIC STIMULATION; SPEECH ACOUSTICS; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; TIME FACTORS; YOUNG ADULT;

EID: 79960660738     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.3592223     Document Type: Article

References (42)

1. American National Standards Institute (1995). ANSI S3.7-R2003: Methods for Coupler Calibration of Earphones, American National Standards Institute, NY.
2. American National Standards Institute (2004). ANSI S3.21-2004: Methods for Manual Pure-Tone Threshold Audiometry, American National Standards Institute, NY.
3. Arnal, L. H., Morillon, B., Kell, C. A., and Giraud, A.-L. (2009). Dual neural routing of visual facilitation in speech processing.. J. Neurosci. 29, 13445-13453. 10.1523/JNEUROSCI.3194-09.2009
4. Bernstein, L. E., Auer, E. T. J., and Takayanagi, S. (2004). Auditory speech detection in noise enhanced by lipreading., Speech Commun. 44, 5-18. 10.1016/j.specom.2004.10.011
5. Berthommier, F. (2003). A phonetically neutral model of the low-level audiovisual interaction., in Proceedings of the International Conference on Audio-Visual Speech Processing, 89-94 (Institut de la Communication Parlée, St. Jorioz, France).
6. Besle, J., Fischer, C., Bidet-Caulet, A., Lecaignard, F., Bertrand, O., and Giard, M.-H. (2008). Visual activation and audiovisual interactions in the auditory cortex during speech perception: intracranial recordings in humans.. J. Neurosci. 28, 14301-14310. 10.1523/JNEUROSCI.2875-08.2008
7. Brancazio, L. and Brancazio, L. (2004). Lexical influences in audiovisual speech perception.. J. Exp. Psychol. Hum. Percept. Perform. 30, 445-463. 10.1037/0096-1523.30.3.445 (Pubitemid 38726716)
8. Bregman, A. (1990). Auditory Scene Analysis: The Perceptual Organization of Sounds (MIT Press, Cambridge, MA), 1-773.
9. Bregman, A. S. (1978). Auditory streaming is cumulative.. J. Exp. Psychol. Hum. Percept. Perform. 4, 380-387. 10.1037/0096-1523.4.3.380
10. Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C., McGuire, P. K., Woodruff, P. W., Iversen, S. D., and David, A. S. (1997). Activation of auditory cortex during silent lipreading., Science 276, 593-596. 10.1126/science.276.5312.593 (Pubitemid 27193683)
11. Cusack, R., and Roberts, B. (2000). Effects of differences in timbre on sequential grouping., Percept. Psychophys. 62, 1112-1120. 10.3758/BF03212092
12. Devergie, A., Berthommier, F., and Grimault, N. (2009). Pairing audio speech and various visual displays: binding or not binding? in Proceedings of the International Conference on Audio-Visual Speech Processing (School of Computing Sciences, Norwich, UK), pp. 140-144.
13. Gaudrain, E., Grimault, N., Healy, E. W., and Béra, J.-C. (2007). Effect of spectral smearing on the perceptual segregation of vowel sequences., Hear. Res. 231, 32-41. 10.1016/j.heares.2007.05.001 (Pubitemid 47225010)
14. Gaudrain, E., Grimault, N., Healy, E. W., and Béra, J.-C. (2008). Streaming of vowel sequences based on fundamental frequency in a cochlear-implant simulation., J. Acoust. Soc. Am. 124, 3076-3087. 10.1121/1.2988289
15. Grant, K. W., and Braida, L. D. (1991). Evaluating the articulation index for auditory- visual input., J. Acoust. Soc. Am. 89, 2952-2960. 10.1121/1.400733
16. Grant, K. W., and Walden, B. E. (1996). Spectral distribution of prosodic information., J. Speech Hear. Res. 39, 228-238.
17. Grant, K. W., and Seitz, P. F. (2000). The use of visible speech cues for improving auditory detection of spoken sentences., J. Acoust. Soc. Am. 108, 1197-1208. 10.1121/1.1288668
18. Grant, K. W. (2001). The effect of speechreading on masked detection thresholds for filtered speech., J. Acoust. Soc. Am. 109, 2272-2275. 10.1121/1.1362687
19. Grant, K. W., van Wassenshove, V., and Poeppel, D. (2003). Discrimination of auditory-visual synchrony., in Proceedings of the International Conference on Audio-Visual Speech Processing (Institut de la Communication Parlée, St jorioz, France), pp. 31-35.
20. Grant, K. W., Wassenhove, V., and Poeppel, D. (2004). Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony., Speech Commun. 44, 43-53. 10.1016/j.specom.2004.06.004
21. Kawahara, H., Masuda-Katsuse, I., and de Cheveigné, A. (1999). Restructuring speech representations using a pitch-adaptive time-frequency- smoothing and an instantaneous-frequency-based f0 extraction: Possible role of a repetitive structure in sounds., Speech Commun. 27, 187-207. 10.1016/S0167-6393(98)00085-5
22. Kayser, C., Petkov, C. I., and Logothetis, N. K. (2008). Visual modulation of neurons in auditory cortex., Cereb. Cortex 18, 1560-1574. 10.1093/cercor/bhm187 (Pubitemid 351882035)
23. Klatt, D. (1980). Software for a cascade/parallel formant synthesizer., J. Acoust. Soc. Am. 67, 971-995. 10.1121/1.383940
24. Levitt, H. (1971). Transformed up-down methods in psychoacoustics., J. Acoust. Soc. Am. 49, 467-477. 10.1121/1.1912375
25. Massaro, D. W., and Cohen, M. M. (1983). Evaluation and integration of visual and auditory information in speech perception., J. Exp. Psychol. Hum. Percept. Perform. 9, 753-771. 10.1037/0096-1523.9.5.753
26. Micheyl, C. and Oxenham, A. J. (2010). Objective and subjective psychophysical measures of auditory stream integration and segregation., J. Assoc. Res. Otolaryngol., http://dx.doi.org/10.1007/s10162-010-0227-2 (date last viewed 10/27/10).
27. Micheyl, C., Tian, B., Carlyon, R. P., and Rauschecker, J. P. (2005). Perceptual organization of tone sequences in the auditory cortex of awake macaques., Neuron 48, 139-148. 10.1016/j.neuron.2005.08.039 (Pubitemid 41384171)
28. Miller, G. A., and Heise, G. A. (1950). The thrill threshold., J. Acoust. Soc. Am. 22, 637-638. 10.1121/1.1906663
29. Miller, L. M. and D'Esposito, M. (2005). Perceptual fusion and stimulus coincidence in the cross-modal integration of speech., J. Neurosci. 25, 5884-5893. 10.1523/JNEUROSCI.0896-05.2005 (Pubitemid 40911419)
30. Moettoenen, R., Krause, C. M., Tiipana, K., and Sams, M. (2002). Processing of changes in visual speech in the human auditory cortex., Cognit. Brain Res. 13, 417-425. 10.1016/S0926-6410(02)00053-8 (Pubitemid 34252971)
31. Moore, B. C. J., and Gockel, H. (2002). Factors influencing sequential stream segregation., Acta Acust. 88, 320-333. (Pubitemid 34732115)
32. Pekkola, J., Ojanen, V., Autti, T., Jskelinen, I. P., Mttnen, R., Tarkiainen, A., and Sams, M. (2005). Primary auditory cortex activation by visual speech: an fmri study at 3 t., Neuroreport 16, 125-128. 10.1097/00001756-200502080-00010 (Pubitemid 40250383)
33. Pressnitzer, D., Sayles, M., Micheyl, C., and Winter, I. M. (2008). Perceptual organization of sound begins in the auditory periphery., Curr. Biol. 18, 1124-1128. 10.1016/j.cub.2008.06.053
34. Rahne, T., Bckmann, M., von Specht, H., and Sussman, E. S. (2007). Visual cues can modulate integration and segregation of objects in auditory scene analysis., Brain. Res. 1144, 127-135. 10.1016/j.brainres.2007.01.074 (Pubitemid 46440839)
35. Rahne, T., and Bckmann-Barthel, M. (2009). Visual cues release the temporal coherence of auditory objects in auditory scene analysis., Brain. Res. 1300, 125-134. 10.1016/j.brainres.2009.08.086
36. Rahne, T., Deike, S., Selezneva, E., Brosch, M. Knig, R., Scheich, H., Bckmann, M., and Brechmann, A. (2008). A multilevel and cross-modal approach towards neuronal mechanisms of auditory streaming., Brain. Res. 1220, 118-131. 10.1016/j.brainres.2007.08.011 (Pubitemid 352004025)
37. Roberts, B., Glasberg, B. R., and Moore, B. C. J. (2002). Primitive stream segregation of tone sequences without differences in fundamental frequency or passband., J. Acoust. Soc. Am. 112, 2074-2085. 10.1121/1.1508784 (Pubitemid 35222474)
38. Schroeder, C. E., Lakatos, P., Kajikawa, Y., Partan, S., and Puce, A. (2008). Neuronal oscillations and visual amplification of speech., Trends Cogn. Sci. 12, 106-113. 10.1016/j.tics.2008.01.002
39. Stainsby, T. H., Moore, B. C., and Glasberg, B. R. (2004). Auditory streaming based on temporal structure in hearing-impaired listeners., Hear. Res. 192, 119-130. 10.1016/j.heares.2004.02.003 (Pubitemid 38639838)
40. Sumby, W. H., and Pollack, I. (1954). Visual contribution to speech intelligibility in noise., J. Acoust. Soc. Am. 26, 212-215. 10.1121/1.1907309
41. van Noorden, L. (1975). Temporal coherence in the perception of tone sequences., Ph.D. dissertation, Technische Hogeschool Eindhovern, Eindhoven, The Netherlands.
42. van Wassenhove, V., Grant, K., and Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech., PNAS 102, 1181-1186. 10.1073/pnas.0408949102 (Pubitemid 40170746)