Top-down and bottom-up modulation of audiovisual integration in speech

European Journal of Cognitive Psychology

Volumn 17, Issue 4, 2005, Pages 541-560

  Colin, Cécile ^a   Radeau, M ^a   Deltenre, P ^a  

^a UNIVERSITÉ LIBRE DE BRUXELLES   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

ADOLESCENT; ADULT; ANALYSIS OF VARIANCE; ARTICLE; AUDIOVISUAL EQUIPMENT; COGNITION; CONTROLLED STUDY; DATA ANALYSIS; EXPERIMENTATION; FEMALE; HUMAN; HUMAN EXPERIMENT; INTEGRATION; MALE; METHODOLOGY; NORMAL HUMAN; PRIORITY JOURNAL; RESEARCH; SPEECH; STIMULUS RESPONSE; TECHNIQUE; VISION;

EID: 23244459876     PISSN: 09541446     EISSN: None     Source Type: Journal    
DOI: 10.1080/09541440440000168     Document Type: Article

References (61)

1. Amano, J., & Sekiyama, K. (1998). The McGurk effect is influenced by the stimulus set size. In D. Burnham, J. Robert-Ribès, & E. Vatikiotis-Bateson (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 43-48). Terrigal, Australia.
2. Benoît, C., Mohamadi, T., & Kandel, S. (1994). Effects of phonetic context on audio-visual intelligibility of speech. Journal of Speech and Hearing Research, 37, 1195-1203.
3. Bertelson, P., Vroomen, J., Wiegeraad, G., & de Gelder, B. (1994). Exploring the relation between McGurk interference and ventriloquism. In Proceedings of the International Conference on Spoken Language Processing (pp. 559-562). Yokohama, Japan.
4. Brandy, W. T. (2002). Speech audiometry. In J. Katz (Ed.), Handbook of clinical audiology (pp. 96-110). Baltimore, MD: Lippincott, Williams & Wilkins.
5. Burnham, D. (1998). Language specificity in the development of auditory-visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II (pp. 27-60). Hove, UK: Psychology Press.
6. Cathiard, M.-A., Schwartz, J.-L., & Abry, C. (2001). Asking a naïve question about the McGurk effect: Why does audio [b] give more [d] percepts with visual [g] than with visual [d]? In D. W. Massaro, J. Light, & K. Geraci (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 138-142). Aalborg, Denmark.
7. Cerrato, L., Leoni, F. A., & Falcone, M. (1998). Is it possible to evaluate the contribution of visual information to the process of speech comprehension? In D. Burnham, J. Robert-Ribès, & E. Vatikiotis-Bateson (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 141-146). Terrigal, Australia.
8. Colin, C. (2001). Etude comportementale et électrophysiologique des processus impliqués dans l'effet McGurk et dans l'effet de ventriloquie. Unpublished doctoral thesis, Free University of Brussels, Belgium.
9. Colin, C., & Radeau, M. (2003). Les illusions McGurk dans la parole: 25 ans de recherches. L'Année Psychologique, 114, 497-542.
10. Colin, C., Radeau, M., Deltenre, P., Demolin, D., & Soquet, A. (2002). The role of sound intensity and stop-consonant voicing on McGurk fusions and combinations. European Journal of Cognitive Psychology, 14, 475-491.
11. Colin, C., Radeau, M., Deltenre, P., & Morais, J. (2001). Rules of intersensory integration in spatial scene analysis and speechreading. Psychologica Belgica, 41, 131-144.
12. Davis, C., & Kim, J. (1998). Repeating and remembering foreign language words: Does seeing help? In D. Burnham, J. Robert-Ribès, & E. Vatikiotis-Bateson (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 121-126). Terrigal, Australia.
13. Dekle, D. J., Fowler, C. A., & Funnell, M. G. (1992). Audiovisual integration in perception of real words. Perception and Psychophysics, 51, 355-362.
14. Demorest, M., & Bernstein, L. (1992). Sources of variability in speechreading sentences: A generalizability analysis. Journal of Speech and Hearing Research, 35, 876-891.
15. Dodd, B., McIntosh, B., & Woodhouse, L. (1998). Early lipreading ability and speech and language development of hearing-impaired pre-schoolers. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II (pp. 229-242). Hove, UK: Psychology Press.
16. Easton, R. D., & Basala, M. (1982). Perceptual dominance during lipreading. Perception and Psychophysics, 32, 562-570.
17. Erber, N. P. (1969). Interaction of audition and vision in the recognition of oral speech stimuli. Journal of Speech and Hearing Research, 12, 423-425.
18. Fixmer, E., & Hawkins, S. (1998). The influence of quality of information on the McGurk effect. In D. Burnham, J. Robert-Ribès, & E. Vatikiotis-Bateson (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 27-32). Terrigal, Australia.
19. Fuster-Duran, A. (1996). Perception of conflicting audio-visual speech: An examination across Spanish and German. In D. G. Stork & M. E. Hennecke (Eds.), Speechreading by humans and machines (pp. 135-143; NATO ASI Series F: Computer and systems sciences, No. 150). Berlin, Germany: Springer-Verlag.
20. Green, K. P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II (pp. 3-25). Hove, UK: Psychology Press.
21. Green, K. P., Kuhl, P. K., Meltzoff, A. N., & Stevens, E. B. (1991). Integrating speech information across talkers, gender and sensory modality: Female faces and male voices in the McGurk effect. Perception and Psychophysics, 50, 524-536.
22. Hardison, D. B. (1996). Bimodal perception by native and nonnative speakers of English: Factors influencing the McGurk effect. Language Learning, 46, 3-73.
23. Hietanen, J. K., Manninen, P., Sams, M., & Surakka, V. (2001). Does audiovisual speech perception use information about facial configuration? European Journal of Cognitive Psychology, 13, 395-407.
24. Jones, J. A., & Munhall, K. G. (1997). The effects of separating auditory and visual sources on audiovisual integration of speech. Canadian Acoustics, 2, 13-19.
25. Jordan, T. R., & Bevan, K. (1997). Seeing and hearing rotated faces: Influences of facial orientation on visual and audiovisual speech recognition. Journal of Experimental Psychology: Human Perception and Performance, 25, 388-403.
26. Jordan, T. R., & Sergeant, P. C. (1998). Effects of facial image size on visual and audio-visual speech recognition. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II (pp. 155-176). Hove, UK: Psychology Press.
27. Jordan, T. R., & Sergeant, P. C. (2000). Effects of distance on visual and audiovisual speech recognition. Language and Speech, 43, 107-124.
28. Jordan, T. R., & Thomas, S. M. (2001). Effects of horizontal viewing angle on visual and audiovisual speech recognition. Journal of Experimental Psychology: Human Perception and Performance, 27, 1386-1403.
29. Kuhl, P., Green, K. P., & Meltzoff, A. N. (1988). Factors affecting the integration of auditory and visual information in speech: The level effect. Journal of the American Society of America, 83(Suppl. 1), S86.
30. Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431-461.
31. Liberman, A. M., & Mattingly, I. G. (1985). The motor-theory of speech revised. Cognition, 21, 1-36.
32. MacDonald, J., Andersen, S., & Bachmann, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception, 29, 1155-1168.
33. MacLeod, A., & Summerfield, Q. (1990). A procedure for measuring auditory and audio-visuals speech-reception thresholds for sentences in noise: Rationale, evaluation, and recommendations for use. British Journal of Audiology, 24, 29-43.
34. Massaro, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
35. Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
36. Massaro, D. W., & Cohen, M. M. (1996). Perceiving speech from inverted faces. Perception and Psychophysics, 58, 1047-1065.
37. McGurk, H. (1988). Developmental psychology and the vision of speech. Inaugural professorial lecture, University of Surrey, Guildford, UK.
38. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
39. Middleweerd, M. J., & Plomp, R. (1987). The effects of speechreading on the speech perception threshold of sentences in noise. Journal of the Acoustical Society of America, 82, 2145-2146.
40. Miller, G. A., & Nicely, P. E. (1955). An analysis of perceptual confusions among some English consonants. Journal of the Acoustical Society of America, 27, 338-352.
41. Mills, A. E., & Thiem, R. (1980). Auditory-visual fusions and illusions in speech perception. Linguistische Berichte, 68, 85-107.
42. Mohamadi, T., & Benoît, C. (1992). Apport de la vision du locuteur à l'intelligibilité de la parole bruitée en français. Bulletin de la Communication Parlée, 2, 31-41.
43. Nelson, M. A., & Hodge, M. M. (2000). Effects of facial paralysis and audiovisual information on stop place identification. Journal of Speech, Language and Hearing Research, 43, 158-171.
44. Posner, M., & Snyder, C. (1975). Attention and cognitive control. In R. L. Solso (Ed.), Information processing and cognition: The Loyola symposium (pp. 55-85). Hillsdale, NJ. Lawrence Erlbaum Associates, Inc.
45. Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97-113). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
46. Repp, B. H., Manuel, S. Y., Liberman, A. M., & Studdert-Kennedy, M. (1983). Exploring the McGurk effect. In Proceedings of the 24th Annual Meeting of the Psychonomic Society (pp. 366-367). San Diego, CA.
47. Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-331.
48. Rosenblum, L. D., Yakel, D. A., & Green, K. P. (2000). Face and mouth inversion effects on visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 26, 806-819.
49. Sahley, T. L., Nodar, R. H., & Musiek, F. E. (1997). Efferent auditory system: Structure and function. San Diego, CA: Singular Publishing Group.
50. Sams, M., Manninen, P., Surakka, V., Helin, P., & Kättö, R. (1998). McGurk effect in Finnish syllables, isolated words, and words in sentences: Effects of word meaning and sentence context. Speech Communication, 26, 75-87.
51. Schwartz, J.-L. (2003). Why the FLMP should not be applied to McGurk data ... or how to better compare models in the Bayesian framework. In J.-L. Schwartz, F. Berthommier, M. A. Cathiard, & D. Sodoyer (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 77-82). St Jorioz, France.
52. Sekiyama, K. (1998). Face or voice? Determinant of compellingness to the McGurk effect. In D. Burnham, J. Robert-Ribès, & E. Vatikiotis-Bateson (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 33-36). Terrigal, Australia.
53. Sekiyama, K., & Tohkura, Y. (1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America, 90, 1797-1805.
54. Sekiyama, K., & Tohkura, Y. (1993). Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics, 21, 427-444.
55. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
56. Summerfield, Q., & McGrath, M. (1984). Detection and resolution of audio-visual incompatibility in the perception of vowels. Quarterly Journal of Experimental Psychology, 36A, 51-74.
57. Tillman, H. G., Pompino-Marschall, B., & Porzig, U. (1984). The effects of visually presented speech movements on the perception of acoustically encoded speech articulation as a function of acoustic desynchronization. in Proceedings of the 10th International Congress of Phonetic Sciences (pp. 469-473). Dordrecht, Holland.
58. Tiippana, K., Sams, M., & Andersen, T.S. (2001). Visual attention influences audiovisual speech perception. In D. W. Massaro, J. Light, & K. Geraci (Eds.), Proceedings of the Auditory-Visual Speech Processing Conference (pp. 167-171). Aalborg, Denmark.
59. Traill, A. (1999). The McGurk effect and !Xóõ clicks. In Proceedings of the International Conference of Phonetic Sciences (pp. 1933-1935). San Francisco, CA.
60. Vroomen, J., & de Gelder, D. (1999). Crossmodal integration: A good fit is no criterion. Trends in Cognitive Science, 4, 37-38.
61. Walker, S., Bruce, V., & O'Malley, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception and Psychophysics, 57, 1124-1133.