Visual contribution to the multistable perception of speech

Perception and Psychophysics

Volumn 69, Issue 8, 2007, Pages 1360-1372

  Sato, Marc ^a,b   Basirat, Anahita ^a   Schwartz, Jean Luc ^a  

^a GIPSA LAB   (France)

^b UNIVERSITY OF PARMA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 36749026494     PISSN: 00315117     EISSN: None     Source Type: Journal    
DOI: 10.3758/BF03192952     Document Type: Article

References (50)

1. ABRY, C., CATHIARD, M.-A., VILAIN, A., LABOISSIÈRE, R., & SCHWARTZ, J.-L. (2006). Some insights in bimodal perception given for free by the natural time course of speech production. In G. Bailly, P. Perrier, & E. Vatikiotis-Bateson (Eds.), Festschrift Christian Benoît. Cambridge, MA: MIT Press.
2. ABRY, C., SATO, M., SCHWARTZ, J.-L., LŒVENBRUCK, H., & CATHIARD, M.-A. (2003). Attention-based maintenance of speech forms in memory: The case of verbal transformations. Behavioral & Brain Sciences, 26, 728-729.
3. BENOÎT, C., MOHAMADI, T., & KANDEL, S. D. (1994). Effects of phonetic context on audio-visual intelligibility of French. Journal of Speech & Hearing Research, 37, 1195-1203.
4. BERNSTEIN, L. E., AUER, E. T., JR., & MOORE, J. K. (2004). Audio-visual speech binding: Convergence or association. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), Handbook of multisensory processes (pp. 203-224). Cambridge, MA: MIT Press.
5. BERNSTEIN, L. [E.], BURNHAM, D., & SCHWARTZ, J.-L. (2002). Special session: Issues in audiovisual spoken language processing (when, where, and how?). In Proceedings of the International Conference on Spoken Language Processing 2002 (pp. 1445-1448). Denver.
6. BERNSTEIN, L. E., DEMOREST, M. E., & TUCKER, P. E. (1998). What makes a good speechreader? First you have to find one. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 211-228). Hove, U.K.: Psychology Press.
7. CALLAN, D. E., JONES, J. A., MUNHALL, K., CALLAN, M. A., KROOS, C., & VATIKIOTIS- BATESON, E. (2003). Neural processes underlying perceptual enhancement by visual speech gestures. NeuroReport, 14, 2213-2218.
8. COSMELLI, D., DAVID, O., LACHAUX, J. P., MARTINERIE, J., GARNERO, L., RENAULT, B., & VARELA, F. (2004). Waves of consciousness: Ongoing cortical patterns during binocular rivalry. Neurolmage, 23, 128-140.
9. DAVIS, C., & KIM, J. (1998). Repeating and remembering foreign language words: Does seeing help? Proceedings of Audio-Visual Speech Processing'98 (pp. 121-125). Sydney.
10. DOHEN, M., LŒVENBRUCK, H., CATHIARD, M.-A., & SCHWARTZ, J.-L. (2004). Audiovisual perception of contrastive focus in French. Speech Communication, 44, 155-172.
11. DUFOUR, S., PEEREMAN, R., PALLIER, C., & RADEAU, M. (2002). VoCoLex: A lexical database on phonological similarity between French words. L'Année Psychologique, 102, 725-746.
12. FOWLER, C. A., & ROSENBLUM, L. D. (1991). The perception of phonetic gestures. In I. G. Mattingly & M. Studdert-Kennedy (Eds.), Modularity and the motor theory of speech perception (pp. 33-59). Hillsdale, NJ: Erlbaum.
13. GREEN, K. P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 3-25). Hove, U.K.: Psychology Press.
14. HICKOK, G., & POEPPEL, D. (2000). Toward a functional neuroanatomy of speech perception. Trends in Cognitive Science, 4, 131-138.
15. HICKOK, G., & POEPPEL, D. (2004). Dorsal and ventral streams: A framework for understanding aspects of the functional anatomy of language. Cognition, 92, 67-99.
16. LALLOUACHE, M. T. (1990). Un poste "visage-parole." Acquisition et traitement de contours labiaux. Actes des XVIIIèmes Journées d'Études sur la Parole (pp. 282-286), Montréal, Québec, Canada.
17. LEOPOLD, D. A., & LOGOTHETIS, N. K. (1999). Multistable phenomena: Changing views in perception. Trends in Cognitive Sciences, 3, 254-264.
18. LEOPOLD, D. A., WILKE, M., MAIER, A., & LOGOTHETIS, N. K. (2002). Stable perception of visually ambiguous patterns. Nature Neuroscience, 5, 605-609.
19. LIBERMAN, A. M., & MATTINGLY, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1-36.
20. LUCE, P. A., PISONI, D. B., & GOLDINGER, S. D. (1990). Similarity neighborhoods of spoken words. In G. T. M. Altman (Ed.), Cognitive models of speech processing: Psycholinguistic and computational perspectives (pp. 122-147). Cambridge, MA: MIT Press.
21. MACKAY, D. G., WULF, G., YIN, C., & ABRAMS, L. (1993). Relations J between word perception and production: New theory and data on the verbal transformation effect. Journal of Memory & Language, 32, 624-646.
22. MACLEOD, A., & SUMMERFIELD, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21, 131-141.
23. MARSLEN-WILSON, W. D. (1987). Functional parallelism in spoken word recognition. Cognition, 25, 71-102.
24. MASSARO, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum.
25. MASSARO, D. W. (1989). A fuzzy logical model of speech perception. In D. Vickers & P. L. Smith (Eds.), Human information processing: measures, mechanisms, and models (pp. 367-379). Amsterdam: North-Holland.
26. MCGURK, H., & MACDONALD, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
27. OJANEN, V., MÖTTÖNEN, R., PEKKOLA, J., JÄÄSKELÄINEN, I. P., JOENSUU, R., AUTTI, T., & SAMS, M. (2005). Processing of audiovisual speech in Broca's area. NeuroImage, 25, 333-338.
28. PEKKOLA, J., LAASONEN, M., OJANEN, V., AUTTI, T., JÄÄSKELÄINEN, I. P., KUJALA, T., & SAMS, M. (2006). Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: An fMRI study at 3T. NeuroImage, 29, 797-807.
29. PITT, M., & SHOAF, L. (2001). The source of a lexical bias in the Verbal Transformation Effect. Language & Cognitive Processes, 16, 715-721.
30. PITT, M., & SHOAF, L. (2002). Linking verbal transformations to their causes. Journal of Experimental Psychology: Human Perception & Performance, 28, 150-162.
31. PORTER, R. J., JR., & CASTELLANOS, F. X. (1980). Speech-production measures of speech perception: Rapid shadowing of VCV syllables. Journal of the Acoustical Society of America, 67, 1349-1356.
32. PORTER, R. J., JR., & LUBKER, J. F. (1980). Rapid reproduction of vowel-vowel sequences: Evidence for a fast and direct acoustic-motoric link-age in speech. Journal of Speech & Hearing Research, 23, 593-602.
33. REISBERG, D., MCLEAN, J., & GOLDFIELD, A. (1987). Easy to hear but hard to understand: A lipreading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lipreading (pp. 97-113). London: Erlbaum.
34. REISBERG, D., SMITH, J. D., BAXTER, A. D., & SONENSHINE, M. (1989). "Enacted" auditory images are ambiguous; "pure" auditory images are not. Quarterly Journal of Experimental Psychology, 41A, 619-641.
35. ROBERT-RIBES, J., SCHWARTZ, J.-L., LALLOUACHE, T., & ESCUDIER, P. (1998). Complementarity and synergy in bimodal speech: Auditory, visual and audiovisual identification of French oral vowels. Journal of the Acoustical Society of America, 103, 3677-3689.
36. SAMS, M., MÖTTÖNEN, R., & SIHVONEN, T. (2005). Seeing and hearing others and oneself talk. Cognitive Brain Research, 23, 429-435.
37. SATO, M., BACIU, M., LŒVENBRUCK, H., SCHWARTZ, J.-L., CATHIARD, M.-A., SEGEBARTH, C., & ABRY, C. (2004). Multistable representation of speech forms: An fMRI study of verbal transformations. Neuro-Image, 23, 1143-1151.
38. SATO, M., SCHWARTZ, J.-L., ABRY, C., CATHIARD, M.-A., & LŒVENBRUCK, H. (2006). Multistable syllables as enacted percept: A source of an asymmetric bias in the verbal transformation effect. Perception & Psychophysics, 68, 458-474.
39. SCHWARTZ, J.-L., ABRY, C., BOE, L. J., & CATHIARD, M.-A. (2002). Phonology in a theory of perception-for-action- control. In J. Durand & B. Laks (Eds.), Phonetics, phonology, and cognition (pp. 254-280). Oxford: Oxford University Press.
40. SCHWARTZ, J.-L., BOË, L. J., & ABRY, C. (2006). Linking dispersion-focalization theory and the maximum utilization of the available distinctive features principle in a perception-for-action-control theory. In M. J. Solé, P. S. Beddor, & M. Ohala (Eds.), Experimental approaches to phonology (pp. 104-124). Oxford: Oxford University Press.
41. SCHWARTZ, J.-L., ROBERT-RIBES, J., & ESCUDIER, P. (1998). Ten years after Summerfield: A taxonomy of models for audio-visual fusion in speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 85-108). Hove, U. K.: Psychology Press.
42. SHOAF, L., & PITT, M. (2002). Does node stability underlie the verbal transformation effect? A test of node structure theory. Perception & Psychophysics, 64, 795-803.
43. SKIPPER, J. I., NUSBAUM, H. C., & SMALL, S. L. (2005). Listening to talking faces: Motor cortical activation during speech perception. NeuroImage, 25, 76-89.
44. SKIPPER, J. I., VAN WASSENHOVE, V., NUSBAUM, H. C., & SMALL, S. L. (2007). Hearing lips and seeing voices: How cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex, 17, 2387-2399.
45. SMITH, J. D., REISBERG, D., & WILSON, M. (1995). The role of subvocalization in auditory imagery. Neuropsychologia, 11, 1433-1454.
46. SUMBY, W. H., & POLLACK, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
47. SUMMERFIELD, Q. (1983). Audio-visual speech perception, lipreading and artificial stimulation. In M. E. Lutman & M. P. Haggard (Eds.), Hearing science and hearing disorders (pp. 131-182). London: Academic Press.
48. SUMMERFIELD, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lipreading (pp. 3-51). London: Erlbaum.
49. WARREN, M. R. (1961). Illusory changes of distinct speech upon repetition: The verbal transformation effect. British Journal of Psychology, 52, 249-258.
50. WARREN, M. R., & GREGORY, R. L. (1958). An auditory analogue of the visual reversible figure. American Journal of Psychology, 71, 612-613.