Discriminating languages by speech-reading

Perception and Psychophysics

Volumn 69, Issue 2, 2007, Pages 218-231

  Soto Faraco, Salvador ^a,d   Navarra, Jordi ^a   Weikum, Whitnev M ^b   Vouloumanos, Athena ^c   Sebastián Gallés, Núria ^a   Werker, Janet F ^b  

^a UNIVERSITY OF BARCELONA   (Spain)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c MCGILL UNIVERSITY   (Canada)

^d HOSPITAL SANT JOAN DE DÉU   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34249946485     PISSN: 00315117     EISSN: None     Source Type: Journal    
DOI: 10.3758/BF03193744     Document Type: Article

References (77)

1. AUER, E. T., JR. (2002). The influence of the lexicon on speech read word recognition: Contrasting segmental and lexical distinctiveness. Psychonomic Bulletin & Review, 9, 341-347.
2. AUER, E. T., JR., & BERNSTEIN, L. E. (1997). Speechreading and the structure of the lexicon: Computationally modeling the effects of reduced phonetic distinctiveness on lexical uniqueness. Journal of the Acoustical Society of America, 102, 3704-3710.
3. BERGER, K. W. (1972). Visemes and homophenous words. Teacher of the Deaf, 70, 396-399.
4. BERNSTEIN, L. E., AUER, E. T., JR., & TUCKER, P. E. (2001). Enhanced speechreading in deaf adults: Can short-term training/practice close the gap for hearing adults? Journal of Speech, Language, & Hearing Research, 44, 5-18.
5. BERNSTEIN, L. [E.], & BENOÎT, C. (1996). For speech perception three senses are better than one. In Proceedings of the 4th International Conference on Spoken Language Processing (ICSLP 96) (Vol. 3, pp. 1477-1480). New York: IEEE Press.
6. BERNSTEIN, L. E., DEMOREST, M. E., & TUCKER, P. E. (1998). What makes a good speechreader? First you have to find one. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 211-227). Hove, U.K.: Psychology Press.
7. BERNSTEIN, L. E., DEMOREST, M. E., & TUCKER, P. E. (2000). Speech perception without hearing. Perception & Psychophysics, 62, 233-252.
8. BERNSTEIN, L. E., IVERSON, P., & AUER, E. T., JR. (1997). Elucidating the complex relationships between phonetic perception and word recognition in audiovisual speech perception. In C. Benoît & R. Campbell (Eds.), Proceedings of the ESCA/ESCOP Workshop on Audio-Visual Speech Processing (pp. 89-92).
9. BOSCH, L., & SEBASTIÁN- GALLÉS, N. (1997). Native-language recognition abilities in 4-month-old infants from monolingual and bilingual environments. Cognition, 65, 33-69.
10. BOSCH, L., & SEBASTIÁN- GALLÉS, N. (2001). Evidence of early language discrimination abilities in infants from bilingual environments. Infancy, 2, 29-49.
11. BURNHAM, D., & DODD, B. (2004). Auditory-visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect. Developmental Psychobiology, 45, 204-220.
12. CALLAN, D. E., JONES, J. A., MUNHALL, K., CALLAN, A. M., KKOOS, C., & VATIKIOTIS- BATESON, E. (2003). Neural processes underlying perceptual enhancement by visual speech gestures. NeuroReport, 14, 2213-2218.
13. CALVERT, G. A., BRAMMER, M. J., BULLMORE, E. T., CAMPBELL, R., IVERSEN, S. D., & DAVID, A. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding. NeuroReport, 10, 2619-2623.
14. CALVERT, G. A., BULLMORE, E. T., BRAMMER, M. J., CAMPBELL, R., WILLIAMS, S. C., MCGUIRE, P. K., ET AL. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593-596.
15. CALVERT, G. A., SPENCE, C., & STEIN, B. E. (EDS.) (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.
16. CAMPBELL, R., DODD, B., & BURNHAM, D. (EDS.) (1998). Hearing by eye II: Advances in the psychology of speechreading and auditory -visual speech. Hove, U.K.: Psychology Press.
17. CONRAD, R. (1977). Lipreading by deaf and hearing children. British Journal of Educational Psychology, 47, 60-65.
18. DESJARDINS, R. N., ROGERS, J., & WERKER, J. F. (1997). An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks. Journal of Experimental Child Psychology, 66, 85-110.
19. DESIARDINS, R. N., & WERKER, J. F. (2004). Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology, 45, 187-203.
20. DODD, B. (1979). Lip reading in infants: Attention to speech presented in- and out-of-synchrony. Cognitive Psychology, 11, 478-484.
21. DODD, B., & CAMPBELL, R. (EDS.) (1987). Hearing by eye: Thepsychology of lip-reading. London: Erlbaum.
22. DODD, B., & MURPHY, J. (1992). Visual thoughts. In R. Campbell (Ed.), Mental lives: Case studies in cognition (pp. 47-60). Oxford: Blackwell.
23. DUPOUX, E., PALLIER, C., SEBASTIAN, N., & MEHLER, J. (1997). A destressing "deafness" in French? Journal of Memory & Language, 36, 406-421.
24. DUPOUX, E., PEPERKAMP, S., & SEBASTIÁ N-GALLÉS, N. (2001). A robust method to study stress "deafness." Journal of the Acoustical Society of America, 110, 1606-1618.
25. FISHER, C. G. (1968). Confusions among visually perceived consonants. Journal of Speech & Hearing Research, 11, 769-804.
26. FOWLER, C. A. (1996). Listeners do hear sounds, not tongues. Journal of the Acoustical Society of America, 99, 1730-1741.
27. HADAR, U., STEINER, T. J., GRANT, E. C., & ROSE, F. C. (1983). Head movement correlates of juncture and stress at sentence level. Language & Speech, 26, 117-129.
28. HADAR, U., STEINER, T. J., GRANT, E. C.,&ROSE, F. C. (1984). The timing of shifts in head posture during conversation. Human Movement Science, 3, 237-245.
29. HEIDER, F., & HEIDER, G. M. (1940). An experimental investigation of lipreading. Psychological Monographs, 52, 124-153.
30. KAMACHI, M., HILL, H., LANDER, K., & VATIKIOTIS-BATESON, E. (2003). "Putting the face to the voice": Matching identity across modality. Current Biology, 13, 1709-1714.
31. KUHL, P. K., & MELTZOFF, A. N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138-1141.
32. KUHL, P. K., & MELTZOFF, A. N. (1984). The intermodal representation of speech in infants. Infant Behavior & Development, 7, 361-381.
33. LIBERMAN, A. M., & MATTINGLY, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1-36.
34. LYXELL, B., & RÖNNBERG, J. (1991). Word discrimination and chronological age related to sentence-based speech-reading skill. British Journal of Audiology, 25, 3-10.
35. MACEACHERN, M. R. (2000). On the visual distinctiveness of words in the English lexicon. Journal of Phonetics, 28, 367-376.
36. MACMILLAN, N. A., & CREELMAN, C. D. (1991). Detection theory: A user s guide. Cambridge: Cambridge University Press.
37. MASSARO, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
38. MASSARO, D. W., COHEN, M. M., & GESI, A. T. (1993). Long-term training, transfer, and retention in learning to lipread. Perception & Psychophysics, 53, 549-562.
39. MATTYS, S. L., BERNSTEIN, L. E., & AUER, E. T., JR. (2002). Stimulusbased lexical distinctiveness as a general word-recognition mechanism. Perception & Psychophysics, 64, 667-679.
40. MCGURK, H., & MACDONALD, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
41. MEHLER, J., JUSCZYK, P., LAMBERTZ, G., HALSTED, N., BERTONCINI, J., & AMIEL- TISON, C. (1988). A precursor of language acquisition in young infants. Cognition, 29, 143-178.
42. MOGFORD, K. (1987). Lip-reading in the prelingually deaf. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 191-211). London: Erlbaum.
43. MUNHALL, K., JONES, J. A., CALLAN, D. E., KURATATE, T., & VATIKIOTIS-BATESON, E. (2004). Head movement improves auditory speech perception. Psychological Science, 15, 133-137.
44. NAVARRA, J., SEBASTIAN-GALLES, N., & SOTO-FARACO, S. (2005). The perception of second language sounds in early bilinguals: New evidence from an implicit measure. Journal of Experimental Psychology: Human Perception & Performance, 31, 912.
45. NAVARRA, J., & SOTO-FARACO, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71, 4-12.
46. NAVARRA, J., SPENCE, C., & SOTO- FARACO, S. (2007). Visual discrimination of rhythm in speech. Manuscript submitted for publication.
47. NAZZI, T., BERTONCINI, J., & MEHLER, J. (1998). Language discrimination by newborns: Toward an understanding of the role of rhythm. Journal of Experimental Psychology: Human Perception & Performance, 24, 756-766.
48. NAZZI, T., JUSCZYK, P. W., & JOHNSON, E. K. (2000). Language discrimination by English-learning 5-month-olds: Effects of rhythm and familiarity. Journal of Memory & Language, 43, 1-19.
49. NITCHIE, E. B. (1916). The use of homophenous words. Volta Review, 18, 85-93.
50. OWENS, E., & BLAZER, B. (1985). Visemes observed by hearing-impaired and normal-hearing adult viewers. Journal of Speech & Hearing Research, 28, 381-393.
51. PATTERSON, M. L., & WERKER, J. F. (1999). Matching phonetic information in lips and voice is robust in 4.5-month-old infants. Infant Behavior & Development, 22, 237-247.
52. PATTERSON, M. L., & WERKER, J. F. (2002). Infants' ability to match dynamic phonetic and gender information in the face and voice. Journal of Experimental Child Psychology, 81, 93-115.
53. RAMUS, F. (2002). Language discrimination by newborns. Annual Review of Language Acquisition, 2, 85-115.
54. RAMUS, F., HAUSER, M. D., MILLER, C., MORRIS, D., & MEHLER, J. (2000). Language discrimination by human newborns and by cottontop tamarin monkeys. Science, 288, 349-351.
55. RAMUS, F., & MEHLER, J. (1999). Language identification with suprasegmental cues: A study based on speech resynthesis. Journal of the Acoustical Society of America, 105, 512-521.
56. RAMUS, F., NESPOR, M., & MEHLER, J. (1999). Correlates of linguistic rhythm in the speech signal. Cognition, 73, 265-292.
57. REISBERG, D., MCLEAN, J., & GOLDFIELD, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97-113). London: Erlbaum.
58. RÖNNBERG, J., SAMUELSSON, S., & LYXELL, B. (1998). Conceptual constraints in sentence-based lipreading in the hearing-impaired. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 143-153). Hove, U.K.: Psychology Press.
59. SACKS, O. (1990). Seeing voices: A journey into the world of the deaf. New York: HarperPerermial.
60. SAMS, M., AULANKO, R., HAMALAINEN, M., HARI, R., LOUNASMAA, O. V., LU, S. T., & SIMOLA, J. (1991). Seeing speech: Visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters, 127, 141-145.
61. SAMUELSSON, S., & RÖNNBERG, J. (1993). Implicit and explicit use of scripted constraints in lip-reading. European Journal of Cognitive Psychology, 5, 201-233.
62. SEBASTIÁN-GALLÉS, N., DUPOUX, E., COSTA, A., & MEHLER, J. (2000). Adaptation to time-compressed speech: Phonological determinants. Perception & Psychophysics, 62, 834-842.
63. SEBASTIÁN-GALLÉS, N., ECHEVERRÍA, S., & BOSCH, L. (2005). The influence of initial exposure on lexical representation: Comparing early and simultaneous bilinguals. Journal of Memory & Language, 52, 240-255.
64. SEBASTIÁN-GALLÉS, N., & SOTO-FARACO, S. (1999). Online processing of native and non-native phonemic contrasts in early bilinguals. Cognition, 72, 111-123.
65. SOLÀ, J., LLORET, M. R., MASCARÓ , J., & PÉREZ SALDANYA, M. (2000). Gramàtica del català contemporani: Volwn I. Introducció: Fonètica ifonologia. Morfologia [Grammar of contemporary Catalan: Vol. 1. Introduction: Phonetics and phonology. Morphology]. Barcelona: Editorial Empúries.
66. SOTO-FARACO, S., NAVARRA, J., & ALSIUS, A. (2004). Assessing automaticity in audiovisual speech integration: Evidence from the speeded classification task. Cognition, 92, B13-B23.
67. SOTO-FARACO, S., SEBASTIÁN- GALLÉS, N., & CUTLER, A. (2001). Segmentai and suprasegmental mismatch in lexical access. Journal of Memory & Language, 45, 412-432.
68. SUMBY, W. H., & POLLACK, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
69. SUMMERFIELD, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). London: Erlbaum.
70. SUMMERFIELD, Q., MACLEOD, A., MCGRATH, M., & BROOKE, M. (1989). Lips, teeth, and the benefits of lipreading. In A. W. Young & H. D. Ellis (Eds.), Handbook of research on face processing (pp. 223-233). Amsterdam: North-Holland.
71. SUOMI, K., MCQUEEN, J. M., & CUTLER, A. (1997). Vowel harmony and speech segmentation in Finnish. Journal of Memory & Language, 36, 422-444.
72. VATIKIOTIS-BATESON, E., MUNHALL, K. G., KASAHARA, Y., GARCIA, F., & YEHIA, H. (1996). Characterizing audiovisual information during speech. In Proceedings of the 4th International Conference on Spoken Language Processing (ICSLP 96) (Vol. 3, pp. 1485-1488). New York: IEEE Press. Available at www.asel.udel.edu/icslp/ cdrom/vol3/1004/a1004.pdf.
73. WALDEN, B. E., ERDMAN, S. A., MONTGOMERY, A. A., SCHWARTZ, D. M., & PROSEK, R. A. (1981). Some effects of training on speech recognition by hearing-impaired adults. Journal of Speech & Hearing Research, 24, 207-216.
74. WALDEN, B. E., PROSEK, R. A., MONTGOMERY, A. A., SCHERR, C. K., & JONES, C. J. (1977). Effects of training on the visual recognition of consonants. Journal of Speech & Hearing Research, 20, 130-145.
75. WEIKUM,W. M., WERKER, J. F.,VOULOUMANOS. A., NAVARRA-ORDOÑO, J., SOTO-FARACO, S., & SEBASTIÁN-GALLÉS, N. (2004, November). When can infants discriminate languages using only visual speech information? Poster presented at the 29th Boston University Conference on Child Development, Boston.
76. YEHIA, H. C., KURATATE, T., & VATIKIOTIS- BATESON, E. (2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 30, 555-568.
77. ZATORRE, R. J. (2001). Do you see what I'm saying? Interactions between auditory and visual cortices in cochlear implant users. Neuron, 31, 13-14.