Language identification from visual-only speech signals

Attention, Perception, and Psychophysics

Volumn 72, Issue 6, 2010, Pages 1601-1613

  Ronquest, Rebecca E ^a   Levi, Susannah V ^b   Pisoni, David B ^a  

^a INDIANA UNIVERSITY   (United States)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ASSOCIATION; HUMAN; LANGUAGE; LINGUISTICS; LIP READING; PHONATION; PHONETICS; SEMANTICS; SPEECH PERCEPTION;

CUES; HUMANS; LIPREADING; MULTILINGUALISM; PHONATION; PHONETICS; PSYCHOLINGUISTICS; SEMANTICS; SPEECH PERCEPTION;

EID: 77957652068     PISSN: 19433921     EISSN: None     Source Type: Journal    
DOI: 10.3758/APP.72.6.1601     Document Type: Article

References (35)

1. Bernstein, L.E., Demorest, M.E., & Tucker, P.E. (1998). What makes a good speechreader? First you have to find one. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 211-227). East Sussex, U.K.: Psychology Press.
2. Bernstein, L.E., Eberhardt, S.P., & Demorest, M.E. (1986). Judgments of intonation and contrastive stress during lipreading. Journal of the Acoustical Society of America, 80(Suppl. 1), S78.
3. Campbell, C.S., & Massaro, D.W. (1997). Perception of visible speech: Influence of spatial quantization. Perception, 26, 627-644.
4. Campbell, R., & Dodd, B. (1980). Hearing by eye. Quarterly Journal of Experimental Psychology, 32, 85-99.
5. Davis, H., & Silverman, S.R. (Eds.) (1970). Hearing and deafness (3rd ed.). New York: Holt, Rinehart & Winston.
6. Demorest, M.E., & Bernstein, L.E. (1992). Sources of variability in speechreading sentences: A generalizability analysis. Journal of Speech & Hearing Research, 35, 876-891.
7. Donaldson, W. (1992). Measuring recognition memory. Journal of Experimental Psychology: General, 121, 275-277.
8. Grabe, E., & Low, E.L. (2002). Durational variability in speech and the rhythm class hypothesis. In C. Gussenhoven & N. Warner (Eds.), Papers in laboratory phonology (Vol. 7, pp. 515-546). Berlin: Mouton de Gruyter.
9. Green, K.P. (1987). The perception of speaking rate using visual information from a talker's face. Perception & Psychophysics, 42, 587-593.
10. Grier, J.B. (1971). Nonparametric indexes for sensitivity and bias: Computing formulas. Psychological Bulletin, 75, 424-429.
11. Hamilton, R.H., Shenton, J.T., & Coslett, H.B. (2006). An acquired deficit of audiovisual speech processing. Brain & Language, 98, 66-73.
12. Hardison, D.M. (2003). Acquisition of second language speech: Effects of visual cues, context, and talker variability. Applied Psycholinguistics, 24, 495-522.
13. Hazan, V., Sennema, A., & Faulkner, A. (2002). Audiovisual perception in L2 learners. In J.H.L. Hansen & B. Pellom (Eds.), Proceedings of the 7th International Conference on Spoken Language Processing (pp. 1685-1688). Denver.
14. Huarte, A., Molina, M., Manrique, M., Olleta, I., & García-Tapia, R. (1996). Protocolo para la valoración de la audición y el lenguaje, en lengua española, en un programa de implantes cocleares. Acta Otorrinolaringologica Española, 47(Suppl. 1), 1-14.
15. Kaiser, A.R., Kirk, K.I., Lachs, L., & Pisoni, D.B. (2003). Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. Journal of Speech, Language, & Hearing Research, 46, 390-404.
16. Lachs, L. (1999). Use of partial stimulus information in spoken word recognition without auditory stimulation. In Research on Spoken Language Processing (Progress Rep. No. 23). Bloomington: Indiana University, Department of Psychology, Speech Research Laboratory.
17. Lachs, L., Weiss, J.W., & Pisoni, D.B. (2002). Use of partial stimulus information by cochlear implant patients and normal hearing listeners in identifying spoken words: Some preliminary analyses. Volta Review, 102, 303-320.
18. Massaro, D.W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum.
19. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
20. Mehler, J., Jusczyk, P., Lambertz, G., Halsted, N., Bertoncini, J., & Amiel-Tison, C. (1988). A precursor of language acquisition in young infants. Cognition, 29, 143-178.
21. Montgomery, A.A., & Jackson, P.L. (1983). Physical characteristics of the lips underlying vowel lipreading performance. Journal of the Acoustical Society of America, 73, 2134-2144.
22. Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71, 4-12.
23. Nazzi, T., Bertoncini, J., & Mehler, J. (1998). Language discrimination by newborns: Toward an understanding of the role of rhythm. Journal of Experimental Psychology: Human Perception & Performance, 24, 756-766.
24. Nazzi, T., Jusczyk, P.W., & Johnson, E.K. (2000). Language discrimination by English-learning 5-month-olds: Effects of rhythm and familiarity. Journal of Memory & Language, 43, 1-19.
25. Pike, K. (1946). The intonation of American English (2nd ed.). Ann Arbor: University of Michigan Press.
26. Ramus, F., & Mehler, J. (1999). Language identification with suprasegmental cues: A study based on speech resynthesis. Journal of the Acoustical Society of America, 105, 512-521.
27. Ramus, F., Nespor, M., & Mehler, J. (1999). Correlates of linguistic rhythm in the speech signal. Cognition, 73, 265-292.
28. Ronquest, R.E., Levi, S.V., & Pisoni, D.P. (2007). Language identification from visual-only speech. Research on Spoken Language Processing (Progress Report No. 28, pp. 95-118). Bloomington: Indiana University, Department of Psychology, Speech Research Laboratory.
29. Rosenblum, L.D., & Saldaña, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 61-81). Hove, U.K.: Psychology Press.
30. Soto-Faraco, S., Navarra, J., Weikum, W.M., Vouloumanos, A., Sebastián-Gallés, N., & Werker, J.F. (2007). Discriminating languages by speech reading. Perception & Psychophysics, 69, 218-237.
31. Sumby, W.H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
32. Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). Hillsdale, NJ: Erlbaum.
33. Walden, B.E., Prosek, R.A., Montgomery, A.A., Scherr, C.K., & Jones, C.J. (1977). Effects of training on the visual recognition of consonants. Journal of Speech & Hearing Research, 20, 130-145.
34. Weikum, W.M., Vouloumanos, A., Navarro, J., Soto-Faraco, S., Sebastián-Gallés, N., & Werker, J.F. (2007). Visual language discrimination in infancy. Science, 316, 1159.
35. Werker, J.F., Frost, P.E., & McGurk, H. (1992). La langue et les lèvres: Cross-language influences on bimodal speech perception. Canadian Journal of Psychology, 46, 551-568.