Specification of cross-modal source information in isolated kinematic displays of speech

Journal of the Acoustical Society of America

Volumn 116, Issue 1, 2004, Pages 507-518

  Lachs, Lorin ^a   Pisoni, David B ^b  

^a CALIFORNIA STATE UNIVERSITY   (United States)

^b INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACOUSTIC NOISE; FINITE DIFFERENCE METHOD; SIGNAL PROCESSING; SPEECH RECOGNITION;

POINT-LIGHT DISPLAYS (PLD); VOCAL IDENTITY;

ACOUSTICS;

ARTICLE; AUDITORY STIMULATION; COLLEGE STUDENT; HUMAN; KINEMATICS; PRIORITY JOURNAL; SPEECH ARTICULATION; SPEECH PERCEPTION; TASK PERFORMANCE;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; BIOMECHANICS; FACIAL EXPRESSION; FEMALE; HUMANS; MALE; RECOGNITION (PSYCHOLOGY); SOUND SPECTROGRAPHY; SPEECH PERCEPTION; VISUAL PERCEPTION; VOCABULARY; VOCAL CORDS;

EID: 3142688301     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.1757454     Document Type: Article

References (49)

1. Auer, Jr., E. T. (2002). "The influence of the lexicon on speechread word recognition: Contrasting segmental and lexical distinctiveness," Psychonom. Bull. Rev. 9(2), 341-347.
2. Bergeson, T. R., Pisoni, D. B., Lachs, L., and Reese, L. (2003). "Audiovisual integration of point light displays of speech by deaf adults following cochlear implantation," in Proceedings of the 15th International Conference on Phonetic Sciences (Causal Productions, Adelaide, Australia), pp. 1469-1472.
3. Bernstein, L. E., Demorest, M. E., and Tucker, P. E. (2000). "Speech perception without hearing," Percept. Psychophys. 62(2), 233-252.
4. Bingham, G. P. (1995). "Dynamics and the problem of visual event recognition," in Mind as Motion: Explorations in the dynamics of cognition, edited by R. F. Port and T. Van Gelder, Cambridge, MA: The (MIT, Cambridge), pp. 403-448.
5. Blesser, B. (1972). "Speech perception under conditions of spectral transformation: I. Phonetic characteristics," J. Speech Hear. Res. 15(1), 1-41.
6. Braida, L. D. (1991). "Crossmodal integration in the identification of consonant segments," Q. J. Exp. Psychol. 43A(3), 647-677.
7. Brancazio, L. (1998). "Contributions of the lexicon to audiovisual speech perception," unpublished Ph.D., dissertation, University of Connecticut.
8. Bruce, V. (1988). Recognising Faces (Erlbaum, Hove, UK).
9. Bruce, V., and Valentine, T. (1988). "When a nod's as good as a wink: The role of dynamic information in facial recognition," in Practical Aspects of Memory: Current research and issues: Vol. 1. Memory in everyday life, edited by M. M. Gruneberg, P. E. Morris, and R. N. Sykes (Wiley, New York), pp. 169-174.
10. Campbell, R., Landis, T., and Regard, M. (1986). "Face recognition and lipreading: A neurological dissociation," Brain 109, 509-521.
11. Fowler, C. A. (1986). "An event approach to the study of speech perception from a direct-realist perspective," J. Phonetics 14, 3-28.
12. Fowler, C. A., and Dekle, D. J. (1991). "Listening with eye and hand: Cross-modal contributions to speech perception," J. Exp. Psychol. Hum. Percept. Perform. 17(3), 816-828.
13. Gibson, J. J. (1979). The Ecological Approach to Visual Perception (Houghton-Mifflin, Boston).
14. Goldinger, S. D. (1998). "Echoes of echoes? An episodic theory of lexical access," Psychol. Rev. 105(2), 251-279.
15. Grant, K. W. (2001). "The effect of speechreading for masked detection thresholds for filtered speech," J. Acoust. Soc. Am. 109, 2272-2275.
16. Grant, K. W., and Seitz, P. F. (1998). "Measures of auditory-visual integration in nonsense syllables and sentences," J. Acoust. Soc. Am. 104, 2438-2450.
17. Grant, K. W., and Seitz, P. F. (2000)."The use of visible speech cues for improving auditory detection of spoken sentences," J. Acoust. Soc. Am. 108(3), 1197-1208.
18. Grant, K. W., Walden, B. E., and Seitz, P. F. (1998). "Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration," J. Acoust. Soc. Am. 103, 2677-2690.
19. Johansson, G. (1973). "Visual perception of biological motion and a model for its analysis," Percept. Psychophys. 14, 201-211.
20. Kamachi, M., Hill, H., Lander, K., and Vatikiotis-Bateson, E. (2003). "'Putting the face to the voice': Matching identity across modality," Curr. Biol. 13, 1709-1714.
21. Kozlowski, L. T., and Cutting, J. E. (1978). "Recognizing the gender of walkers from point-lights moutned on ankles: Some second thoughts," Percept. Psychophys. 23, 459.
22. Lachs, L., and Pisoni, D. B. (2004a). "Crossmodal source information and spoken word recognition," J. Exp. Psychol. Hum. Percept. Perform. 30(2), 378-396.
23. Lachs, L., and Pisoni, D. B. (in press). "Crossmodal source identification in speech perception," Ecol. Psychol.
24. Lachs, L., and Pisoni, D. B. (submitted). "Spoken word recognition without audition," Percept. Psychophys.
25. Lewkowicz, D. J. (2001). "Infants' perception of the audible, visible and bimodal attributes of multimodal syllables," Child Dev. 71(5), 1241-1257.
26. Liberman, A., and Mattingly, I. (1985). "The motor theory revised," Cognition 21, 1-36.
27. Luce, P. A., and Pisoni, D. B. (1998). "Recognizing spoken words: The neighborhood activation model," Ear Hear. 19, 1-36.
28. Massaro, D. W. (1998). Perceiving Talking Faces: From speech perception to a behavioral principle (MIT, Cambridge, MA).
29. McGurk, H., and MacDonald, J. (1976). "Hearing lips and seeing voices," Nature (London) 264, 746-748.
30. Munhall, K. G., and Vatikiotis-Bateson, E. (1998). "The moving face during speech communication," in Hearing by Eye II: Advances in the Psychology of Speechreading and Auditory-visual Speech, edited by R. Campbell, B. Dodd, and D. Burnham (Psychology, East Sussex, UK), pp. 123-139.
31. Reisberg, D., McLean, J., and Goldfield, A. (1987). "Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli," in Hearing by Eye: The psychology of lip reading, edited by B. Dodd and R. Campbell (Erlbaum, Hillsdale, NJ), pp. 97-114.
32. Remez, R. E., Fellowes, J. M., and Rubin, P. E. (1997). "Talker identification based on phonetic information," J. Exp. Psychol. Hum. Percept. Perform. 23(5), 651-666.
33. Remez, R. E., Rubin, P. E., Pisoni, D. B., and Carrell, T. D. (1981). "Speech perception without traditional speech cues," Science 212, 947-950.
34. Remez, R. E., Fellowes, J. M., Pisoni, D. B., Goh, W. D., and Rubin, P. E. (1999). "Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances," Speech Commun. 26(1-2), 65-73.
35. Rosenblum, L. D. (1994). "How special is audiovisual speech integration?" Curr. Psychol. Cognition 13(1), 110-116.
36. Rosenblum, L. D., and Saldaña, H. M. (1996). "An audiovisual test of kinematic primitives for visual speech perception," J. Exp. Psychol. Hum. Percept. Perform. 22(2), 318-331.
37. Rosenblum, L. D., Johnson, J. A., and Saldaña, H. M. (1996). "Point-light facial displays enhance comprehension of speech in noise," J. Speech Lang. Hear. Res. 39, 1159-1170.
38. Rosenblum, L. D., Smith, N. M., Nichols, S. M., Hale, S., and Lee, J. (under review). "Hearing a face: Cross-modal speaker matching using isolated visible speech," Percept. Psychophys.
39. Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nodarse, B. C., and Niehaus, R. P. (2002). "Visual speech information for face recognition," Percept. Psychophys. 64(2), 220-229.
40. Runeson, S. (1994). "Perception of biological motion: The KSD-principle and the implications of the distal versus optimal approach," in Perceiving Events and Objects, edited by G. Jansson, S. S. Bergström, and W. Epstein (Erlbaum, Hillsdale, NJ), pp. 383-405.
41. Runeson, S., and Frykholm, G. (1981). "Visual perception of lifted weight," J. Exp. Psychol. Hum. Percept. Perform. 7, 733-740.
42. Schultz, M., Norton, S., Conway-Fithian, S., and Reed, C. (1984). "A survey of the use of the Tadoma method in the United States and Canada," Volta Rev. 86, 282-292.
43. Schwartz, J.-L., Robert-Ribes, J., and Escudier, P. (1998). "Ten years after Summerfield: A taxonomy of models for audio-visual fusion in speech perception," in Hearing by Eye II, edited by R. Campbell, B. Dodd, and D. Burnham (Psychology, East Sussex, UK), pp. 85-108.
44. Smith, Z. M., Delgutte, B., and Oxenham, A. J. (2002). "Chimaeric sounds reveal dichotomies in auditory perception," Nature (London) 416, 87-90.
45. Sumby, W. H., and Pollack, I. (1954). "Visual contribution of speech intelligibility in noise," J. Acoust. Soc. Am. 26, 212-215.
46. Summerfield, Q. (1987). "Some preliminaries to a comprehensive account of audio-visual speech perception," in Hearing by Eye: The Psychology of Lip-Reading, edited by B. Dodd and R. Campbell (Erlbaum, Hillsdale, NJ), pp. 3-51.
47. Vatikiotis-Bateson, E., Hill, H., Kamachi, M., Lander, K., and Munhall, K. (2002). "The stimulus as basis for audiovisual integration," paper presented at the International Conference on Spoken Language Processing, Denver, CO.
48. Vatikiotis-Bateson, E., Munhall, K. G., Hirayama, M., Lee, Y. V., and Terzepoulos, D. (1997). "The dynamics of audiovisual behavior in speech," in Speechreading by Humans and Machines, edited by D. G. Stork and M. E. Hennecke (Springer-Verlag, Berlin), pp. 221-232.
49. Yehia, H., Rubin, P. E., and Vatikiotis-Bateson, E. (1998). "Quantitative association of vocal-tract and facial behavior," Speech Commun. 26(1-2), 23-43.