Hearing a face: Cross-modal speaker matching using isolated visible speech

Perception and Psychophysics

Volumn 68, Issue 1, 2006, Pages 84-93

  Rosenblum, Lawrence D ^a   Smith, Nicolas M ^a   Nichols, Sarah M ^a   Hale, Steven ^a   Lee, Joanne ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; FACE; FACIAL EXPRESSION; FEMALE; HUMAN; MALE; SPEECH PERCEPTION; VISION;

ADOLESCENT; ADULT; FACE; FACIAL EXPRESSION; FEMALE; HUMANS; MALE; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 33646466062     PISSN: 00315117     EISSN: None     Source Type: Journal    
DOI: 10.3758/BF03193658     Document Type: Article

References (46)

1. AMERMAN, J. D., & DANILOFF, R. G. (1977). Aspects of lingual coarticulation. Journal of Phonetics, 5, 107-113.
2. BARCLAY, C. D., CUTTING, J. E., & KOZLOWSKI, L. T. (1978). Temporal and spatial factors in gait perception that influence gender recognition. Perception & Psychophysics, 23, 145-152.
3. BASSILI, J. N. (1978). Facial motion in the perception of faces and of emotional expression. Journal of Experimental Psychology: Human Perception & Performance, 4, 373-379.
4. BERRY, D. S. (1990). What can a moving face tell us? Journal of Personality & Social Psychology, 58, 1004-1014.
5. BLADON, R. A. W., & AL-BAMERNI, A. (1976). Coarticulation resistance in English /1/. Journal of Phonetics, 4, 137-150.
6. BLOKLAND, A., & ANDERSON, A. H. (1998). Effect of low frame-rate video on intelligibility of speech. Speech Communication, 26, 97-103.
7. BRAUNSTEIN, M. L., HOFFMAN, D. D., & POLLICK, F. E. (1990). Discriminating rigid from nonrigid motion: Minimum points and views. Perception & Psychophysics, 47, 205-214.
8. BRICKER, P. D., & PRUZANSKY, S. (1966). Effects of stimulus content and duration on talker identification. Journal of the Acoustical Society of America, 40, 1441-1449.
9. BRUCE, V., & VALENTINE, T. (1988). When a nod's as good as a wink: The role of dynamic information in facial recognition. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory: Current research and issues. Vol. 1: Memory in everyday life (pp. 169-174). New York: Wiley.
10. BURTON, A. M., BRUCE, V., & DENCH, N. (1993). What's the difference between men and women? Evidence from facial measurement. Perception, 22, 153-176.
11. CHRISTIE, F., & BRUCE, V. (1998). The role of dynamic information in the recognition of unfamiliar faces. Memory & Cognition, 26, 780-790.
12. COLLINS, S. A., & MISSING, C. (2003). Vocal and visual attractiveness are related in women. Animal Behaviour, 65, 997-1004.
13. FELLOWES, J. M., REMEZ, R. E., & RUBIN, P. E. (1997). Perceiving the sex and identity of a talker without natural vocal timbre. Perception & Psychophysics, 59, 839-849.
14. HILDRETH, E. C., GRZYWACZ, N. M., ADELSON, E. H., & INADA, V. K. (1990). The perceptual buildup of three-dimensional structure from motion. Perception & Psychophysics, 48, 19-36.
15. JOHANSSON, G. (1973). Visual perception of biological motion and a model for its analysis. Perception & Psychophysics, 14, 201-211.
16. KAMACHI, M., HILL, H., LANDER, K., & VATIKIOTIS-BATESON, E. (2003). Putting the face to the voice: Matching identity across modality. Current Biology, 13, 1709-1714.
17. KNAPPMEYER, B., THORNTON, I. M., & BÜLTHOFF, H. H. (2003). The use of facial motion and facial form during the processing of identity. Vision Research, 23, 1927-1936.
18. KNIGHT, B., & JOHNSTON, A. (1997). The role of movement in face recognition. Visual Cognition, 4, 265-273.
19. LACHS, L.. & PISONI, D. B. (2004a). Cross-modal source information and spoken word recognition. Journal of Experimental Psychology: Human Perception & Performance, 30, 378-396.
20. LACKS, L., & PISONI, D. B. (2004b). Crossmodal source identification in speech perception. Ecological Psychology, 16, 159-187.
21. LACKS, L., & PISONI, D. B. (2004c). Specification of cross-modal source information in isolated kinematic displays of speech. Journal of the Acoustical Society of America, 116, 507-518.
22. LANDER, K., CHRISTIE, F., & BRUCE, V. (1999). The role of movement in the recognition of famous faces. Memory & Cognition, 27, 974-985.
23. MCGURK, H., & MACDONALD, J. W. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
24. NYGAARD, L. C. (2005). Perceptual integration of linguistic and non-linguistic properties of speech. In D. B. Pisoni & R. E. Remez (Eds.), Handbook of speech perception (pp. 390-414). Malden, MA: Blackwell.
25. NYGAARD, L. C., & PISONI, D. B. (1998). Talker-specific learning in speech perception. Perception & Psychophysics, 60, 355-376.
26. NYGAARD, L. C., SOMMERS, M. S., & PISONI, D. B. (1994). Speech perception as a talker-contingent process. Psychological Science, 5, 42-46.
27. PAVLOVA, M. A. (1995). Biological motion perception under various presentation rates. Perception, 24(Suppl.), 112.
28. POLLICK, F. E. (1997). The perception of motion and structure in structure-from-motion: Comparisons of affine and Euclidean formulations. Vision Research, 37, 447-466.
29. REMEZ, R. E., FELLOWES, J. M., & RUBIN, P. E. (1997). Talker identification based on phonetic information. Journal of Experimental Psychology: Human Perception & Performance, 23, 651-666.
30. REMEZ, R. E., RUBIN, P. E., NYGAARD, L. C., & HOWELL, W. A. (1987). Perceptual normalization of vowels produced by sinusoidal voices. Journal of Experimental Psychology: Human Perception & Performance, 13, 40-61.
31. REMEZ, R. E., RUBIN, P. E., PISONI, D. B., & CARRELL, T. D. (1981). Speech perception without traditional speech cues. Science, 212, 947-950.
32. ROSENBLUM, L. D. (2004). Perceiving articulatory events: Lessons for an ecological psychoacoustics. In J. G. Neuhoff(Ed.), Ecological psychoacoustics (pp. 219-248). Amsterdam: Elsevier.
33. ROSENBLUM, L. D. (2005). The primacy of multimodal speech perception. In D. Pisoni & R. Remez (Eds.), Handbook of speech perception (pp. 51-78). Malden, MA: Blackwell.
34. ROSENBLUM, L. D., JOHNSON, J. A., & SALDAÑA, H. M. (1996). Point-light facial displays enhance comprehension of speech in noise. Journal of Speech & Hearing Research, 39, 1159-1170.
35. ROSENBLUM, L. D., & SALDAÑA, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception & Performance, 22, 318-331.
36. ROSENBLUM, L. D., SMITH, N., & NIEHUS, R. P. (2006). Look who's talking: Recognizing friends from visible articulation. Manuscript submitted for publication.
37. ROSENBLUM, L. D., YAKEL, D. A., BASEER, N., PANCHAL, A., NODARSE, B. C., & NIEHUS, R. P. (2002). Visual speech information for face recognition. Perception & Psychophysics, 64, 220-229.
38. SCHWEINBERGER, S. R., & SOUKUP, G. R. (1998). Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. Journal of Experimental Psychology: Human Perception & Performance, 24, 1748-1765.
39. SHEFFERT, S. M., & OLSON, E. (2004). Audiovisual speech facilitates voice learning. Perception & Psychophysics, 66, 352-362.
40. SHEFFERT, S. M., PISONI, D. B., FELLOWES, J. M., & REMEZ, R. E. (2002). Learning to recognize talkers from natural, sinewave, and reversed speech samples. Journal of Experimental Psychology: Human Perception & Performance, 28, 1447-1469.
41. SOMMERS, M. S., NYGAARD, L. C., & PISONI, D. B. (1994). Stimulus variability and spoken word recognition: Effects of variability in speaking rate and overall amplitude. Journal of the Acoustical Society of America, 96, 1314-1324.
42. THORNTON, I. M., PINTO, J., & SHIFFRAR, M. (1998). The visual perception of human locomotion. Cognitive Neuropsychology, 15, 535-552.
43. VAN LANCKER, D., KREIMAN, J., & EMMOREY, K. (1985). Familiar voice recognition: Patterns and parameters. I: Recognition of backward voices. Journal of Phonetics, 13, 19-38.
44. VITKOVITCH, M., & BARBER, P. (1994). Effect of video frame rate on subjects' ability to shadow one of two competing verbal passages. Journal of 'Speech & Hearing Research, 37, 1204-1210.
45. WILLIAMS, D. R., VERBRUGGE, R. R., & STUDDERT-KENNEDY, M. (1983). Judging sinewave stimuli as speech and nonspeech. Journal of the Acoustical Society of America, 74, S66.
46. YAKEL, D. A., ROSENBLUM, L. D., & FORTIER, M. A. (2000). Effects of talker variability on speechreading. Perception & Psychophysics, 62, 1405-1412.