It's Not What You Say but the Way You Say It: Matching Faces and Voices

Journal of Experimental Psychology: Human Perception and Performance

Volumn 33, Issue 4, 2007, Pages 905-914

  Lander, Karen ^a   Hill, Harold ^b   Kamachi, Miyuki ^c   Vatikiotis Bateson, Eric ^d  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^c KOGAKUIN UNIVERSITY   (Japan)

^d UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

face and voice matching; face identity; voice identity

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; FACE; FEMALE; HUMAN; MALE; RECOGNITION; SPEECH PERCEPTION; VISION; VOICE;

ADOLESCENT; ADULT; FACE; FEMALE; HUMANS; MALE; RECOGNITION (PSYCHOLOGY); SPEECH PERCEPTION; VISUAL PERCEPTION; VOICE; VOICE QUALITY;

EID: 34548848519     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/0096-1523.33.4.905     Document Type: Article

References (30)

1. Auberge V., and Cathiard M. Can we hear the prosody of smile?. Speech Communication 40 (2003) 87-97
2. Bernstein L., Demorest M.E., and Tucker P.E. What makes a good speechreader? First you have to find one. In: Campbell R., Dodd B., and Burnham D. (Eds). Hearing by eye II (1998), Psychology Press, Hove, England 211-227
3. Cohen J.D., MacWhinney B., Flatt M., and Provost J. PsyScope: A new graphic interactive environment for designing psychology experiments. Behavior Research Methods, Instruments, & Computers 25 (1993) 257-271
4. DeCarlo D., and Metaxas D. Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision 38 (2000) 99-127
5. De Gelder B., and Vroomen J. The perception of emotions by eye and ear. Cognition and Emotion 14 (2000) 289-311
6. Dohen M., Loevenbruck H., Cathiard M.-A., and Schwartz J.-L. Visual perception of contrastive focus in reiterant French speech. Speech Communication 44 (2004) 155-172
7. Gagné J.P., Rochette A.J., and Charest M. Auditory, visual and audiovisual clear speech. Speech Communication 37 (2002) 213-230
8. Hargus Ferguson S., and Kewley-Port D. Vowel intelligibility in clear and conversational speech for normal-hearing and hearing-impaired listeners. Journal of the Acoustical Society of America 112 (2002) 259-271
9. Janse E. Word perception in fast speech: Artificially time compressed vs. naturally produced fast speech. Speech Communication 42 (2004) 155-173
10. Kamachi M., Hill H., Lander K., and Vatikiotis-Bateson E. Putting the face to the voice: Matching identity across modality. Current Biology 13 (2003) 1709-1714
11. Kawahara H., Masuda-Katsuse I., and de Cheveigné A. Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds. Speech Communication 27 (1999) 187-207
12. Lachs L., and Pisoni D.B. Crossmodal source identification in speech perception. Ecological Psychology 16 (2004) 159-187
13. Lander K., and Chuang L. Why are moving faces easier to recognize?. Visual Cognition 12 (2005) 429-442
14. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976, December 23) 746-748
15. Miller J.L., Grosjean F., and Lomanto C. Articulation rate and its variability in spontaneous speech: A reanalysis and some implications. Phonetica 41 (1984) 215-225
16. Munhall K.G., and Buchan J.N. Something in the way she moves. Trends in Cognitive Sciences 8 (2004) 51-53
17. Munhall K.G., Jones J.A., Callan D., Kuratate T., and Vatikiotis-Bateson E. Visual prosody and speech intelligibility: Head movement improves auditory speech perception. Psychological Science 15 (2004) 133-137
18. O'Toole A.J., Roark D., and Abdi H. Recognizing moving faces: A psychological and neural synthesis. Trends in Cognitive Science 6 (2002) 261-266
19. Picheny M.A., Durlach N.I., and Braida L.D. Speaking clearly for the hard of hearing I: Intelligibility differences between clear and conversational speech. Journal of Speech and Hearing Research 28 (1985) 96-103
20. Remez R.E., Fellowes J.M., and Rubin P.E. Talker identification based on phonetic information. Journal of Experimental Psychology: Human Perception and Performance 23 (1997) 651-666
21. Rosenblum L.D., Smith N.M., Nichols S., Lee J., and Hale S. Hearing a face: Crossmodal speaker matching using isolated visible speech. Perception & Psychophysics 68 (2006) 84-93
22. Smiljanic R., and Bradlow A.R. Production and perception of clear speech in Croatian and English. Journal of the Acoustical Society of America 118 (2005) 1677-1688
23. Srinivasan R.J., and Massaro D.W. Perceiving prosody from the face and voice: Distinguishing statements from echoic questions in English. Language and Speech 46 (2003) 1-22
24. Sumby W.H., and Pollack I. Visual contribution to speech information in noise. Journal of the Acoustical Society of America 26 (1954) 212-215
25. Tartter V. Hearing smiles and frowns in normal and whisper registers. Journal of the Acoustical Society of America 96 (1994) 2101-2107
26. Thomas F., and Johnston O. Disney animation: The illusion of life (1981), Abbeville Press, New York
27. Uchanski R., Choi S., Braida L., Reed C., and Durlach N. Speaking clearly for the hard of hearing IV: Further studies of the role of speaking rate. Journal of Speech and Hearing Research 39 (2002) 494-509
28. Vatikiotis-Bateson E., Munhall K., Hirayama M., Lee Y.C., and Terzopoulos D. The dynamics of audiovisual behaviour in speech. In: Stork D.G., and Hennecke M.E. (Eds). Speechreading by humans and machines 150 (1996), Springer-Verlag, Berlin, Germany 221-232
29. Yehia H.C., Kuratate T., and Vatikiotis-Bateson E. Linking facial animation, head motion and speech acoustics. Journal of Phonetics 30 (2002) 555-568
30. Yehia H.C., Rubin P.E., and Vatikiotis-Bateson E. Quantitative association of vocal-tract and facial behaviour. Speech Communication 26 (1998) 23-44