The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception

Brain Research

Volumn 1242, Issue , 2008, Pages 162-171

  Buchan, Julie N ^a   Paré, Martin ^a   Munhall, Kevin G ^a  

^a QUEEN S UNIVERSITY   (Canada)

Author keywords

Audiovisual speech; Dynamic face; Eyetracking; Face processing; Identity; Speech perception

Indexed keywords

ADULT; ARTICLE; CONTROLLED STUDY; EYE FIXATION; EYE TRACKING; FACIAL EXPRESSION; FEMALE; GAZE; HUMAN; MALE; NOISE; OUTCOME ASSESSMENT; PRIORITY JOURNAL; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; VISUAL INFORMATION;

ADULT; BEHAVIOR; BRAIN; FACE; FEMALE; FIXATION, OCULAR; HUMANS; MALE; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 55549141825     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2008.06.083     Document Type: Article

References (48)

1. Althoff R.R., and Cohen N.J. Eye-movement-based memory effect: a reprocessing effect in face perception. J. Exp. Psychol. Learn. 25 (1999) 997-1010
2. Ambadar Z., Schooler J.W., and Cohn J.F. Deciphering the enigmatic face: the importance of facial dynamics in interpreting subtle facial expressions. Psychol. Sci. 16 (2005) 403-410
3. Baron-Cohen S., Wheelwright S., Hill J., Raste Y., and Plumb I. The "Reading the mind in the eyes" test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J. Child Psychol. Psychiatry 42 (2001) 241-251
4. Barton J., Radcliffe N., Cherkasova M., Edelman J., and Intriligator J. Information processing during face recognition: the effects of familiarity, inversion and morphing on scanning fixations. Perception 353 (2006) 1089-1105
5. Bernstein L.E., Demorest M.E., and Tucker P.E. Speech perception without hearing. Percept. Psychophys. 62 (2000) 233-252
6. Buchan J.N., Paré M., and Munhall K.G. Spatial statistics of gaze fixations during dynamic face processing. Soc. Neurosci. 2 (2007) 1-13
7. Emery N.J. The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci. Behav. Res. 24 (2000) 581-604
8. Erber N.P. Interaction of audition and vision in the recognition of oral speech stimuli. J. Speech Hear. Res. 12 (1969) 423-425
9. Everdell I.T., Marsh H., Yurick M., Munhall K.G., and Paré M. Gaze behavior in audiovisual speech perception: Asymmetrical distribution of face-directed fixations. Perception 36 (2007) 1535-1545
10. Ghazanfar A.A., Turesson H.K., Maier J.X., van Dinther R., Patterson R.D., and Logothetis N.K. Vocal-tract resonances as indexical cues in rhesus monkeys. Curr. Biol. 17 (2007) 425-430
11. Henderson J.M. Human gaze control during real-world scene perception. Trends Cogn. Sci. 7 (2003) 498-504
12. Henderson J.M., and Pierce G.L. Eye movements during scene viewing: evidence for mixed control of fixation durations. Psychon. B Rev. 15 (2008) 566-573
13. Henderson J.M., Williams C.C., and Falk R.J. Eye movements are functional during face learning. Mem. Cognit. 33 (2005) 98-106
14. Hill H., and Johnson A. Categorizing sex and identity from the biological motion of faces. Curr. Biol. 11 (2001) 880-885
15. Hsiao J.H., and Cottrell G.W. The influence of number of eye fixations on face recognition. J. Vis. 7 (2007) 494a
16. Lachs L., and Pisoni D.B. Cross-modal source information and spoken word recognition. J. Exp. Psychol. Human 30 (2004) 378-396
17. Lander K., and Bruce V. Recognizing famous faces: exploring the benefits of facial motion. Ecol. Psychol. 14 (2000) 385-388
18. Lansing C.R., and McConkie G.W. Attention to facial regions in segmental and prosodic visual speech perception tasks. J. Speech Lang. Hear R. 42 (1999) 526-539
19. Lansing C.R., and McConkie G.W. Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Percept. Psychophys. 65 (2003) 536-552
20. Kalikow D.N., Stevens K.N., and Elliott L.L. Development of a test of speech intelligibility in noise using sentence materials with controlled word predictability. J. Acoust. Soc. Am. 61 (1977) 1337-1351
21. Kamachi M., Hill H., Lander L., and Vatikiotis-Bateson E. 'Putting the face to the voice': matching identity across modality. Curr. Biol. 13 (2003) 1709-1714
22. Knappmeyer B., Thornton I.M., and Bülthoff H.H. The use of facial motion and facial form during the processing of identity. Vision Res. 43 (2003) 1921-1936
23. MacDonald J., Andersen S., and Bachmann T. Hearing by eye: how much spatial degradation can be tolerated?. Perception 29 (2000) 1155-1168
24. MacDonald J., and McGurk H. Visual influences on speech perception processes. Percept. Psychophys. 24 (1978) 253-257
25. McGurk H., and MacDonald J.W. Hearing lips and seeing voices. Nature 264 (1976) 746-748
26. Munhall K.G., Kroos C., Jozan G., and Vatikiotis-Bateson E. Spatial frequency requirements of audiovisual speech perception. Percept. Psychophys. 66 (2004) 538-574
27. Nygaard L.C., and Pisoni D.B. Talker-specific learning in speech perception. Percept. Psychophys. 60 (1998) 355-376
28. Nygaard L.C., Sommers M.S., and Pisoni D.B. Speech perception as a talker-contingent process. Psychol. Sci. 5 (1994) 42-46
29. O'Neill J.J. Contributions of the visual components of oral symbols to speech comprehension. J. Speech Hearing Disord. 19 (1954) 429-439
30. O'Toole A.J., Roark D.A., and Abdi H. Recognizing moving faces: a psychological and neural synthesis. Trends Cogn. Sci. 6 (2002) 261-266
31. Paré M., Richler R.C., ten Hove M., and Munhall K.G. Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect. Percept. Psychophys. 65 (2003) 533-567
32. Parkhurst D., Law K., and Niebur E. Modeling the role of salience in the allocation of overt visual attention. Vision Res. 42 (2002) 107-123
33. Parkhurst D.J., and Niebur E. Scene content selected by active vision. Spatial Vis. 16 (2003) 125-154
34. Rayner K., Liversedge S.P., White S.J., and Vergilino-Perez D. Reading disappearing text: cognitive control of eye movements. Psychol. Sci. 14 (2003) 385-388
35. Rosenblum L.D., Miller R.M., and Sanchez K. Lip-read me now, hear me better later: cross-modal transfer of talker-familiarity effects. Psychol. Sci. 18 (2007) 392-396
36. Thomas S.M., and Jordan T.R. Contributions of oral and extraoral facial movement to visual and audiovisual speech perception. J. Exp. Psychol. Hum. 30 (2004) 873-888
37. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., and Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 17 (2007) 1147-1153
38. Schyns P.G., Bonnar L., and Gosselin F. Show me the features! Understanding recognition from the use of visual information. Psychol. Sci. 13 (2002) 402-409
39. Schweinberger S.R., and Soukup G.R. Asymmetric relationships among perceptions of facial identity, emotion and facial speech. J. Exp. Psychol. Hum. 24 (1998) 1748-1795
40. Stacey P.C., Walker S., and Underwood J.D. Face processing and familiarity: evidence from eye-movement data. Br. J. Psychol. 96 (2005) 407-422
41. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
42. Vatikiotis-Bateson E., Eigsti I.M., Yano S., and Munhall K.G. Eye movement of perceivers during audiovisual speech perception. Percept. Psychophys. 60 (1998) 926-940
43. Vinette C., Gosselin F., and Schyns P.G. Spatio-temporal dynamics of face recognition in a flash: it's in the eyes. Cogn. Sci. 28 (2004) 289-301
44. Walker S., Bruce V., and O'Malley C. Facial identity and facial speech processing: familiar faces and voices in the McGurk effect. Percept. Psychophys. 57 (1995) 124-1133
45. Walker-Smith G.J., Gale A.G., and Findlay J.M. Eye movement strategies involved in face perception. Perception 6 (1977) 313-326
46. Yakel D.A., Rosenblum L.D., and Fortier M.A. Effects of talker variability on speechreading. Percept. Psychophys. 62 (2000) 1405-1412
47. Yarbus A.L. Eye Movements and Vision (1967), Plenum Press, New York
48. Yehia H.C., Rubin P.E., and Vatikiotis-Bateson E. Quantitative association of vocal-tract and facial behavior. Speech. Commun. 26 (1998) 23-44