Seeing and Hearing Rotated Faces: Influences of Facial Orientation on Visual and Audiovisual Speech Recognition

Journal of Experimental Psychology: Human Perception and Performance

Volumn 23, Issue 2, 1997, Pages 388-403

  Jordan, Timothy R ^a   Bevan, Kim ^a  

^a UNIVERSITY OF ST ANDREWS   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; AUDITORY STIMULATION; FEMALE; HEARING; HUMAN; MALE; ORIENTATION; PHOTOSTIMULATION; PHYSIOLOGY; SPEECH; VISION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY PERCEPTION; FEMALE; HUMANS; MALE; ORIENTATION; PHOTIC STIMULATION; SPEECH; VISUAL PERCEPTION;

EID: 0031110826     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/0096-1523.23.2.388     Document Type: Article

References (47)

1. Bertelson, P., Vroomen, J., Wiegeraad, G., & de Gelder, B. (1994, September). Exploring the relation between McGurk interference and ventriloquism. Paper presented at the International Conference on Spoken Language Processing, Yokohama, Japan.
2. Binnie, C. A., Montgomery, A. A., & Jackson, P. L. (1974). Auditory and visual contributions to the perception of selected English consonants for normally hearing and hearing-impaired listeners. In H. Birk Nielsen & E. Kampp (Eds.), Visual and audio-visual perception of speech (Scandinavian Suppl. 4, pp. 181-209). Stockholm: Almquist & Wiksell.
3. Bruce, V., & Valentine, T. (1988). When a nod's as good as a wink: The role of dynamic information in facial recognition. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory: Current research and issues (Vol. 1, pp 169-174). New York: Wiley.
4. Campbell, R. (1992). The neuropsychology of lip-reading. Philosophical Transactions of the Royal Society of London, B33S pp. 39-45.
5. Campbell, R., Landis, T., & Regard, M. (1986). Face recognition and lipreading: A neurological dissociation. Brain, 109, 509-521.
6. Cohen, M. M. (1984). Processing of visual and auditory information in speech perception. Unpublished doctoral dissertation, University of California, Santa Cruz.
7. Conrad, R. (1979). The deaf schoolchild. London: Harper & Row.
8. Diesch, E. (1995). Left and right hemifield advantages of fusions and combinations in audiovisual speech perception. Quarterly Journal of Experimental Psychology, 48A, 320-333.
9. Gailey, L. (1987). Psychological parameters of lip-reading skill. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 115-141). Hillsdale, NJ: Erlbaum.
10. Green, K. P. (1994). The influence of an inverted face on the McGurk effect [abstract]. Journal of the Acoustical Society of America, 95, 3014.
11. Green, K. P., & Kuhl, P. K. (1989). The role of visual information in the processing of place and manner features in speech perception. Perception & Psychophysics, 45, 34-42.
12. Green, K. P., & Kuhl, P. K. (1991). Integral processing of visual place and auditory voicing information during phonetic perception. Journal of Experimental Psychology: Human Perception and Performance, 17, 278-288.
13. Green, K. P., Kuhl, P. K., & Meltzoff, A. N. (1988). Factors affecting the integration of auditory and visual information in speech: The effect of vowel environment. Journal of the Acoustical Society of America, 84(Suppl.), 155.
14. Green, K. P., Kuhl, P. K., Meltzoff, A. N., & Stevens, E. B. (1991). Integrating speech information across talkers, gender and sensory modality: Female faces and male voices in the McGurk effect. Perception & Psychophysics, 50, 524-536.
15. Green, K. P., & Miller, J. L. (1985). On the role of visual rate information in phonetic perception. Perception & Psychophysics, 38, 269-276.
16. Humphreys, G. W., Donnelly, N., & Riddoch, M. J. (1993). Expression is computed separately from facial identity and it is computed separately for moving and static faces: Neuropsychological evidence. Neuropsychologia, 31, 173-181.
17. Johnson, J. A., Rosenblum, L. D., & Saldana, H. M. (1994). The contribution of reduced visual image to speech perception in noise [abstract]. Journal of the Acoustical Society of America, 95, 3009.
18. Liberman, A. M. (1982). On finding that speech is special. American Psychologist, 37, 148-167.
19. MacDonald, J., & McGurk, H. (1978). Visual influences on speech perception processes. Perception & Psychophysics, 24, 253-257.
20. Manuel, S. Y., Repp, B. H., Studdert-Kennedy, M., & Liberman, A. M. (1983). Exploring the "McGurk effect" [abstract]. Journal of the Acoustical Society of America, 74, S66.
21. Marr, D., & Nishihara, H. K. (1978). Representation and recognition of the spatial organization of three-dimensional shapes. Proceedings of the Royal Society of London, B200, 269-294.
22. Massaro, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum.
23. Massaro, D. W., & Cohen, M. M. (1983). Evaluation and integration of visual and auditory information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 9, 753-771.
24. Massaro, D. W., & Cohen, M. M. (1994). Perceiving speech from inverted faces. Unpublished manuscript.
25. Massaro, D. W., Cohen, M. M., Gesi, A., & Heredia, R. (1993). Bimodal speech perception: An examination across languages. Journal of Phonetics, 24, 445-478.
26. McGrath, M., & Summerfield, Q. (1985). Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. Journal of the Acoustical Society of America, 77, 678-685.
27. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
28. Perrett, D. I., May, K. A., & Yoshikawa, S. (1994). Facial shape and judgments of female attractiveness. Nature, 368, 239-242.
29. Repp, B. H., Manuel, S. Y., Liberman, A. M., & Studdert-Kennedy, M. (1983, November). Exploring the McGurk effect. Paper presented at the 24th Annual Meeting of the Psychonomic Society, San Diego, California.
30. Rock, I. (1974). The perception of disoriented figures. Scientific American, 230, 78-85.
31. Rönnberg, J. (1993). Cognitive characteristics of skilled tactiling: The case of GS. European Journal of Cognitive Psychology, 5, 19-33.
32. Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-331.
33. Saldaña, H. M., Rosenblum, L. D., & Osinga, T. (1992). Visual influence on heard speech syllables with a reduced visual image [abstract]. Journal of the Acoustical Society of America, 92, 2340.
34. Samuelsson, S., & Rönnberg, J. (1993). Implicit and explicit use of scripted constraints in lip-reading. European Journal of Cognitive Psychology, 5, 201-233.
35. Sekiyama, K., & Tohkura, Y. (1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America, 90, 1797-1805.
36. Shepard, R. N., & Metzler, J. (1971). Mental rotation of three dimensional objects. Science, 171, 701-703.
37. Sudman, J. A., & Berger, K. W. (1971). Two-dimension vs. three-dimension viewing in speech-reading. Journal of Communication Disorders, 4, 195-198.
38. Summerfield, A. Q. (1979). Use of visual information for phonetic perception. Phonetica, 36, 314-331.
39. Summerfield, A. Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). Hillsdale, NJ: Erlbaum.
40. Summerfield, A. Q. (1992). Lipreading and audio-visual speech perception, in V. Bruce, A. Cowey, A. W. Ellis, & D. I. Perrett (Eds.), Processing the facial image (pp. 71-78). Oxford, England: Oxford University Press.
41. Summerfield, A. Q., MacLeod, P., McGrath, M., & Brooke, N. M. (1989). Lips, teeth, and the benefits of lipreading. In A. W. Young & H. D. Ellis (Eds.), Handbook of research on face processing (pp. 223-233). Amsterdam: North-Holland.
42. Summerfield, A. Q., & McGrath, M. (1984). Detection and resolution of audio-visual incompatibility in the perception of vowels. Quarterly Journal of Experimental Psychology, 36A, 51-74.
43. Utley, J. (1946). A test of lip-reading ability. Journal of Speech Disorders, 11, 109-116.
44. Valentine, T. (1988). Upside-down faces: A review of the effect of inversion upon face recognition. British Journal of Psychology, 79, 471-491.
45. Valentine, T., & Bruce, V. (1988). Mental rotation of faces. Memory & Cognition, 16, 556-566.
46. Walden, B. E., Prosek, R. A., Montgomery, A. A., Scherr, C. K., & Jones, C. J. (1977). Effects of training on the visual recognition of consonants. Journal of Speech and Hearing Research, 20, 130-145.
47. Woodward, M. F., & Barber, C. G. (1960). Phoneme perception in lipreading. Journal of Speech and Hearing Research, 3, 212-222.