Effects of horizontal viewing angle on visual and audiovisual speech recognition

Journal of Experimental Psychology: Human Perception and Performance

Volumn 27, Issue 6, 2001, Pages 1386-1403

  Jordan, Timothy R ^a   Thomas, Sharon M ^a  

^a UNIVERSITY OF NOTTINGHAM   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; HUMAN; PHONETICS; RECOGNITION; SPEECH PERCEPTION; VISION;

HUMANS; PHONETICS; RECOGNITION (PSYCHOLOGY); SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 85047683853     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/0096-1523.27.6.1386     Document Type: Article

References (74)

1. Aiello, J. R. (1991). Human spatial behavior. In D. Stokols & I. Altman (Eds.), Handbook of environmental psychology (Vol. 1, pp. 389-504). Melbourne, FL: Krieger.
2. Benoit, C., Guiard-Marigny, T., Le Goff, B., & Adjoudani, A. (1996). Which components of the face do humans and machines best speechread? In D. G. Stork & M. E. Hennecke (Eds.), Speechreading by humans and machines: Models, systems and applications (NATO ASI Series F: Computer and System Sciences, Vol. 150, pp. 315-328). Berlin, Germany: Springer-Verlag.
3. Bernstein, L. E., Demorest, M. E., & Tucker, P. E. (1998). What makes a good speechreader? First you have to find one. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 211-227). London: Psychology Press.
4. Biederman, I., & Gerhardstein, P. C. (1993). Recognizing depth-rotated objects: Evidence and conditions for three-dimensional viewpoint invariance. Journal of Experimental Psychology: Human Perception and Performance, 19, 1162-1182.
5. Biederman, I., & Gerhardstein, P. C. (1995). Viewpoint-dependent mechanisms in visual object recognition: Reply to Tarr and Bülthoff (1995). Journal of Experimental Psychology: Human Perception and Performance, 21, 1506-1514.
6. Biederman, I., & Kalocsai, P. (1997). Neurocomputational bases of object and face recognition. Philosophical Transactions of the Royal Society of London, Series B, 352, 1203-1219.
7. Binnie, C. A., Montgomery, A. A., & Jackson, P. L. (1974). Auditory and visual contributions to the perception of selected English consonants for normally hearing and hearing-impaired listeners. In H. Birk Nielsen & E. Kampp (Eds.), Visual and audio-visual perception of speech (Suppl. 4, pp. 181-209). Stockholm: Almquist & Wiksell.
8. Bruce, V., Green, P. R., & Georgeson, M. A. (1996). Visual perception: Physiology, psychology and ecology (3rd ed.). London: Erlbaum.
9. Bruce, V., & Valentine, T. (1988). When a nod's as good as a wink: The role of dynamic information in facial recognition. In M. M. Gruneberg, P. E. Morris, and R. N. Sykes (Eds.), Practical aspects of memory: Current research and issues (Vol. 1, pp. 169-176). Chichester, England: Wiley.
10. Bruce, V., Valentine, T., & Baddeley, A. D. (1987). The basis of the 3/4 view advantage in face recognition. Applied Cognitive Psychology, 1, 109-120.
11. Campbell, R. (1992). The neuropsychology of lip-reading. Philosophical Transactions of the Royal Society of London, Series B, 335, 39-45.
12. Campbell, R., Dodd, B., & Burnham, D. (1998). Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech. London: Psychology Press.
13. Campbell, R., Landis, T., & Regard, M. (1986). Face recognition and lipreading: A neurological dissociation. Brain, 109, 509-521.
14. Carey, S. (1992). Becoming a face expert. In V. Bruce, A. Cowey, A. W. Ellis, & D. I. Perrett (Eds.), Processing the facial image (pp. 95-103). Oxford: Oxford University Press.
15. Conrad, R. (1979). The deaf schoolchild. London: Harper & Row.
16. Diamond, R., & Carey, S. (1986). Why faces are not special: An effect of expertise. Journal of Experimental Psychology: General, 115, 107-117.
17. Edelman, S., & Bülthoff, H. H. (1992). Orientation dependence in the recognition of familiar and novel views of three-dimensional objects. Vision Research, 32, 2385-2400.
18. Erber, N. P. (1974). Effects of angle, distance, and illumination on visual reception of speech by profoundly deaf children. Journal of Speech and Hearing Research, 17, 99-112.
19. Gailey, L. (1987). Psychological parameters of lip-reading skill. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 115-141). Hillsdale, NJ: Erlbaum.
20. Green, K. P., Kuhl, P. K., Meltzoff, A. N., & Stevens, E. B. (1991). Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect. Perception & Psychophysics, 50, 524-536.
21. Hill, H., & Bruce, V. (1996). Effects of lighting on the perception of facial surfaces. Journal of Experimental Psychology: Human Perception and Performance, 22, 986-1004.
22. Hill, H., Schyns, P. G., & Akamatsu, S. (1997). Information and viewpoint dependence in face recognition. Cognition, 62, 201-222.
23. Humphreys, G. W., Donnelly, N., & Riddoch, M. J. (1993). Expression is computed separately from facial identity and it is computed separately for moving and static faces: Neuropsychological evidence. Neuropsychologia, 31, 173-181.
24. IJsseldijk, F. J. (1992). Speechreading performance under different conditions of video image, repetition, and speech rate. Journal of Speech and Hearing Research, 35, 466-471.
25. Johnson, J. A., Rosenblum, L. D., & Saldaña, H. M. (1994, June). The contribution of a reduced visual image to speech perception in noise. Poster presented at the Spring meeting of the Acoustical Society of America, Cambridge, MA. Abstract obtained from Journal of the Acoustical Society of America, 95, 3009.
26. Jordan, T. R., & Bevan, K. M. (1997). Seeing and hearing rotated faces: Influences of facial orientation on visual and audiovisual speech recognition. Journal of Experimental Psychology: Human Perception and Performance, 23, 388-403.
27. Jordan, T. R., McCotter, M. V., & Thomas, S. M. (2000). Visual and audiovisual speech perception with color and gray scale facial images. Perception & Psychophysics, 62, 1394-1404.
28. Jordan, T. R., & Sergeant, P. C. (1998). Effects of facial image size on visual and audiovisual speech recognition. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 155-176). London: Psychology Press.
29. Jordan, T. R., & Sergeant, P. C. (2000). Effects of distance on visual and audiovisual speech recognition. Language and Speech, 43, 107-124.
30. Krouse, F. L. (1981). Effects of pose, pose change, and delay on face recognition performance. Journal of Applied Psychology, 66, 651-654.
31. Krueger, W. C. F. (1929). The effect of overlearning on retention. Psychological Review, 12, 71-78.
32. Larr, A. L. (1959). Speechreading through closed circuit television. Volta Review, 61, 19-21.
33. Laver, J., & Hutcheson, S. (1972). Communication in face to face interaction. Oxford, England: Oxford University Press.
34. Liberman, A. M. (1982). On finding that speech is special. American Psychologist, 37, 148-167.
35. Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1-36.
36. Logie, R. H., Baddeley, A. D., & Woodhead, M. M. (1987). Face recognition, pose, and ecological validity. Applied Cognitive Psychology, 1, 53-69.
37. Marassa, L. K., & Lansing, C. R. (1995). Visual word recognition in two facial motion conditions: Full-face versus lips-plus-mandible. Journal of Speech and Hearing Research, 38, 1387-1394.
38. Massaro, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum.
39. Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
40. Massaro, D. W., & Cohen, M. M. (1990). Perception of synthesized audible and visible speech. Psychological Science, 1, 55-63.
41. Massaro, D. W., & Cohen, M. M. (1996). Perceiving speech from inverted faces. Perception & Psychophysics, 58, 1047-1065.
42. McGurk, H., & MacDonald, J. (1976, December 23/30). Hearing lips and seeing voices. Nature, 264, 746-748.
43. Middleweerd, M. J., & Plomp, R. (1987). The effect of speechreading on the speech-reception threshold of sentences in noise. Journal of the Acoustical Society of America, 82, 2145-2147.
44. Miller, G. A., & Nicely, P. E. (1955). An analysis of perceptual confusions among some English consonants. Journal of the Acoustical Society of America, 27, 338-352.
45. Montgomery, A. A., & Jackson, P. L. (1983). Physical characteristics of the lips underlying vowel lipreading performance. Journal of the Acoustical Society of America, 73, 2134-2144.
46. Munhall, K. G., & Vatikiotis-Bateson, E. (1998). The moving face during speech communication. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 123-139). London: Psychology Press.
47. Neely, K. K. (1956). Effect of visual factors on the intelligibility of speech. Journal of the Acoustical Society of America, 28, 1275-1277.
48. Nelson, T. O., Leonesio, R. J., Shimamura, A. P., Landwehr, F., & Narens, L. (1982). Overlearning and the feeling of knowing. Journal of Experimental Psychology: Learning, Memory, & Cognition, 8, 279-288.
49. Perrett, D. I., Smith, P. A. J., Potter, D. D., Mistlin, A. J., Head, A. S., Milner, A. D., & Jeeves, M. A. (1985). Visual cells in the temporal cortex sensitive to face view and gaze direction. Proceedings of the Royal Society of London, Series B, 233, 293-317.
50. Petajan, E. D. (1984). Automatic lipreading to enhance speech recognition. Proceedings of the IEEE Communications Society (pp. 265-272). Atlanta, GA: IEEE.
51. Repp, B. H., Manuel, S. Y., Liberman, A. M., & Studdert-Kennedy, M. (1983, November). Exploring the McGurk effect. Paper presented at the 24th annual meeting of the Psychonomic Society, San Diego, CA.
52. Roberts, P. L., & MacLeod, C. (1995a). Representational consequences of two modes of learning. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 48(A), 296-319.
53. Roberts, P. L., & MacLeod, C. (1995b). Two data structures in cognition. In P. Slezak, P. Caelli, & R. Clark (Eds.), Perspectives on cognitive science: Theories, experiments, and foundations (pp. 157-166). Norwood, NJ: Ablex.
54. Roberts, P. L., & MacLeod, C. (1999). Automatic and strategic retrieval of structure knowledge following two modes of learning. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 52(A), 31-46.
55. Rönnberg, J. (1993). Cognitive characteristics of skilled tactiling: The case of GS. European Journal of Cognitive Psychology, 5, 19-33.
56. Rosenblum, L. D., Johnson, J. A., & Saldaña, H. M. (1996). Visual kinematic information for embellishing speech in noise. Journal of Speech and Hearing Research, 39, 1159-1170.
57. Rosenblum, L. D., & Saldaña, H. M. (1992). Discrimination tests of visually influenced syllables. Perception & Psychophysics, 52, 461-473.
58. Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-331.
59. Rosenblum, L. D., Yakel, D. A., & Green, K. P. (2000). Face and mouth inversion effects on visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 26, 806-819.
60. Saldaña, H. M., Rosenblum, L. D., & Osinga, T. (1992). Visual influence on heard speech syllables with a reduced visual image [Abstract]. Journal of the Acoustical Society of America, 92, 2340.
61. Samuelsson, S., & Rönnberg, J. (1993). Implicit and explicit use of scripted constraints in lip-reading. European Journal of Cognitive Psychology, 5, 201-233.
62. Sergent, J. (1984). An investigation into component and configurai processes underlying face perception. British Journal of Psychology, 75, 221-242.
63. Sudman, J. A., & Berger, K. W. (1971). Two-dimension vs. three-dimension viewing in speech-reading. Journal of Communication Disorders, 4, 195-198.
64. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
65. Summerfield, A. Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). Hillsdale, NJ: Erlbaum.
66. Summerfield, A. Q. (1992). Lipreading and audio-visual speech perception. In V. Bruce, A. Cowey, A. W. Ellis, & D. I. Perrett (Eds.), Processing the facial image (pp. 71-78). Oxford, England: Oxford University Press.
67. Summerfield, A. Q., & McGrath, M. (1984). Detection and resolution of audio-visual incompatibility in the perception of vowels. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 36(A), 51-74.
68. Tong, F., & Nakayama, K. (1999). Robust representations for faces: Evidence from visual search. Journal of Experimental Psychology: Human Perception and Performance, 25, 1016-1035.
69. Utley, J. (1946). A test of lip-reading ability. Journal of Speech Disorders, 11, 109-116.
70. Valentine, T., & Bruce, V. (1988). Mental rotation of faces. Memory & Cognition, 16, 556-566.
71. Vatikiotis-Bateson, E., Munhall, K. G., Hirayama, M., Lee, Y. V., & Terzopoulos, D. (1996). Dynamics of facial motion in speech: Kinematic and electromyographic studies of orofacial structures. In D. G. Stork & M. Hennecke (Eds.), Speechreading by humans and machines (NATO ASI Series F: Computer and System Sciences, Vol. 150, pp. 221-232). Berlin, Germany: Springer-Verlag.
72. Walden, B. E., Prosek, R. A., Montgomery, A. A., Scherr, C. K., & Jones, C. J. (1977). Effects of training on the visual recognition of consonants. Journal of Speech and Hearing Research, 20, 130-145.
73. Woodward, M. F., & Barber, C. G. (1960). Phoneme perception in lip-reading. Journal of Speech and Hearing Research, 3, 212-222.
74. Wozniak, V. D., & Jackson, P. L. (1979). Visual vowel and diphthong perception from two horizontal viewing angles. Journal of Speech and Hearing Research, 22, 354-365.