Spatial frequency requirements for audiovisual speech perception

Perception and Psychophysics

Volumn 66, Issue 4, 2004, Pages 574-583

  Munhall, K G ^a   Kroos, C ^b   Jozan, G ^b   Vatikiotis Bateson, E ^b,c  

^a QUEEN S UNIVERSITY   (Canada)

^b ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 4544302649     PISSN: 00315117     EISSN: None     Source Type: Journal    
DOI: 10.3758/BF03194902     Document Type: Article

References (58)

1. BANKS, M., GEISLER, W., & BENNETT, P. (1987). The physical limits of grating visibility. Vision Research, 27, 1915-1924.
2. BENCH, J., & BAMFORD, J. M. (EDS.) (1979). Speech-hearing tests and the spoken language of hearing-impaired children. London: Academic Press.
3. BENOÎT, C., GUIARD-MARIGNY, T., LE GOFF, B., & ADJOUDANI, A. (1996). Which components of the face do humans and machines best speechread? In D. G. Stork & M. E. Hennecke (Eds.), Speechreading by humans and machines: Models, systems, and applications (pp. 315-328). Berlin: Springer-Verlag.
4. BROOKE, N. M., & TEMPLETON, P. D. (1990). Visual speech intelligibility of digitally processed facial images. Proceedings of the Institute of Acoustics, 12, 483-490.
5. BROOKS, B. E., ROSIELLE, L. J., & COOPER, E. E. (2002). The priming of face recognition after metric transformations. Perception, 31, 297-313.
6. BRUCE, V., & YOUNG, A. (1986). Understanding face recognition. British Journal of Psychology, 77, 305-327.
7. CALVERT, G. A., & CAMPBELL, R. (2003). Reading speech from still and moving faces: The neural substrates of visible speech. Journal of Cognitive Neuroscience, 15, 57-70.
8. CALVERT, G. A., HANSEN, P. C., IVERSEN, S. D., & BRAMMER, M. J. (2001). Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuro-Image, 14, 427-438.
9. CAMPBELL, C., & MASSARO, D. (1997). Perception of visible speech: Influence of spatial quantization. Perception, 26, 129-146.
10. CAMPBELL, R., ZIHL, J., MASSARO, D., MUNHALL, K., & COHEN, M. M. (1997). Speechreading in the akinetopsic patient, L.M. Brain, 120, 1793-1803.
11. COSTEN, N. P., PARKER, D. M., & CRAW, I. (1996). Effects of high-pass and low-pass spatial filtering on face identification. Perception & Psychophysics, 58, 602-612.
12. DAVIS, H., & SILVERMAN, S. R. (EDS.) (1970). Hearing and deafness (3rd ed.). New York: Holt, Rinehart & Winston.
13. GEISLER, W. S. (1989). Sequential ideal-observer analysis of visual discriminations. Psychological Review, 96, 267-314.
14. GOLD, J., BENNETT, P., & SEKULER, A. (1999). Identification of band-pass filtered letters and faces by human and ideal observers. Vision Research, 39, 3537-3560.
15. GRANT, K., & GREENBERG, S. (2001). Speech intelligibility derived from asynchronous processing of auditory-visual information. In Proceedings of AVSP-01 (pp. 132-137). Scheelsminde, Denmark.
16. GREENBERG, S., & ARAI, T. (2001). The relation between speech intelligibility and the complex modulation spectrum. In Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech-2001, pp. 473-476). Aalborg, Denmark.
17. HAYES, T., MORRONE, M. C., & BURR, D. C. (1986). Recognition of positive and negative bandpass-filtered images. Perception, 15, 595-602.
18. HILL, H., & JOHNSTON, A. (2001). Categorizing sex and identity from the biological motion of faces. Current Biology, 11, 880-885.
19. JORDAN, T., & SERGEANT, P. (2000). Effects of distance on visual and audiovisual speech recognition. Language & Speech, 43, 107-124.
20. JOZAN, G. (2001). Analysis of talking faces: Tools for filtering and feature tracking (ATR Tech. Rep. No. TR-HIS-0004, pp. 1-18). Kyoto: ATR Laboratories.
21. KNAPPMEYER, B., THORNTON, I., & BÜLTHOFF, H. (2003). The use of facial motion and facial form during the processing of identity. Vision Research, 43, 1921-1936.
22. KROOS, C., KURATATE, T., & VATIKIOTIS-BATESON, E. (2002). Video-based face motion measurement. Journal of Phonetics, 30, 569-590.
23. LANDER, K., CHRISTIE, F., & BRUCE, V. (1999). The role of movement in the recognition of famous faces. Memory & Cognition, 27, 974-985.
24. LANSING, C. R., & MCCONKIE, G. W. (1999). Attention to facial regions in segmental and prosodic visual speech perception tasks. Journal of Speech, Language, & Hearing Research, 42, 526-538.
25. LEGGE, G. E., PELLI, D. G., RUBIN, G. S., & SCHLESKE, M. M. (1985). Psychophysics of reading: I. Normalization vision. Vision Research, 25, 239-252.
26. LOFFLER, G., & WILSON, H. R. (2001). Detecting shape deformation of moving patterns. Vision Research, 41, 991-1006.
27. LU, Z.-L., & SPERLING, G. (2001). Three-systems theory of human visual motion perception: Review and update. Journal of the Optical Society of America A, 18, 2331-2370.
28. MACDONALD, J., ANDERSEN, S., & BACHMANN, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception, 29, 1155-1168.
29. MAJAJ, N. J., PELLI, D. G., KURSHAN, P., & PALOMARES, M. (2002). The role of spatial frequency channels in letter identification. Vision Research, 42, 1165-1184.
30. MASSARO, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
31. MUNHALL, K. G., GRIBBLE, P., SACCO, L., & WARD, M. (1996). Temporal constraints on the McGurk effect. Perception & Psychophysics, 58, 351-362.
32. MUNHALL, K. G., SERVOS, P., SANTI, A., & GOODALE, M. (2002). Dynamic visual speech perception in a patient with visual form agnosia. NeuroReport, 13, 1793-1796.
33. MUNHALL, K. G., & VATIKIOTIS-BATESON, E. (1998). The moving face during speech communication. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye: Pt. 2. The psychology of speechreading and audiovisual speech (pp. 123-139). London: Taylor & Francis, Psychology Press.
34. NÄSÄNEN, R. (1999). Spatial frequency bandwidth used in the recognition of facial images. Vision Research, 39, 3824-3833.
35. NICHOLSON, K. G., BAUM, S., CUDDY, L. L., & MUNHALL, K. G. (2001). A case of multimodal aprosodia: Impaired auditory and visual speech prosody perception in a patient with right hemisphere damage. In Proceedings of AVSP-01 (pp. 62-65). Scheelsminde, Denmark.
36. OHALA, J. J. (1975). The temporal regulation of speech. In G. Fant & M. A. A. Tatham (Eds.), Auditory analysis and perception of speech (pp. 431-453). London: Academic Press.
37. PARÉ, M., RICHLER, R. C., TEN HOVE, M., & MUNHALL, K. G. (2003). Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect. Perception & Psychophysics, 65, 553-567.
38. PARISH, D. H., & SPERLING, G. (1991). Object spatial frequencies, retinal spatial frequencies, noise, and the efficiency of letter discrimination. Vision Research, 31, 1399-1415.
39. PELLI, D. G., FARELL, B., & MOORE, D. C. (2003). The remarkable inefficiency of word recognition. Nature, 423, 752-756.
40. PRASAD, L., & IYENGAR, S. S. (1997). Wavelet analysis with applications to image processing. Boca Raton, FL: CRC Press.
41. REMEZ, R. E., FELLOWES, J. M., PISONI, D. B., GOH, W. D., & RUBIN, P. E. (1998). Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances. Speech Communication, 26, 65-73.
42. ROSENBLUM, L. D., JOHNSON, J. A., & SALDAÑA, H. M. (1996). Point-light facial displays enhance comprehension of speech in noise. Journal of Speech & Hearing Research, 39, 1159-1170.
43. ROSENBLUM, L. D., & SALDAÑA, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception & Performance, 22, 318-331.
44. SCHYNS, P., & OLIVA, A. (1999). Dr. Angry and Mr. Smile: When categorization flexibly modifies the perception of faces in rapid visual presentations. Cognition, 69, 243-265.
45. SIMONCELLI, E. P., & OLSHAUSEN, B. A. (2001). Natural image statistics and neural representation. Annual Review of Neuroscience, 24, 1193-1216.
46. SOLOMON, J. A., & PELLI, D. G. (1994). The visual filter mediating letter identification. Nature, 369, 395-397.
47. STONE, J. V. (1998). Object recognition using spatiotemporal signatures. Vision Research, 38, 957-951.
48. SUMBY, W. H., & POLLACK, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
49. THOMAS, S. M., & JORDAN, T. R. (2002). Determining the influence of Gaussian blurring on inversion effects with talking faces. Perception & Psychophysics, 64, 932-944.
50. TJAN, B. S., BRAJE, W. L., LEGGE, G. E., & KERSTEN, D. (1995). Human efficiency for recognizing 3-D objëcts 1) objects in luminance noise. Vision Research, 35, 3053-3069.
51. VATIKIOTIS-BATESON, E., EIGSTI, I.-M., YANO, S., & MUNHALL, K. G. (1998). Eye movement of perceivers during audiovisual speech perception. Perception & Psychophysics, 60, 926-940.
52. VATIKIOTIS-BATESON, E., MUNHALL, K. G., HIRAYAMA, M., KASAHARA, Y., & YEHIA, H. (1996). Physiology-based synthesis of audiovisual speech. In Proceedings of the 4th Speech Production Seminar: Models and Data (pp. 241-244). Autrans, France.
53. VITKOVICH, M., & BARBER, P. (1996). Visible speech as a function of image quality: Effects of display parameters on lipreading ability. Applied Cognitive Psychology, 10, 121-140.
54. VUILLEUMIER, P., R, ARMONY, J. L., DRIVER, J., & DOLAN, R. J. (2003). Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nature Neuroscience, 6, 624-631.
55. WALKER, S., BRUCE, V., & O'MALLEY, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception & Psychophysics, 57, 1124-1133.
56. WENGER, M. J., & TOWNSEND, J. T. (2000). Spatial frequencies in short-term memory for faces: A test of three frequency-dependent hypotheses. Memory & Cognition, 28, 125-142.
57. YEHIA, H. C., KURATATE, T., & VATIKIOTIS-BATESON, E. (2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 30, 555-568.
58. YEHIA, H. C., RUBIN, P. E., & VATIKIOTIS-BATESON, E. (1998). Quantitative association of vocal-tract and facial behavior. Speech Communication, 26, 23-44.