Cross-Modal Source Information and Spoken Word Recognition

Journal of Experimental Psychology: Human Perception and Performance

Volumn 30, Issue 2, 2004, Pages 378-396

  Lachs, Lorin ^a   Pisoni, David B ^b,c  

^a CALIFORNIA STATE UNIVERSITY   (United States)

^b INDIANA UNIVERSITY   (United States)

^c INDIANA UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 1842589295     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/0096-1523.30.2.378     Document Type: Article

References (54)

1. Bingham, G. P. (1995). Dynamics and the problem of visual event recognition. In R. F. Port & T. Van Gelder (Eds.), Mind as motion: Explorations in the dynamics of cognition (pp. 403-448). Cambridge, MA: MIT Press.
2. Bingham, G. P., Schmidt, R. C., & Rosenblum, L. D. (1995). Dynamics and the orientation of kinematic forms in visual event recognition. Journal of Experimental Psychology: Human Perception and Performance, 21, 1473-1493.
3. Blesser, B. A. (1969). Perception of spectrally transformed speech. Unpublished doctoral dissertation, Massachusetts Institute of Technology, Cambridge.
4. Blesser, B. (1972). Speech perception under conditions of spectral transformation: I. Phonetic characteristics. Journal of Speech and Hearing Research, 15(1), 1-41.
5. Bradlow, A. R., Torretta, G. M., & Pisoni, D. B. (1996). Intelligibility of normal speech: I. Global and fine-grained acoustic-phonetic talker characteristics. Speech Communication, 20, 255-273.
6. Bricker, P. D., & Pruzansky, S. (1976). Speaker recognition. In N. J. Lass (Ed.), Contemporary issues in experimental phonetics (pp. 295-326). New York: Academic Press.
7. Browman, C. P., & Goldstein, L. (1995). Dynamics and articulatory phonology. In R. F. Port & T. Van Gelder (Eds.), Mind as motion. Cambridge, MA: MIT Press.
8. Bruce, V. (1988). Recognising faces. Hove, United Kingdom: Erlbaum.
9. Creelman, C. D. (1957). Case of the unknown talker. Journal of the Acoustical Society of America, 29, 655.
10. Fourcin, A. J. (1968). Speech-source interference. IEEE Transactions in Audio Electroacoustics, ACC-16, 65-67.
11. Fowler, C. A. (1996). Listeners do hear sounds, not tongues. Journal of the Acoustical Society of America, 99, 1730-1741.
12. Fowler, C. A., & Galantucci, B. (in press). The relation of speech perception and speech production. In D. B. Pisoni & R. E. Remez (Eds.), Handbook of speech perception. Oxford, United Kingdom: Blackwell.
13. Gaver, W. W. (1993). What in the world do we hear?: An ecological approach to auditory event perception. Ecological Psychology, 5(1), 1-29.
14. Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin.
15. Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251-279.
16. Grant, K. W. (2001). The effect of speechreading for masked detection thresholds for filtered speech. Journal of the Acoustical Society of America, 109, 2272-2275.
17. Green, K. P. (1996). The use of auditory and visual information in phonetic perception. In D. G. Stork & M. E. Hennecke (Eds.), Speechreading by humans and machines. Berlin, Germany: Springer-Verlag.
18. Green, K. P., & Gerdeman, A. (1995). Cross-modal discrepancies in coarticulation and the integration of speech information: The McGurk effect with mismatched vowels. Journal of Experimental Psychology: Human Perception and Performance, 21, 1409-1426.
19. Harnsberger, J. D., Svirsky, M. A., Kaiser, A. R., Pisoni, D. B., Wright, R., & Meyer, T. A. (2001). Perceptual "vowel spaces" of cochlear implant users: Implications for the study of auditory adaptation to spectral shift. Journal of the Acoustical Society of America, 109, 2135-2145.
20. Hirahara, T., & Kato, H. (1992). The effects of F0 on vowel identification. In Y. Tohkura, E. Vatikiotis-Bateson, & Y. Sagisaka (Eds.), Speech perception, production, and linguistic structure (pp. 89-112). Tokyo: Ohmsha.
21. Hollien, H., & Klepper, B. (1984). The speaker recognition problem. Advances in Forensic Psychology & Psychiatry, 1, 87-111.
22. Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception and Psychophysics, 14, 201-211.
23. Johnson, K., Flemming, E., & Wright, R. (1993). The hyperspace effect: Phonetic targets are hyperarticulated. Language, 69, 505-528.
24. Kawahara, H. (1997, April). Speech representation and transformation using adaptive interpolation of weighted spectrum: Vocoder revisited. Paper presented at the Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Munich, Germany.
25. Kawahara, H., Katayose, H., de Cheveigné, A., & Patterson, R. D. (1999, September). Fixed points analysis of frequency to instantaneous frequency mapping for accurate estimation of F0 and periodicity. Paper presented at EUROSPEECH '99, Budapest, Hungary.
26. Kawahara, H., Masuda-Kastuse, I., & Cheveigné, A. (1999). Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds. Speech Communication, 27, 187-207.
27. Lachs, L. (2002). Vocal tract kinematics and crossmodal information. Unpublished doctoral dissertation, Indiana University, Bloomington.
28. Lachs, L., & Hernández, L. R. (1998). Update: The Hoosier Audiovisual Multitalker Database. In Research on spoken language processing progress report (No. 22, pp. 377-388). Bloomington, IN: Speech Research Laboratory, Indiana University.
29. Lachs, L., McMichael, K., & Pisoni, D. B. (2003). Speech perception and implicit memory: Evidence for detailed episodic encoding. In J. Bowers & C. Marsolek (Eds.), Rethinking implicit memory (pp. 215-239). Oxford, United Kingdom: Oxford University Press.
30. Lachs, L., & Pisoni, D. B. (in press). Crossmodal source identification in speech perception. Ecological Psychology.
31. Löfqvist, A., & Gracco, V. L. (1997). Lip and jaw kinematics in bilabial stop consonant production. Journal of Speech, Language, and Hearing Research, 40, 877-893.
32. Lucero, J. C., & Munhall, K. G. (1999). A model of facial biomechanics for speech production. Journal of the Acoustical Society of America, 106, 2834-2842.
33. Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
34. McGurk, H., & MacDonald, J. (1976, December 23). Hearing lips and seeing voices. Nature, 264, 746-748.
35. Mullennix, J. W., & Pisoni, D. B. (1990). Stimulus variability and processing dependencies in speech perception. Perception & Psychophysics, 47, 379-390.
36. Mullennix, J. W., Pisoni, D. B., & Martin, C. S. (1989). Some effects of talker variability on spoken word recognition. Journal of the Acoustical Society of America, 85, 365-378.
37. Pisoni, D. B. (1993). Long-term memory in speech perception: Some new findings on talker variability, speaking rate and perceptual learning, Speech Communication, 13, 109-125.
38. Pisoni, D. B. (1997). Some thoughts on "normalization" in speech perception. In K. Johnson & J. W. Mullennix (Eds.), Talker variability in speech processing (pp. 9-32). San Diego, CA: Academic Press.
39. Remez, R. E., Fellowes, J. M., & Rubin, P. E. (1997). Talker identification based on phonetic information. Journal of Experimental Psychology: Human Perception and Performance, 23, 651-666.
40. Remez, R. E., Rubin, P. E., Berns, S. M., Pardo, J. S., & Lang, J. M. (1994). On the perceptual organization of speech. Psychological Review, 101, 129-156.
41. Remez, R. E., Rubin, P. E., Pisoni, D. B., & Carrell, T. D. (1981, May 22). Speech perception without traditional speech cues. Science, 212, 947-950.
42. Rosenblum, L. D. (1994). How special is audiovisual speech integration? Current Psychology of Cognition, 13(1), 110-116.
43. Rosenblum, L. D. (in press). The primacy of multimodal speech perception. In D. B. Pisoni & R. E. Remez (Eds.), Handbook of speech perception, Oxford, United Kingdom: Blackwell.
44. Rosenblum, L. D., Johnson, J. A., & Saldaña, H. M. (1996). Point-light facial displays enhance comprehension of speech in noise. Journal of Speech, Language, and Hearing Research, 39, 1159-1170.
45. Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nodarse, B. C., & Niehaus, R. P. (2002). Visual speech information for face recognition. Perception & Psychophysics, 64(2), 220-229.
46. Runeson, S., & Frykholm, G. (1983). Kinematic specification of dynamics as an informational basis for person-and-action perception: Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112, 585-615.
47. Sheffert, S. M., Lachs, L., & Hernández, L. R. (1996). The Hoosier Audiovisual Multitalker Database. In Research on spoken language processing progress report (No. 21, pp. 578-583). Bloomington, IN: Speech Research Laboratory, Indiana University.
48. Smith, Z. M., Delgutte, B., & Oxenham, A. J. (2002, March 7). Chimaeric sounds reveal dichotomies in auditory perception. Nature, 416, 87-90.
49. Sumby, W. H., & Pollack, I. (1954). Visual contribution of speech intel-ligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
50. Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). Hillsdale, NJ: Erlbaum.
51. Vatikiotis-Bateson, E., Munhall, K. G., Hirayama, M., Lee, Y. V., & Terzepoulos, D. (1997). The dynamics of audiovisual behavior in speech. In D. G. Stork & M. E. Hennecke (Eds.), Speechreading by humans and machines (pp. 221-232). Berlin, Germany: Springer-Verlag.
52. Vatikiotis-Bateson, E., Munhall, K. G., Kasahara, Y., Garcia, F., & Yehia, H. (1996, October). Characterizing audiovisual information during speech. Paper presented at the International Conference on Spoken Language Processing, Philadelphia, PA.
53. Verbrugge, R. R., Strange, W., Shankweiler, D. P., & Edman, T. R. (1976). What information enables a listener to map a talker's vowel space? Journal of the Acoustical Society of America, 60, 198-212.
54. Yehia, H., Kuratate, T., & Vatikiotis-Bateson, E. (2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 30, 555-568.