Degrading phonetic information affects matching of audiovisual speech in adults, but not in infants

Cognition

Volumn 130, Issue 1, 2014, Pages 31-43

  Baart, Martijn ^a,b   Vroomen, Jean ^c   Shaw, Kathleen ^c   Bortfeld, Heather ^c,d  

^a BASQUE CENTER ON COGNITION   (Spain)

^b TILBURG UNIVERSITY   (Netherlands)

^c UNIVERSITY OF CONNECTICUT   (United States)

^d HASKINS LABORATORIES   (United States)

Author keywords

Adults; Audiovisual speech integration; Infants; Phonetic correspondence; Sine wave speech

Indexed keywords

ADULT; ARTICLE; ASSOCIATION; CHILD; DEVELOPMENT; FACE; FEMALE; HUMAN; INFANT; INFORMATION; MALE; PERFORMANCE; PHONETICS; PRIORITY JOURNAL; SINE WAVE SPEECH; SOUND; SPEECH; COMPARATIVE STUDY; LANGUAGE DEVELOPMENT; LIP READING; PHYSIOLOGY; SPEECH PERCEPTION; VISION; YOUNG ADULT;

ADULT; CUES; FACE; FEMALE; HUMANS; INFANT; LANGUAGE DEVELOPMENT; LIPREADING; MALE; PHONETICS; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84886904344     PISSN: 00100277     EISSN: 18737838     Source Type: Journal    
DOI: 10.1016/j.cognition.2013.09.006     Document Type: Article

References (79)

1. Bahrick L.E. Infants' perception of substance and temporal synchrony in multimodal events. Infant Behavior and Development 1983, 6(4):429-451.
2. Bahrick L.E. Infants' intermodal perception of two levels of temporal structure in natural events. Infant Behavior and Development 1987, 10:387-416.
3. Boersma, P., & Weenink, K. (2005). Praat: Doing phonetics by computer. <>. http://www.fon.hum.uva.nl/praat.
4. Bortfeld H., Fava E., Boas D.A. Identifying cortical lateralization of speech processing in infants using near-infrared spectroscopy. Developmental Neuropsychology 2009, 34(1):52-65.
5. Bortfeld H., Wruck E., Boas D.A. Assessing infants' cortical response to speech using near-infrared spectroscopy. Neuroimage 2007, 34(1):407-415.
6. Bristow D., Dehaene-Lamberts G., Mattout J., Soares C., Gliga T., Baillet S., et al. Hearing faces: How the infant brain matches the face it sees with the speech it hears. Journal of Cognitive Neuroscience 2008, 21(5):905-921.
7. Bruce V., Campbell R.N., Doherty-Sneddon G., Import A., Langton S., McAuley S., et al. Testing face processing skills in children. British Journal of Development Psychology 2000, 18:319-333.
8. Burnham D., Dodd B. Auditory-visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect. Developmental Psychobiology 2004, 45(4):204-220.
9. Burnham D., Dodd B. Auditory-visual speech perception as a direct process: The McGurk effect in human infants and across languages. Speechreading by humans and machines 1996, 103-114. Springer-Verlag, Berlin. D.G. Stork, M.E. Hennecke (Eds.).
10. Burnham D., Sekiyama K. When auditory-visual speech perception develops: The locus of the Japanese McGurk effect. Australian Journal of Psychology 2004, 56:108.
11. Desjardins R.N., Rogers J., Werker J.F. An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks. Journal of Experimental Child Psychology 1997, 66:85-110.
12. Desjardins R.N., Werker J.F. Is the integration of heard and seen speech mandatory for infants?. Developmental Psychobiology 2004, 45:187-203.
13. Dodd B. Lip reading in infants: Attention to speech presented in- and out-of-synchrony. Cognitive Psychology 1979, 11(4):478-484.
14. Dodd B., McIntosh B., Erdener D., Burnham D. Perception of the auditory-visual illusion in speech perception by children with phonological disorders. Clinical Linguistics & Phonetics 2008, 22(1):69-82.
15. Eimas P.D., Siqueland E.R., Jusczyk P., Vigorito J. Speech perception in infants. Science 1971, 171:303-306.
16. Eskelund K., Tuomainen J., Andersen T.S. Multistage audiovisual integration of speech: Dissociating identification and detection. Experimental Brain Research 2011, 208(3):447-457.
17. Fant G. Acoustic theory of speech production 1960, Mouton, The Hague.
18. Grant K.W. The effect of speechreading on masked detection thresholds for filtered speech. Journal of the Acoustical Society of America 2001, 109(5):2272-2275.
19. Grant K.W., Seitz P.F. The use of visible speech cues for improving auditory detection of spoken sentences. Journal of the Acoustical Society of America 2000, 108(3):1197-1208.
20. Grant K.W., van Wassenhove V., Poeppel D. Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony. Speech Communication 2004, 44(1-4):43-53.
21. Hockley N.S., Polka L.A. A developmental study of audiovisual speech perception using the McGurk paradigm. Journal of the Acoustical Society of America 1994, 96(5):3309.
22. Hollich G., Newman R.S., Jusczyk P.W. Infants' use of synchronized visual information to separate streams of speech. Child Development 2005, 76(3):598-613.
23. Homae F., Watanabe H., Nakano T., Taga G. Functional development in the infant brain for auditory pitch processing. Human Brain Mapping 2012, 33(3):596-608.
24. Homae, F., Watanabe, H., & Taga, G. (2013). Speech and sine wave speech processing in the infant brain, Poster presented at the Workshop on Infant Language Development (WILD). June 20-22, Donostia - San Sebastián, Spain.
25. Hyde D.C., Jones B.L., Flom R., Porter C.L. Neural signatures of face-voice synchrony in 5-month-old human infants. Developmental Psychobiology 2011, 53(4):359-370.
26. Jeffers J., Barley M. Speechreading (lipreading) 1971, Charles C. Thomas, Springfield, Illinois.
27. Jusczyk P.W., Luce P.A. Infants' sensitivity to phonotactic patterns in the native language. Journal of Memory and Language 1994, 33(5):630-645.
28. Klucharev V., Möttönen R., Sams M. Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Cognitive Brain Research 2003, 18(1):65-75.
29. Kotilahti K., Nissilä I., Näsi T., Lipiäinen L., Noponen T., Meriläinen P., et al. Hemodynamic responses to speech and music in newborn infants. Human Brain Mapping 2010, 31(4):595-603.
30. Kuhl P.K. Human adults and human infants show a "perceptual magnet effect" for the prototypes of speech categories, monkeys do not. Perception & Psychophysics 1991, 50(2):93-107.
31. Kuhl P.K., Meltzoff A.N. The bimodal perception of speech in infancy. Science 1982, 218:1138-1141.
32. Kuhl P.K., Meltzoff A.N. The intermodal representation of speech in infants. Infant Behavior and Development 1984, 7(3):361-381.
33. Kuhl P.K., Meltzoff A.N. Infant vocalizations in response to speech: Vocal imitation and developmental change. Journal of the Acoustical Society of America 1996, 100(4):2425-2438.
34. Kuhl P.K., Stevens E., Hayashi A., Deguchi T., Kiritani S., Iverson P. Infants show a facilitation effect for native language phonetic perception between 6 and 12months. Developmental Science 2006, 9(2):F13-F21.
35. Kuhl P.K., Williams K.A., Lacerda F., Stevens K.N., Lindblom B. Linguistic experience alters phonetic perception in infants by 6months of age. Science 1992, 255(5044):606-608.
36. Kuhl P.K., Williams K.A., Meltzoff A.N. Cross-modal speech perception in adults and infants using nonspeech auditory stimuli. Journal of Experimental Psychology: Human Perception and Performance 1991, 17(3):829-840.
37. Kushnerenko E., Teinonen T., Volein A., Csibra G. Electrophysiological evidence of illusory audiovisual speech percept in human infants. Proceedings of the National academy of Sciences of the United States of America 2008, 105(32):11442-11445.
38. Lewkowicz D.J. Developmental changes in infants' bisensory response to synchronous durations. Infant Behavior and Development 1986, 9:335-353.
39. Lewkowicz D.J. Perception of auditory-visual temporal synchrony in human infants. Journal of Experimental Psychology; Human Perception and Performance 1996, 25(5):1094-1106.
40. Lewkowicz D.J. Infants' perception of the audible, visible, and bimodal attributes of multimodal syllables. Child Development 2000, 71(5):1241-1257.
41. Lewkowicz D.J. Learning and discrimination of audiovisual events in human infants: the hierarchical relation between intersensory temporal synchrony and rhythmic pattern cues. Developmental Psychology 2003, 39(5):795-804.
42. Lewkowicz D.J. Infant perception of audio-visual speech synchrony. Developmental Psychology 2010, 46(1):66-77.
43. Lieberman P., Blumstein S. Speech physiology, speech perception, and acoustic phonetics 1988, Cambridge University Press, Cambridge.
44. Massaro D.W. Children's perception of visual and auditory speech. Child Development 1984, 55:1777-1788.
45. Massaro D.W. Speech perception by ear and eye: A paradigm for psychological inquiry 1987, Lawrence Erlbaum Associates Inc, Hillsdale, NJ.
46. Massaro D.W. Perceiving Talking Faces: From Speech Perception to a Behavioral Principle 1998, The MIT Press, Cambridge.
47. Massaro D.W., Cohen M.M., Campbell C.S., Rodriguez T. Bayes factor of model selection validates FLMP. Psychonomic Bulletin & Review 2001, 8:1-17.
48. Massaro D.W., Thompson L.A., Barron B., Laren E. Developmental changes in visual and auditory contributions to speech perception. Journal of Experimental Child Psychology 1986, 41:93-113.
49. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
50. Minagawa-Kawai Y., van der Lely H., Ramus F., Sato Y., Mazuka R., Dupoux E. Optical brain imaging reveals general auditory and language-specific processing in early infant development. Cerebral Cortex 2011, 21:254-261.
51. Parise C.V., Spence C., Ernst M.O. When correlation implies causation in multisensory integration. Current Biology 2012, 22(1):46-49.
52. Patterson M.L., Werker J.F. Matching phonetic information in lips and voice is robust in 4.5-month-old infants. Infant Behavior and Development 1999, 22:237-247.
53. Patterson M.L., Werker J.F. Two-month-old infants match phonetic information in lips and voice. Developmental Science 2003, 6(2):191-196.
54. Peña M., Maki A., Kovacic D., Dehaene-Lambertz G., Koizumi H., Bouquet F., et al. Sounds and silence: An optical topography study of language recognition at birth. Proceedings of the National academy of Sciences of the United States of America 2003, 100:11702-11705.
55. Pons F., Teixidó M., Garcia-Morera J., Navarra J. Short-term experience increases infants' sensitivity to audiovisual asynchrony. Infant Behavior and Development 2012, 35:815-818.
56. Pons F., Lewkowicz D.J., Soto-Faraco S., Sebastián-Gallés N. Narrowing of intersensory speech perception in infancy. Proceedings of the National Academy of Sciences of the United States of America 2009, 106(26):10598-10602.
57. Rosen S. Temporal information in speech: acoustic, auditory and linguistic aspects. Philosophical Transactions of the Royal Society B: Biological Sciences 1992, 336(1278):367-373.
58. Rosen S., Iverson P. Constructing adequate non-speech analogues: what is special about speech anyway?. Developmental Science 2007, 10(2):165-168.
59. Remez R.E., Rubin P.E., Pisoni D.B., Carrell T.D. Speech perception without traditional speech cues. Science 1981, 212:947-949.
60. Rosenblum L.D., Schmuckler M.A., Johnson J.A. The McGurk effect in infants. Perception & Psychophysics 1997, 59:347-357.
61. Ross L.A., Molholm S., Blanco D., Gomez-Ramirez M., Saint-Amour D., Foxe J.J. The development of multisensory speech perception continues into the late childhood years. European Journal of Neuroscience 2011, 33(12):2329-2337.
62. Schwartz J.L., Berthommier F., Savariaux C. Seeing to hear better: Evidence for early audio-visual interactions in speech identification. Cognition 2004, 93(2):B69-78.
63. Sekiyama, K., & Burnham, D. (2004). Issues in the development of auditory-visual speech perception: Adults, infants, and children. Paper presented at the 8th International Conference on Spoken Language Processing, Jeiu Island, Korea.
64. Stekelenburg J.J., Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience 2007, 19(12):1964-1973.
65. Stekelenburg J.J., Vroomen J. Electrophysiological correlates of predictive coding of auditory location in the perception of natural audiovisual events. Frontiers in Integrative Neuroscience 2012, 6.
66. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 1954, 26:212-215.
67. Swingley D., Pinto J.P., Fernald A. Continuous processing in word recognition at 24months. Cognition 1999, 71:73-108.
68. Teinonen T., Aslin R.N., Alku P., Csibra G. Visual speech contributes to phonetic learning in 6-month-old infants. Cognition 2008, 108(3):850-855.
69. Telkemeyer S., Rossi S., Koch S.P., Nierhaus T., Steinbrink J., Poeppel D., et al. Sensitivity of newborn auditory cortex to the temporal structure of sounds. The Journal of Neuroscience 2009, 29(47):14726-14733.
70. Tuomainen J., Andersen T.S., Tiippana K., Sams M. Audio-visual speech perception is special. Cognition 2005, 96(1):B13-22.
71. van Wassenhove V., Grant K.W., Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America 2005, 102(4):1181-1186.
72. van Wassenhove V., Grant K.W., Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 2007, 45(3):598-607.
73. Vatakis A., Spence C. Audiovisual synchrony perception for music, speech, and object actions. Brain Research 2006, 1111(1):134-142.
74. Vouloumanos A., Werker J.F. Tuned to the signal: the privileged status of speech for young infants. Developmental Science 2004, 7(3):270-276.
75. Vouloumanos A., Werker J.F. Listening to language at birth: evidence for a bias for speech in neonates. Developmental Science 2007, 10(2):159-171.
76. Vroomen J., Baart M. Phonetic recalibration only occurs in speech mode. Cognition 2009, 110(2):254-259.
77. Vroomen J., Stekelenburg J.J. Visual anticipatory information modulates multisensory interactions of artificial audiovisual stimuli. Journal of Cognitive Neuroscience 2010, 22:1583-1596.
78. Vroomen J., Stekelenburg J.J. Perception of intersensory synchrony in audiovisual speech: Not that special. Cognition 2011, 118(1):75-83.
79. Werker J.F., Tees R.C. Cross-language speech perception: Evidence for perceptual reorganization during the first year of life. Infant Behavior and Development 1984, 7:49-63.