Perception of the multisensory coherence of fluent audiovisual speech in infancy: Its emergence and the role of experience

Journal of Experimental Child Psychology

Volumn 130, Issue , 2015, Pages 147-162

  Lewkowicz, David J ^a   Minar, Nicholas J ^b   Tift, Amy H ^b   Brandon, Melissa ^b  

^a NORTHEASTERN UNIVERSITY   (United States)

^b FLORIDA ATLANTIC UNIVERSITY   (United States)

Author keywords

Auditory; Development; Infancy; Multisensory perception; Speech; Visual

Indexed keywords

ASSOCIATION; AUDITORY STIMULATION; CHILD DEVELOPMENT; FACE; FEMALE; HUMAN; INFANT; LANGUAGE; MALE; PHOTOSTIMULATION; PROCEDURES; SENSE OF COHERENCE; SPEECH PERCEPTION; VISION;

ACOUSTIC STIMULATION; CHILD DEVELOPMENT; CUES; FACE; FEMALE; HUMANS; INFANT; LANGUAGE; MALE; PHOTIC STIMULATION; SENSE OF COHERENCE; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 84910024652     PISSN: 00220965     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jecp.2014.10.006     Document Type: Article

References (53)

1. Bahrick L.E. Infants' perception of substance and temporal synchrony in multimodal events. Infant Behavior and Development 1983, 6:429-451.
2. Bahrick L.E., Hernandez-Reif M., Flom R. The development of infant learning about specific face-voice relations. Developmental Psychology 2005, 41:541-552.
3. Bahrick L.E., Moss L., Fadil C. Development of visual self-recognition in infancy. Ecological Psychology 1996, 8:189-208.
4. Bahrick L.E., Netto D., Hernandez-Reif M. Intermodal perception of adult and child faces and voices by infants. Child Development 1998, 69:1263-1275.
5. Bremner A.J., Lewkowicz D.J., Spence C. Multisensory development 2012, Oxford University Press, Oxford, UK.
6. Brookes H., Slater A., Quinn P.C., Lewkowicz D.J., Hayes R., Brown E. Three-month-old infants learn arbitrary auditory-visual pairings between voices and faces. Infant and Child Development 2001, 10:75-82.
7. Chandrasekaran C., Trubanova A., Stillittano S., Caplier A., Ghazanfar A.A. The natural statistics of audiovisual speech. PLoS Computational Biology 2009, 5(7):e1000436.
8. Dixon N.F., Spitz L.T. The detection of auditory visual desynchrony. Perception 1980, 9:719-721.
9. Dodd B. Lip reading in infants: Attention to speech presented in- and out-of-synchrony. Cognitive Psychology 1979, 11:478-484.
10. Dodd B., Burnham D. Processing speechread information. Volta Review 1988, 90:45-60.
11. Fernald A. Intonation and communicative intent in mothers' speech to infants: Is the melody the message?. Child Development 1989, 60:1497-1510.
12. Gibson E.J. Principles of perceptual learning and development 1969, Appleton, New York.
13. Grant K.W., Seitz P.F. Measures of auditory-visual integration in nonsense syllables and sentences. Journal of the Acoustical Society of America 1998, 104:2438-2450.
14. Grant K.W., van Wassenhove V., Poeppel D. Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony. Speech Communication 2004, 44:43-53.
15. Hillock-Dunn A., Wallace M.T. Developmental changes in the multisensory temporal binding window persist into adolescence. Developmental Science 2012, 15:688-696.
16. Hunnius S., Geuze R.H. Developmental changes in visual scanning of dynamic faces and abstract stimuli in infants: A longitudinal study. Infancy 2004, 6:231-255.
17. Jesse A., Johnson E.K. Prosodic temporal alignment of co-speech gestures to speech facilitates referent resolution. Journal of Experimental Psychology: Human Perception and Performance 2012, 38:1567-1581.
18. Kubicek C., de Boisferon A.H., Dupierrix E., Pascalis O., Lœvenbruck H., Gervain J., et al. Cross-modal matching of audio-visual German and French fluent speech in infancy. PLoS ONE 2014, 9:e89275.
19. Kuhl P.K., Meltzoff A.N. The bimodal perception of speech in infancy. Science 1982, 218:1138-1141.
20. Lewkowicz D.J. Developmental changes in infants' bisensory response to synchronous durations. Infant Behavior and Development 1986, 9:335-353.
21. Lewkowicz D.J. Infants' response to temporally based intersensory equivalence: The effect of synchronous sounds on visual preferences for moving stimuli. Infant Behavior and Development 1992, 15:297-324.
22. Lewkowicz D.J. Infants' responsiveness to the auditory and visual attributes of a sounding/moving stimulus. Perception & Psychophysics 1992, 52:519-528.
23. Lewkowicz D.J. Infants' response to the audible and visible properties of the human face: I. Role of lexical-syntactic content, temporal synchrony, gender, and manner of speech. Developmental Psychology 1996, 32:347-366.
24. Lewkowicz D.J. Perception of auditory-visual temporal synchrony in human infants. Journal of Experimental Psychology: Human Perception and Performance 1996, 22:1094-1106.
25. Lewkowicz D.J. Infants' perception of the audible, visible, and bimodal attributes of multimodal syllables. Child Development 2000, 71:1241-1257.
26. Lewkowicz D.J. Infant perception of audio-visual speech synchrony. Developmental Psychology 2010, 46:66-77.
27. Lewkowicz D.J. Early experience and multisensory perceptual narrowing. Developmental Psychobiology 2014, 56:292-315.
28. Lewkowicz D.J., Flom R. The audio-visual temporal binding window narrows in early childhood. Child Development 2014, 85:685-694.
29. Lewkowicz D.J., Ghazanfar A.A. The decline of cross-species intersensory perception in human infants. Proceedings of the National Academy of Sciences of the United States of America 2006, 103:6771-6774.
30. Lewkowicz D.J., Ghazanfar A.A. The emergence of multisensory systems through perceptual narrowing. Trends in Cognitive Sciences 2009, 13:470-478.
31. Lewkowicz D.J., Hansen-Tift A.M. Infants deploy selective attention to the mouth of a talking face when learning speech. Proceedings of the National Academy of Sciences of the United States of America 2012, 109:1431-1436.
32. Lewkowicz D.J., Leo I., Simion F. Intersensory perception at birth: Newborns match non-human primate faces and voices. Infancy 2010, 15:46-60.
33. Lewkowicz D.J., Pons F. Recognition of amodal language identity emerges in infancy. International Journal of Behavioral Development 2013, 37(2):90-94.
34. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:229-239.
35. Munhall K.G., Vatikiotis-Bateson E. Spatial and temporal constraints on audiovisual speech perception. The handbook of multisensory processes 2004, 177-188. MIT Press, Cambridge, MA. G.A. Calvert, C. Spence, B.E. Stein (Eds.).
36. Patterson M.L., Werker J.F. Matching phonetic information in lips and voice is robust in 4.5-month-old infants. Infant Behavior & Development 1999, 22:237-247.
37. Patterson M.L., Werker J.F. Infants' ability to match dynamic phonetic and gender information in the face and voice. Journal of Experimental Child Psychology 2002, 81:93-115.
38. Patterson M.L., Werker J.F. Two-month-old infants match phonetic information in lips and voice. Developmental Science 2003, 6:191-196.
39. Piaget J. The origins of intelligence in children 1952, International Universities Press, New York.
40. Pons F., Lewkowicz D.J. Infant perception of audio-visual speech synchrony in familiar and unfamiliar fluent speech. Acta Psychologica 2014, 149:142-147.
41. Pons F., Lewkowicz D.J., Soto-Faraco S., Sebastián-Gallés N. Narrowing of intersensory speech perception in infancy. Proceedings of the National Academy of Sciences of the United States of America 2009, 106:10598-10602.
42. Rosenblum L.D. Speech perception as a multimodal phenomenon. Current Directions in Psychological Science 2008, 17:405-409.
43. Saffran J.R., Werker J.F., Werner L.A. The infant's auditory world: Hearing, speech, and the beginnings of language. Handbook of child psychology: Cognition, perception, and language 2006, Vol. 2:58-108. John Wiley, Hoboken, NJ. 6th ed.
44. Scott L.S., Pascalis O., Nelson C.A. A domain general theory of the development of perceptual discrimination. Current Directions in Psychological Science 2007, 16:197-201.
45. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 1954, 26:212-215.
46. Summerfield A.Q. Use of visual information in phonetic perception. Phonetica 1979, 36:314-331.
47. Thelen E., Smith L.B. A dynamic systems approach to the development of cognition and action 1994, MIT Press, Cambridge, MA.
48. Walker-Andrews A.S. Intermodal perception of expressive behaviors: Relation of eye and voice?. Developmental Psychology 1986, 22:373-377.
49. Walker-Andrews A.S., Bahrick L.E., Raglioni S.S., Diaz I. Infants' bimodal perception of gender. Ecological Psychology 1991, 3:55-75.
50. Walton G.E., Bower T.G. Amodal representations of speech in infants. Infant Behavior and Development 1993, 16:233-243.
51. Werker J.F., Tees R.C. Speech perception as a window for understanding plasticity and commitment in language systems of the brain. Developmental Psychobiology 2005, 46:233-234.
52. Werker J.F., Yeung H.H., Yoshida K.A. How do infants become experts at native-speech perception?. Current Directions in Psychological Science 2012, 21:221-226.
53. Yehia H., Rubin P., Vatikiotis-Bateson E. Quantitative association of vocal-tract and facial behavior. Speech Communication 1998, 26:23-43.