Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour

European Journal of Neuroscience

Volumn 38, Issue 9, 2013, Pages 3363-3369

  Kushnerenko, Elena ^a   Tomalski, Przemyslaw ^a,b   Ballieux, Haiko ^a   Ribeiro, Helena ^a   Potton, Anita ^a   Axelsson, Emma L ^a   Murphy, Elizabeth ^a   Moore, Derek G ^a  

^a UNIVERSITY OF EAST LONDON   (United Kingdom)

^b UNIVERSITY OF WARSAW   (Poland)

Author keywords

Audiovisual integration; Audiovisual mismatch response; Event related potentials (ERPs); Eye tracking; McGurk illusion

Indexed keywords

ACTION POTENTIAL; AGE; ARTICLE; AUDIOVISUAL MISMATCH RESPONSE; AUDIOVISUAL SPEECH; BEHAVIOR; BIRTH WEIGHT; BRAIN FUNCTION; CONTROLLED STUDY; ELECTROPHYSIOLOGY; EVENT RELATED POTENTIAL; EYE MOVEMENT; EYE TRACKING; FEMALE; GESTATIONAL AGE; HABITUATION; HUMAN; HUMAN EXPERIMENT; INFANT; MALE; NERVOUS SYSTEM PARAMETERS; NORMAL HUMAN; PRIORITY JOURNAL; SPEECH; SPEECH ARTICULATION;

AUDIOVISUAL INTEGRATION; AUDIOVISUAL MISMATCH RESPONSE; EVENT-RELATED POTENTIALS (ERPS); EYE-TRACKING; MCGURK ILLUSION;

AGE FACTORS; BRAIN; EVOKED POTENTIALS; EYE MOVEMENTS; FEMALE; HUMANS; INFANT; MALE; PSYCHOMOTOR PERFORMANCE; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 84887195912     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/ejn.12317     Document Type: Article

References (31)

1. Alsius, A., Navarra, J., Campbell, R. & Soto-Faraco, S. (2005) Audiovisual integration of speech falters under high attention demands. Curr. Biol., 15, 839-843.
2. Bristow, D., Dehaene-Lambertz, G., Mattout, J., Soares, C., Gliga, T., Baillet, S. & Mangin, J.-F.F. (2009) Hearing faces: how the infant brain matches the face it sees with the speech it hears. J. Cognitive Neurosci., 21, 905-921.
3. Buchan, J.N. & Munhall, K.G. (2011) The influence of selective attention to auditory and visual speech on the integration of audiovisual speech information. Perception, 40, 1164-1182.
4. Burnham, D. & Dodd, B. (2004) Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. Dev. Psychobiol., 45, 204-220.
5. Dehaene-Lambertz, G. & Dehaene, S. (1994) Speed and cerebral correlates of syllable discrimination in infants. Nature, 28, 293-294.
6. Desjardins, R.N. & Werker, J.F. (2004) Is the integration of heard and seen speech mandatory for infants? Dev. Psychobiol., 45, 187-203.
7. Elsabbagh, M., Volein, A., Csibra, G., Holmboe, K., Garwood, H., Tucker, L., Krljes, S., Baron-Cohen, S., Bolton, P., Charman, T., Baird, G. & Johnson, M.H. (2009) Neural correlates of eye gaze processing in the infant broader autism phenotype. Biol. Psychiat., 65, 31-38.
8. Farroni, T., Csibra, G., Simion, F. & Johnson, M.H. (2002) Eye contact detection in humans from birth. Proc. Natl. Acad. Sci. USA, 99, 9602-9605.
9. Friederici, A.D., Friedrich, M. & Christophe, A. (2007) Brain responses in 4-month-old infants are already language specific. Curr. Biol., 17, 1208-1211.
10. Grice, S.J., Halit, H., Farroni, T., Baron-Cohen, S., Bolton, P. & Johnson, M.H. (2005) Neural correlates of eye-gaze detection in young children with autism. Cortex, 41, 342-353.
11. Guiraud, J.A., Kushnerenko, E., Tomalski, P., Davies, K., Ribeiro, H., Johnson, M.H. & Team, T.B. (2011) Differential habituation to repeated sounds in infants at high risk for autism. NeuroReport, 22, 845-849.
12. Hessler, D., Jonkers, R., Stowe, L. & Bastiaanse, R. (2013) The whole is more than the sum of its parts - Audiovisual processing of phonemes investigated with ERPs. Brain Lang., 124, 213-224.
13. Hillis, J.M., Ernst, M.O., Banks, M.S. & Landy, M.S. (2002) Combining sensory information: mandatory fusion within, but not between, senses. Science, 298, 1627-1630.
14. Jääskeläinen, I.P., Ojanen, V., Ahveninen, J., Auranen, T., Levänen, S., Möttönen, R., Tarnanen, I. & Sams, M. (2004) Adaptation of neuromagnetic N1 responses to phonetic stimuli by visual speech in humans. NeuroReport, 15, 2741-2744.
15. Kushnerenko, E., Čeponien{approaches the limit}, R., Balan, P., Fellman, V., Näätänen, R. & Huotilainen, M. (2002a) Maturation of the auditory event-related potentials during the 1st year of life. NeuroReport, 13, 16-22.
16. Kushnerenko, E., Čeponien, R., Balan, P., Fellman, V., Näätänen, R. & Huotilainen, M. (2002b) Maturation of the auditory change-detection response in infants: a longitudinal ERP study. NeuroReport, 13, 1843-1848.
17. Kushnerenko, E., Winkler, I., Horváth, J., Näätänen, R., Pavlov, I., Fellman, V. & Huotilainen, M. (2007) Processing acoustic change and novelty in newborn infants. Eur. J. Neurosci., 26, 265-274.
18. Kushnerenko, E., Teinonen, T., Volein, A. & Csibra, G. (2008) Electrophysiological evidence of illusory audiovisual speech percept in human infants. Proc. Natl. Acad. Sci. USA, 105, 11442-11445.
19. Lewkowicz, D.J. & Hansen-Tift, A.M.C.-P. (2012) Infants deploy selective attention to the mouth of a talking face when learning speech. Proc. Natl. Acad. Sci. USA, 109, 1431-1436.
20. McGurk, H. & MacDonald, J. (1976) Hearing lips and seing voices. Nature, 264, 746-748.
21. Morr, M.L., Shafer, V.L., Kreuzer, J.A. & Kurtzberg, D. (2002) Maturation of mismatch negativity in typically developing infants and preschool children. Ear Hearing, 23, 118-136.
22. Nardini, M., Bedford, R. & Mareschal, D. (2010) Fusion of visual cues is not mandatory in children. Proc. Natl. Acad. Sci. USA, 107, 17041-17046.
23. Paré, M., Richler, R.C., Ten Hove, M. & Munhall, K. G. (2003) Gaze behavior in audiovisual speech perception: the influence of ocular fixations on the McGurk effect. Percept. Psychophys., 65, 553-567.
24. Rosenblum, L.D., Schmuckler, M.A. & Johnson, J.A. (1997) The McGurk effect in infants. Percept. Psychophys., 59, 347-357.
25. Saint-Amour, D., De Sanctis, P., Molholm, S., Ritter, W. & Foxe, J.J.C.-P. (2007) Seeing voices: high-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion. Neuropsychologia, 45, 587-597.
26. Schwartz, J.L. (2010) A reanalysis of McGurk data suggests that audiovisual fusion in speech perception is subject-dependent. J. Acoust. Soc. Am., 127, 1584-1594.
27. Tiippana, K., Andersen, T.S. & Sams, M. (2004) Visual attention modulates audiovisual speech perception. Eur. J. Cogn. Psychol., 16, 457-472.
28. Tomalski, P., Ribeiro, H., Ballieux, H., Axelsson, E.L., Murphy, E., Moore, D.G. & Kushnerenko, E. (2012) Exploring early developmental changes in face scanning patterns during the perception of audiovisual mismatch of speech cues. Eur. J. Dev. Psychol., doi: 10.1080/17405629.2012.728076. [Epub ahead of print].
29. Tucker, D.M. (1993) Spatial sampling of head electrical fields: the geodesic sensor net. Electroen. Clin. Neuro., 87, 154-163.
30. Van Wassenhove, V., Grant, K.W. & Poeppel, D. (2005) Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. USA, 102, 1181-1186.
31. Wagner, J.B., Luyster, R.J., Yim, J.Y., Tager-Flusberg, H. & Nelson, C.A. (2013) The role of early visual attention in social development. Int. J. Behav. Dev., 37, 118-124.