The temporal binding window for audiovisual speech: Children are like little adults

Neuropsychologia

Volumn 88, Issue , 2016, Pages 74-82

  Hillock Dunn, Andrea ^a   Grantham, D Wesley ^a   Wallace, Mark T ^a  

^a VANDERBILT UNIVERSITY   (United States)

Author keywords

Auditory; Development; Maturation; Mcgurk effect; Multisensory; Temporal; Visual

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; AUDIOVISUAL SPEECH; CHILD; COMPREHENSION; CONTROLLED STUDY; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; MCGURK EFFECT; MEASUREMENT ACCURACY; MULTISENSORY INTEGRATION; PERCEPTION; SENSORY SYSTEM; SPEECH PERCEPTION; AGE; ASSOCIATION; AUDITORY STIMULATION; PATTERN RECOGNITION; PHOTOSTIMULATION; TIME FACTOR; VISUAL ILLUSION; YOUNG ADULT;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AGE FACTORS; CHILD; CUES; FEMALE; HUMANS; MALE; OPTICAL ILLUSIONS; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; SPEECH PERCEPTION; TIME FACTORS; YOUNG ADULT;

EID: 84961226895     PISSN: 00283932     EISSN: 18733514     Source Type: Journal    
DOI: 10.1016/j.neuropsychologia.2016.02.017     Document Type: Article

References (82)

1. Astle, D.E., Scerif, G., Interactions between attention and visual short-term memory (VSTM): what can Be learnt from individual and developmental differences?. Neuropsychologia 49:6 (2011), 1435–1445.
2. Bahrick, L.E., Infants’ perception of substance and temporal synchrony in multimodal events. Infant Behav. Dev. 6 (1983), 429–451.
3. Bahrick, L.E., Infants’ intermodal perception of two levels of temporal structure in natural events. Infant Behav. Dev. 10 (1987), 387–416.
4. Bahrick, L.E., Intermodal learning in infancy: learning on the basis of two kinds of invariant relations in audible and visible events. Child. Dev. 59:1 (1988), 197–209.
5. Bahrick, L.E., Pickens, J.N., Amodal relations: The basis for intermodal perception and learning in infancy. Lickliter, D.J.L.R., (eds.) The Development of intersensory Perception: Comparative Perspectives, 1994, Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, England.
6. Burnham, D., Dodd, B., Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. Dev. Psychobiol. 45:4 (2004), 204–220.
7. Colonius, H., Diederich, A., Multisensory interaction in saccadic reaction time: a time-window-of-integration model. J. Cogn. Neurosci. 16:6 (2004), 1000–1009.
8. Dixon, N.F., Spitz, L., The detection of auditory visual desynchrony. Perception 9:6 (1980), 719–721.
9. Dodd, B., Lip reading in infants: attention to speech presented in- and out-of-synchrony. Cognit. Psychol. 11:4 (1979), 478–484.
10. Downing, H.C., Barutchu, A., Crewther, S.G., Developmental trends in the facilitation of multisensory objects with distractors. Front Psychol., 5, 2014, 1559.
11. Erber, N.P., Interaction of audition and vision in the recognition of oral speech stimuli. J. Speech Hear Res. 12:2 (1969), 423–425.
12. Ernst, M.O., Learning to integrate arbitrary signals from vision and touch. J. Vis.(5), 2007, 7 1–14.
13. Fixmer, E., Hawkins, S. 1998. The influence of quality of information on the McGurk effect. Paper presented at the Australian Workshop on Auditory-Visual Speech Processing.
14. Foss-Feig, J.H., Kwakye, L.D., Cascio, C.J., Burnette, C.P., Kadivar, H., Stone, W.L., et al. An extended multisensory temporal binding window in autism spectrum disorders. Exp. Brain Res. 203:2 (2010), 381–389.
15. Frens, M.A., Van Opstal, A.J., Van der Willigen, R.F., Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Percept. Psychophys. 57:6 (1995), 802–816.
16. Fujisaki, W., Shimojo, S., Kashino, M., Nishida, S., Recalibration of audiovisual simultaneity. Nat. Neurosci. 7:7 (2004), 773–778.
17. Gathercole, S.E., Cognitive approaches to the development of short-term memory. Trends Cognit. Sci. 3:11 (1999), 410–419.
18. Grant, K.W., Seitz, P.F., The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am. 108:3 Pt 1 (2000), 1197–1208.
19. Grant, K.W., van Wassenhove, V., Poeppel, D., Detection of auditory (cross-spectral) and auditory–visual (cross-modal) synchrony. Speech Commun. 44:1–4 (2004), 43–53.
20. Greenhouse, S.W., Geisser, S., On methods in the analysis of profile data. Psychometrika 24:2 (1959), 95–112.
21. Hairston, W.D., Burdette, J.H., Flowers, D.L., Wood, F.B., Wallace, M.T., Altered temporal profile of visual-auditory multisensory interactions in dyslexia. Exp. Brain Res. 166:3–4 (2005), 474–480.
22. Hall, J.W. 3rd, Grose, J.H., Buss, E., Dev, M.B., Spondee recognition in a two-talker masker and a speech-shaped noise masker in adults and children. Ear Hear 23:2 (2002), 159–165.
23. Helbig, H.B., Ernst, M.O., Optimal integration of shape information from vision and touch. Exp. Brain Res. 179:4 (2007), 595–606.
24. Hillock-Dunn, A., Wallace, M.T., Developmental changes in the multisensory temporal binding window persist into adolescence. Dev. Sci. 15:5 (2012), 688–696.
25. Hillock, A.R., Powers, A.R., Wallace, M.T., Binding of sights and sounds: age-related changes in multisensory temporal processing. Neuropsychologia 49:3 (2011), 461–467.
26. Hnath-Chisolm, T.E., Laipply, E., Boothroyd, A., Age-related Changes on a children's test of sensory-level speech perception capacity. J. Speech Lang. Hear. Res. 41:1 (1998), 94–106.
27. Hockley, N.S., Polka, L., A developmental study of audiovisual speech perception using the McGurk paradigm. J. Acoust. Soc. Am., 96(5), 1994 3309-3309.
28. Jones, J.A., Callan, D.E., Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. NeuroReport 14:8 (2003), 1129–1133.
29. Jones, J.A., Jarick, M., Multisensory integration of speech signals: the relationship between space and time. Exp. Brain Res. 174:3 (2006), 588–594.
30. Kaufman, A.S., Kaufman, N.L. 2004. Kaugman Brief Intelligence Test, Second Edition. Minneapolis, MN.
31. Klingberg, T., Forssberg, H., Westerberg, H., Increased brain activity in frontal and parietal cortex underlies the development of visuospatial working memory capacity during childhood. J. Cognit. Neurosci. 14:1 (2002), 1–10.
32. Kwakye, L.D., Foss-Feig, J.H., Cascio, C.J., Stone, W.L., Wallace, M.T., Altered auditory and multisensory temporal processing in autism spectrum disorders. Front Integr. Neurosci., 4, 2011, 129.
33. Kyle, F.E., Campbell, R., Mohammed, T., Coleman, M., MacSweeney, M., Speechreading development in deaf and hearing children: introducing the test of child speechreading. J. Speech Lang. Hear. Res. 56:2 (2013), 416–426.
34. Levene, H., Robust tests for equality of variances1. Contrib. Probab. Stat. : Essays Honor Harold Hotel. 2 (1960), 278–292.
35. Lewkowicz, D.J., Developmental Changes in infants’ bisensory response to synchronous durations. Infant Behav. Dev. 9:3 (1986), 335–353.
36. Lewkowicz, D.J., Sensory dominance in infants: I. six-month-old infants’ response to auditory-visual compounds. Dev. Psychol. 24:2 (1988), 155–171.
37. Lewkowicz, D.J., Sensory dominance in infants: II. ten-month-old infants’ response to auditory-visual compounds. Dev. Psychol. 24:2 (1988), 172–182.
38. Lewkowicz, D.J., Infants’ response to temporally based intersensory equivalence: the effect of synchronous sounds on visual preferences for moving stimuli. Infant Behav. Dev. 15:3 (1992), 297–324.
39. Lewkowicz, D.J., Perception of auditory-visual temporal synchrony in human infants. J. Exp. Psychol. : Hum. Percept. Perform. 22:5 (1996), 1094–1106.
40. Lewkowicz, D.J., Infants’ perception of the audible, visible and bimodal attributes of multimodal syllables. Child. Dev. 71:5 (2000), 1241–1257.
41. Lewkowicz, D.J., Infant perception of audio-visual speech synchrony. Dev. Psychol. 46:1 (2010), 66–77.
42. Lewkowicz, D.J., Flom, R., 2013. The Audiovisual Temporal Binding Window Narrows in Early Childhood. Child Development, n/a-n/a.
43. Lin, C.C., Hsiao, C.K., Chen, W.J., Development of sustained attention assessed using the continuous performance test among children 6–15 years of age. J. Abnorm. Child. Psychol. 27:5 (1999), 403–412.
44. Macleod, A., Summerfield, Q., Quantifying the contribution of vision to speech perception in noise. Br. J. Audiol. 21:2 (1987), 131–141.
45. Massaro, D.W., Children's perception of visual and auditory speech. Child. Dev. 55:5 (1984), 1777–1788.
46. Massaro, D.W., Cohen, M.M., Smeele, P.M., Perception of asynchronous and conflicting visual and auditory speech. J. Acoust. Soc. Am. 100:3 (1996), 1777–1786.
47. Massaro, D.W., Thompson, L.A., Barron, B., Laren, E., Developmental Changes in visual and auditory contributions to speech perception. J. Exp. Child. Psychol. 41:1 (1986), 93–113.
48. McGurk, H., Macdonald, J., Hearing lips and seeing voices. Nature 264:5588 (1976), 746–748.
49. Mendelson, M.J., Ferland, M.B., Auditory-visual transfer in four-month-old infants. Child. Dev. 53:4 (1982), 1022–1027.
50. Meredith, M.A., Nemitz, J.W., Stein, B.E., Determinants of multisensory integration in superior colliculus neurons. I. temporal factors. J. Neurosci. 7:10 (1987), 3215–3229.
51. Munhall, K.G., Gribble, P., Sacco, L., Ward, M., Temporal constraints on the McGurk effect. Percept. Psychophys. 58:3 (1996), 351–362.
52. Pang, E.W., Taylor, M.J., Tracking the development of the N1 from age 3 to adulthood: an examination of speech and non-speech stimuli. Clin. Neurophysiol. 111:3 (2000), 388–397.
53. Pons, F., Andreu, L., Sanz-Torrent, M., Buil-Legaz, L., Lewkowicz, D.J., Perception of audio-visual speech synchrony in Spanish-speaking children with and without specific language impairment. J. Child. Lang. 40:3 (2013), 687–700.
54. Powers, A.R. 3rd, Hillock, A.R., Wallace, M.T., Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29:39 (2009), 12265–12274.
55. Robinson, C.W., Sloutsky, V.M., Auditory dominance and its change in the course of development. Child. Dev. 75:5 (2004), 1387–1401.
56. Rose, S.A., Feldman, J.F., Jankowski, J.J., Futterweit, L.R., Visual and auditory temporal processing, cross-modal transfer, and reading. J. Learn Disabil. 32:3 (1999), 256–266.
57. Rosenblum, L.D., Schmuckler, M.A., Johnson, J.A., The McGurk effect in infants. Percept. Psychophys. 59:3 (1997), 347–357.
58. Satterthwaite, F.E., An approximate distribution of estimates of variance components. Biom. Bull. 2:6 (1946), 110–114.
59. Schwartz, J.L., Berthommier, F., Savariaux, C., Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 93:2 (2004), B69–B78.
60. Sekiyama, K., Kanno, I., Miura, S., Sugita, Y., Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 47:3 (2003), 277–287.
61. Sekiyama, K., Tohkura, Y. i, McGurk effect in non-English listeners: few visual effects for japanese subjects hearing Japanese syllables of high auditory intelligibility. J. Acoust. Soc. Am. 90:4 (1991), 1797–1805.
62. Sloutsky, V.M., Napolitano, A.C., IS a picture worth a thousand Words? Preference for auditory modality in young children. Child. Dev. 74:3 (2003), 822–833.
63. Soto-Faraco, S., Alsius, A., Deconstructing the McGurk-MacDonald illusion. J. Exp. Psychol. Hum. Percept. Perform. 35:2 (2009), 580–587.
64. Spelke, E.S., Perceiving bimodally specified events in infancy. Dev. Psychol. 15 (1979), 626–636.
65. Stein, B.E., Huneycutt, W.S., Meredith, M.A., Neurons and behavior: the same rules of multisensory integration apply. Brain Res. 448:2 (1988), 355–358.
66. Stevenson, R.A., Wallace, M.T., Multisensory temporal integration: task and stimulus dependencies. Exp. Brain Res. 227:2 (2013), 249–261.
67. Stevenson, R.A., Wallace, M.T., Altieri, N., The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech. Front. Psychol., 5, 2014, 352.
68. Stevenson, R.A., Wilson, M.M., Powers, A.R., Wallace, M.T., The effects of visual training on multisensory temporal processing. Exp. Brain Res. 225:4 (2013), 479–489.
69. Stevenson, R.A., Zemtsov, R.K., Wallace, M.T., Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. J. Exp. Psychol. Hum. Percept. Perform. 38:6 (2012), 1517–1529.
70. Sumby, W.H., Pollack, I., Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26:2 (1954), 212–215.
71. ten Oever, S., Sack, A.T., Wheat, K.L., Bien, N., van Atteveldt, N., Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs. Front. Psychol., 4, 2013, 331.
72. Tremblay, C., Champoux, F., Voss, P., Bacon, B.A., Lepore, F., Theoret, H., Speech and non-speech audio-visual illusions: a developmental study. PLoS One, 2(1), 2007, e742.
73. Tye-Murray, Hale, N., Spehar, S., Myerson, B., Sommers, J., M.S., 2013. Lipreading in School-age Children: The Roles of Age, Hearing Status, and Cognitive Ability, J Speech Lang Hear Res.
74. van Wassenhove, V., Grant, K.W., Poeppel, D., Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45:3 (2007), 598–607.
75. Vatakis, A., Spence, C., Audiovisual synchrony perception for music, speech, and object actions. Brain Res. 1111:1 (2006), 134–142.
76. Vatakis, A., Spence, C., Crossmodal binding: evaluating the “unity assumption” using audiovisual speech stimuli. Percept. Psychophys. 69:5 (2007), 744–756.
77. Vouloumanos, A., Kiehl, K.A., Werker, J.F., Liddle, P.F., Detection of sounds in the auditory stream: event-related fMRI evidence for differential activation to speech and nonspeech. J. Cognit. Neurosci. 13:7 (2001), 994–1005.
78. Vroomen, J., Keetels, M., de Gelder, B., Bertelson, P., Recalibration of temporal order perception by exposure to audio-visual asynchrony. Brain Res. Cogn. Brain Res. 22:1 (2004), 32–35.
79. Wallace, M.T., Stevenson, R.A., 2014. The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities, Neuropsychologia.
80. Welch, B.L., The generalization of ‘Student's’ problem when several different population varlances are involved. Biometrika 34:1–2 (1947), 28–35.
81. Witten, I.B., Knudsen, E.I., Why seeing is believing: merging auditory and visual worlds. Neuron 48:3 (2005), 489–496.
82. Zampini, M., Guest, S., Shore, D.I., Spence, C., Audio-visual simultaneity judgments. Percept. Psychophys. 67:3 (2005), 531–544.