Sources of confusion in infant audiovisual speech perception research

Frontiers in Psychology

Volumn 6, Issue DEC, 2015, Pages

  Shaw, Kathleen E ^a   Bortfeld, Heather ^b,c  

^a UNIVERSITY OF CONNECTICUT   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c HASKINS LABORATORIES   (United States)

Author keywords

Audiovisual perception; Infant perception; Multimodal integration; Sine wave speech; Speech disorders; Speech perception; Temporal binding window

Indexed keywords

EID: 84954215080     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.01844     Document Type: Short Survey

References (76)

1. Baart, M., Bortfeld, H., and Vroomen, J. (2015). Phonetic matching of auditory and visual speech develops during childhood: evidence from sine-wave speech. J. Exp. Child Psychol. 129, 157-164. doi: 10.1016/j.jecp.2014.08.002
2. Baart, M., Vroomen, J., Shaw, K., and Bortfeld, H. (2014). Degrading phonetic information affects matching of audiovisual speech in adults, but not in infants. Cognition 130, 31-43. doi: 10.1016/j.cognition.2013.09.006
3. Bahrick, L. E. (1988). Intermodal learning in infancy: learning on the basis of two kinds of invariant relations in audible and visible events. Child Dev. 59, 197-209. doi: 10.2307/1130402
4. Bahrick, L. E., Lickliter, R., and Flom, R. (2004). Intersensory redundancy guides the development of selective attention, perception, and cognition in infancy. Curr. Dir. Psychol. Sci. 13, 99-102. doi: 10.1111/j.0963-7214.2004.00283.x
5. Bebko, J. M., Weiss, J. A., Demark, J. L., and Gomez, P. (2006). Discrimination of temporal synchrony in intermodal events by children with autism and children with developmental disabilities without autism. J. Child Psychol. Psychiatry 47, 88-98. doi: 10.1111/j.1469-7610.2005.01443.x
6. Bergeson, T. R., Pisoni, D. B., and Davis, R. A. O. (2005). Development of audiovisual comprehension skills in prelingually deaf children with cochlear implants. Ear Hear. 26, 149-164. doi: 10.1097/00003446-200504000-00004
7. Busnel, M. C., and Granier-Deferre, C. (1983). "And what of fetal audition?" in The Behavior of Human Infants, eds A. Oliveirio and M. Zappella (New York, NY: Plenum), 93-126.
8. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., and Ghazanfar, A. A. (2009). The natural statistics of audiovisual speech. PLoS Comput. Biol. 5:e1000436. doi: 10.1371/journal.pcbi.1000436
9. de Boer-Schellekens, L., Eussen, M., and Vroomen, J. (2013). Diminished sensitivity of audiovisual temporal order in autism spectrum disorder. Front. Integr. Neurosci. 7:8. doi: 10.3389/fnint.2013.00008
10. DeCasper, A. J., Lecanuet, J. P., Busnel, M. C., Granier-Deferre, C., and Maugeais, R. (1994). Fetal reactions to recurrent maternal speech. Infant Behav. Dev. 17, 159-164. doi: 10.1016/0163-6383(94)90051-5
11. DeCasper, A. J., and Spence, M. J. (1986). Prenatal maternal speech influences newborns' perception of speech sounds. Infant Behav. Dev. 9, 133-150. doi: 10.1016/0163-6383(86)90025-1
12. Dick, A. S., Solodkin, A., and Small, S. L. (2010). Neural development of networks for audiovisual speech comprehension. Brain Lang. 114, 101-114. doi: 10.1016/j.bandl.2009.08.005
13. Dodd, B. (1979). Lip reading in infants: attention to speech presented in-and out-of-synchrony. Cogn. Psychol. 11, 478-484. doi: 10.1016/0010-0285(79)90021-5
14. Dogge, M., Schaap, M., Custers, R., Wegner, D. M., and Aarts, H. (2012). When moving without volition: implied self-causation enhances binding strength between involuntary actions and effects. Conscious. Cogn. 21, 501-506. doi: 10.1016/j.concog.2011.10.014
15. Foss-Feig, J. H., Kwakye, L. D., Cascio, C. J., Burnette, C. P., Kadivar, H., Stone, W. L., et al. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Exp. Brain Res. 203, 381-389. doi: 10.1007/s00221-010-2240-4
16. Goren, C. C., Sarty, M., and Wu, P. Y. (1975). Visual following and pattern discrimination of face-like stimuli by newborn infants. Pediatrics 56, 544-549.
17. Graven, S. N., and Browne, J. V. (2008). Sensory development in the fetus, neonate, and infant: introduction and overview. Newborn Infant Nurs. Rev. 8, 169-172. doi: 10.1053/j.nainr.2008.10.007
18. Hairston, W. D., Burdette, J. H., Flowers, D. L., Wood, F. B., and Wallace, M. (2005). Altered temporal profile of visual-auditory multisensory interactions in dyslexia. Exp. Brain Res. 166, 474-480. doi: 10.1007/s00221-005-2387-6
19. Hepper, P. G., and Shahidullah, B. S. (1994). Development of fetal hearing. Arch. Dis. Child. 71, F81-F87. doi: 10.1136/fn.71.2.F81
20. Hillock, A. R., Powers, A. R., and Wallace, M. (2011). Binding of sights and sounds: age-related changes in multisensory temporal processing. Neuropsychologia 49, 461-467. doi: 10.1016/j.neuropsychologia.2010.11.041
21. Innes-Brown, H., Barutcha, A., Shivdasani, M. N., Crewther, D. P., Grayden, D. B., and Paolini, A. G. (2011). Susceptibility to the flash-beep illusion is increased in children compared to adults. Dev. Sci. 14, 1089-1099. doi: 10.1111/j.1467-7687.2011.01059.x
22. Jardri, R., Pins, D., Houfflin-Debarge, V., Chaffiotte, C., Rocourt, N., Pruvo, J. P., et al. (2008). Fetal cortical activation to sound at 33 weeks of gestation: a functional MRI study. Neuroimage 42, 10-18. doi: 10.1016/j.neuroimage.2008.04.247
23. Kisilevsky, B. S., Hains, S. M. J., Jacquet, A. Y., Granier-Deferre, C., and Lecanuet, J. P. (2004). Maturation of fetal response to music. Dev. Sci. 7, 550-559. doi: 10.1111/j.1467-7687.2004.00379.x
24. Kubicek, C., Hillairet de Boisferon, A., Dupierrix, E., Pascalis, O., Lœvenbruck, H., Gervain, J., et al. (2014). Cross-modal matching of audio-visual German and French fluent speech in infancy. PLoS ONE 9:e89275. doi: 10.1371/journal.pone.0089275
25. Kuhl, P. K., and Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science 218, 1138-1141. doi: 10.1126/science.7146899
26. Kuhl, P. K., and Meltzoff, A. N. (1984). The intermodal representation of speech in infants. Infant Behav. Dev. 7, 361-381. doi: 10.1016/S0163-6383(84)80050-8
27. Kuhl, P. K., Williams, K. A., and Meltzoff, A. N. (1991). Cross-modal speech perception in adults and infants using nonspeech auditory stimuli. J. Exp. Psychol. Hum. Percept. Perform. 17, 829-840. doi: 10.1037/0096-1523.17.3.829
28. Kushnerenko, E., Teinonen, T., Volein, A., and Csibra, G. (2008). Electrophysiological evidence of illusory audiovisual speech percept in human infants. Proc. Natl. Acad. Sci. U.S.A. 105, 11442-11445. doi: 10.1073/pnas.0804275105
29. Kushnerenko, E., Tomalski, P., Ballieux, H., Ribeiro, H., Potton, A., Axelsson, E., et al. (2013). Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour. Eur. J. Neurosci. 38, 3363-3369. doi: 10.1111/ejn.12317
30. Lecanuet, J. P. (1996). "Prenatal auditory experience," in Musical Beginnings: Origins and Development of Musical Competence, eds I. Deliege and J. Slobada (New York, NY: Oxford University Press), 3-34.
31. Legerstee, M. (1990). Infants use multimodal information to imitate speech sounds. Infant Behav. Dev. 13, 343-354. doi: 10.1016/0163-6383(90)90039-B
32. Lewkowicz, D. (1992). Infants' responsiveness to the auditory and visual attributes of a sounding/moving stimulus. Percept. Psychophys. 52, 519-528. doi: 10.3758/BF03206713
33. Lewkowicz, D. J. (1994). Limitations on infants' response to rate-based auditory-visual relations. Dev. Psychol. 30, 880-892. doi: 10.1037/0012-1649.30.6.880
34. Lewkowicz, D. J. (2003). Learning and discrimination of audiovisual events in human infants: the hierarchical relation between intersensory temporal synchrony and rhythmic pattern cues. Dev. Psychol. 39, 795-804. doi: 10.1037/0012-1649.39.5.795
35. Lewkowicz, D. J. (2010). Infant perception of audio-visual speech synchrony. Dev. Psychol. 46, 66-77. doi: 10.1037/a0015579
36. Lewkowicz, D. J., Minar, N. J., Tift, A. H., and Brandon, M. (2015). Perception of the multisensory coherence of fluent audiovisual speech in infancy: its emergence and the role of experience. J. Exp. Child Psychol. 130, 147-162. doi: 10.1016/j.jecp.2014.10.006
37. Liberman, A. M., and Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition 21, 1-36. doi: 10.1016/0010-0277(85)90021-6
38. Lickliter, R., and Bahrick, L. E. (2000). The development of infant intersensory perception: advantages of a comparative convergent-operations approach. Psychol. Bull. 126, 260-280. doi: 10.1037/0033-2909.126.2.260
39. Massaro, D. W. (1984). Children's perception of visual and auditory speech. Child Dev. 55, 1777-1788. doi: 10.2307/1129925
40. Matchin, W., Groulx, K., and Hickok, G. (2014). Audiovisual speech integration does not rely on the motor system: evidence from articulatory suppression, the McGurk Effect, and fMRI. J. Cogn. Neurosci. 26, 606-620. doi: 10.1162/jocn_a_00515
41. McGurk, H., and MacDonald, J. (1976). Hearing lips and seeing voices. Nature 264, 746-748. doi: 10.1038/264746a0
42. Middelweerd, M. J., and Plomp, R. (1987). The effect of speechreading on the speech-reception threshold of sentences in noise. J. Acoust. Soc. Am. 82, 2145-2147. doi: 10.1121/1.395659
43. Miller, L. M., and D'Esposito, M. (2005). Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 25, 164-181. doi: 10.1523/JNEUROSCI.0896-05.2005
44. Morton, J., and Johnson, M. H. (1991). CONSPEC and CONLERN: a two-process theory of infant face recognition. Psychol. Rev. 98, 164-181. doi: 10.1037/0033-295X.98.2.164
45. Möttönen, R., Calvert, G. A., Jääskeläinen, I. P., Matthews, P. M., Thesen, T., Tuomainen, J., et al. (2006). Perceiving identical sounds as speech or non-speech modulates activity in the left posterior superior temporal sulcus. Neuroimage 30, 563-569. doi: 10.1016/j.neuroimage.2005.10.002
46. Nagy, E. (2008). Innate intersubjectivity: newborns' sensitivity to communication disturbance. Dev. Psychol. 44, 1779-1784. doi: 10.1037/a0012665
47. Navarra, J., and Soto-Faraco, S. (2007). Hearing lips in a second language: visual articulatory information enables the perception of second language sounds. Psychol. Res. 71, 4-12. doi: 10.1007/s00426-005-0031-5
48. Parise, C. V., Spence, C., and Ernst, M. O. (2012). When correlation implies causation in multisensory integration. Curr. Biol. 22, 46-49. doi: 10.1016/j.cub.2011.11.039
49. Patterson, M. L., and Werker, J. F. (2003). Two-month-old infants match phonetic information in lips and voice. Dev. Sci. 6, 191-196. doi: 10.1111/1467-7687.00271
50. Patterson, M. L., and Werker, J. F. (1999). Matching phonetic information in lips and voice is robust in 4.5-month-old infants. Infant Behav. Dev. 22, 237-247. doi: 10.1016/S0163-6383(99)00003-X
51. Pons, F., Teixidó, M., Garcia-Morera, J., and Navarra, J. (2012). Short-term experience increases infants' sensitivity to audiovisual asynchrony. Infant Behav. Dev. 35, 815-818. doi: 10.1016/j.infbeh.2012.06.006
52. Pujol, R., Lavigne-Rebillard, M., and Uziel, A. (1991). Development of the human cochlea. Acta Otolaryngol. 111, 7-13. doi: 10.3109/00016489109128023
53. Remez, R. E., Rubin, P. E., Pisoni, D. B., and Carrell, T. D. (1981). Speech perception without traditional speech cues. Science 212, 947-949. doi: 10.1126/science.7233191
54. Rosenblum, L. D., Schmuckler, M. A., and Johnson, J. A. (1997). The McGurk effect in infants. Percept. Psychophys. 59, 347-357. doi: 10.3758/BF03211902
55. Rubel, E. W., and Ryals, B. M. (1983). Development of the place principle: acoustic trauma. Science 219, 512-514. doi: 10.1126/science.6823549
56. Shaw, K., Baart, M., Depowski, N., and Bortfeld, H. (2015). Infants' preference for native audiovisual speech dissociated from congruency preference. PLoS ONE 10:e0126059. doi: 10.1371/journal.pone.0126059
57. Slater, A. (2002). Visual perception in the newborn infant: issues and debates. Intellectica 34, 57-76. Available online at: http://intellectica.org/SiteArchives/archives/n34/n34_table.htm
58. Stein, B. E., Burr, D., Constantinidis, C., Laurienti, P. J., Alex Meredith, M., Perrault, T. J., et al. (2010). Semantic confusion regarding the development of multisensory integration: a practical solution. Eur. J. Neurosci. 31, 1713-1720. doi: 10.1111/j.1460-9568.2010.07206.x
59. Stekelenburg, J. J., and Vroomen, J. (2012). Electrophysiological evidence for a multisensory speech-specific mode of perception. Neuropsychologia 50, 1425-1431. doi: 10.1016/j.neuropsychologia.2012.02.027
60. Stevenson, R. A., Zemtsov, R. K., and Wallace, M. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. J. Exp. Psychol. Hum. Percept. Perform. 38, 1517-1529. doi: 10.1037/a0027339
61. Sumby, W. H., and Pollack, I. (1954). Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26, 212-215. doi: 10.1121/1.1907309
62. Teinonen, T., Aslin, R. N., Alku, P., and Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition 108, 850-855. doi: 10.1016/j.cognition.2008.05.009
63. Tenenbaum, E. J., Shah, R. J., Sobel, D. M., Malle, B. F., and Morgan, J. L. (2013). Increased focus on the mouth among infants in the first year of life: a longitudinal eye-tracking study. Infancy 18, 534-553. doi: 10.1111/j.1532-7078.2012.00135.x
64. Tenenbaum, E. J., Sobel, D. M., Sheinkopf, S. J., Malle, B. F., and Morgan, J. L. (2015). Attention to the mouth and gaze following in infancy predict language development. J. Child Lang. 42, 1173-1190. doi: 10.1017/S0305000914000725
65. Tomalski, P. (2015). Developmental trajectory of audiovisual speech integration in early infancy: a review of studies using the McGurk paradigm. Psychol. Lang. Commun. 19, 77-100. doi: 10.1515/plc-2015-0006
66. Tuomainen, J., Andersen, T. S., Tiippana, K., and Sams, M. (2005). Audio-visual speech perception is special. Cognition 96, B13-B22. doi: 10.1016/j.cognition.2004.10.004
67. Turkewitz, G., and Kenny, P. A. (1982). Limitations on input as a basis for neural organization and perceptual development: a preliminary theoretical statement. Dev. Psychobiol. 15, 357-368. doi: 10.1002/dev.420150408
68. Van Wassenhove, V., Grant, K. W., and Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45, 598-607. doi: 10.1016/j.neuropsychologia.2006.01.001
69. Vroomen, J., and Baart, M. (2009). Phonetic recalibration only occurs in speech mode. Cognition 110, 254-259. doi: 10.1016/j.cognition.2008.10.015
70. Vroomen, J., and Stekelenburg, J. J. (2011). Perception of intersensory synchrony in audiovisual speech: not that special. Cognition 118, 75-83. doi: 10.1016/j.cognition.2010.10.002
71. Wallace, M., and Stein, B. (2007). Early experience determines how the senses will interact. J. Neurophysiol. 97, 921-926. doi: 10.1152/jn.00497.2006
72. Wallace, M. T., and Stevenson, R. A. (2014). The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia 64, 105-123. doi: 10.1016/j.neuropsychologia.2014.08.005
73. Watson, T. L., Robbins, R. A., and Best, C. T. (2014). Infant perceptual development for faces and spoken words: an integrated approach. Dev. Psychobiol. 56, 1454-1481. doi: 10.1002/dev.21243
74. Weikum, W. M., Vouloumanos, A., Navarra, J., Soto-Faraco, S., Sebastián-Gallés, N., and Werker, J. F. (2007). Visual language discrimination in infancy. Science 316, 1159-1159. doi: 10.1126/science.1137686
75. Yeung, H. H., and Werker, J. F. (2013). Lip movements affect infants' audiovisual speech perception. Psychol. Sci. 24, 603-612. doi: 10.1177/0956797612458802
76. Yu, L., Rowland, B. A., and Stein, B. E. (2010). Initiating the development of multisensory integration by manipulating sensory experience. J. Neurosci. 30, 4904-4913. doi: 10.1523/JNEUROSCI.5575-09.2010