You can raise your eyebrows, I don’t mind: are monolingual and bilingual infants equally good at learning from the eyes region of a talking face?

Auditory-Visual Speech Processing 2015, AVSP 2015, held in conjunction with Facial Analysis and Animation, FAA 2015 - 1st Joint Conference on Facial Analysis, Animation, and Auditory-Visual Speech Processing, FAAVSP 2015

Volumn , Issue , 2015, Pages 7-11

  Fort, Mathilde ^a   Escrichs, Anira ^a   Ayneto Gimeno, Alba ^a   Sebastián Gallés, Núria ^a  

^a UNIVERSITAT POMPEU FABRA   (Spain)

Author keywords

Attention; Bilingualism; Early language acquisition; Eyes; Infancy; Mouth; Talking faces

Indexed keywords

SPEECH PROCESSING;

ATTENTION; BILINGUALISM; BILINGUALS; EARLY LANGUAGE ACQUISITION; EYE; EYE REGIONS; INFANCY; LANGUAGE ACQUISITION; MOUTH; TALKING FACE;

ANIMATION;

EID: 85007180001     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (62)

1. W. H. Sumby and I. Pollack, "Visual Contribution to Speech Intelligibility in Noise," Journal of the Acoustical Society of America, vol. 26, pp. 212-215, 1954.
2. C. Benoît, et al., "Effects of phonetic context on audio-visual intelligibility of French.," Journal Of Speech And Hearing Research, vol. 37, pp. 1195-1203, 1994.
3. C. A. Binnie, et al., "Auditory and visual contributions to the perception of consonants," Journal Of Speech And Hearing Research, vol. 17, pp. 619-630, 1974.
4. N. P. Erber, "Interaction of audition and vision in the recognition of oral speech stimuli.," Journal Of Speech And Hearing Research, vol. 12, pp. 423-425, 1969.
5. M. Fort, Spinelli, E, Savariaux, C, Kandel, S, "The word superiority effect in audiovisual speech perception," Speech Communication, vol. 52 pp. 525-532, 2010.
6. M. Fort, Spinelli, E., Savariaux, C., Kandel, S., "Audiovisual vowel monitoring and the word superiority effect in children," International Journal of Behavior and Development, vol. In Press, 2012.
7. A. MacLeod and Q. Summerfield, "Quantifying the contribution of vision to speech perception in noise.," British Journal of Audiology, vol. 21, pp. 131-141, 1987.
8. L. a. Ross, et al., "Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments.," Cerebral Cortex, vol. 17, pp. 1147-53, 2007.
9. J. L. Schwartz, et al., "Seeing to hear better: evidence for early audio-visual interactions in speech identification.," Cognition, vol. 93, pp. B69-78, 2004.
10. D. Reisberg, et al., "Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli," in Hearing by Eye: The Psychology of LipReading, B. Dodd and R. Campbell, Eds., ed Londres: Erlbaum Associates, 1987, pp. 97-113.
11. J. Navarra and S. Soto-Faraco, "Hearing lips in a second language: visual articulatory information enables the perception of second language sounds.," Psychological Research, vol. 71, pp. 4-12, 2007.
12. P. Arnold and F. Hill, "Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact," British Journal of Psychology, vol. 92, pp. 339-355, 2001.
13. J. L. Schwartz, et al., "Ten years after Summerfield: A taxonomy of models for audio-visual fusion in speech perception.," in Hearing by Eye II: Advances in the psychology of speechreading and audiovisual speech, R. Campbell, et al., Eds., ed Hove: Psychology Press, 1998, pp. 85-108.
14. Q. A. Summerfield, "Some preliminaries to a comprehensive account of audio-visual speech perception," in Hearing by Eye: The Psychology of LipReading, ed Londres: Erlbaum Associates, 1987, pp. 3-51.
15. D. J. Lewkowicz and A. M. Hansen-Tift, "Infants deploy selective attention to the mouth of a talking face when learning speech," Proc Natl Acad Sci U S A, vol. 109, pp. 1431-6, Jan 31 2012.
16. E. Vatikiotis-Bateson, et al., "Eye movement of perceivers during audiovisual speech perception.," Perception & Psychophysics, vol. 60, pp. 926-940, 1998.
17. C. Benoît, et al., "Which components of the face do humans and machines best speechread?," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds., ed: Berlin: NATO-ASI Series 150 Springer, 1996.
18. C. R. Lansing and G. W. McConkie, "Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences," Percept Psychophys, vol. 65, pp. 536-52, May 2003.
19. J. N. Buchan, et al., "The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception," Brain Res, vol. 1242, pp. 162-71, Nov 25 2008.
20. I. T. Everdell, et al., "Gaze behaviour in audiovisual speech perception: asymmetrical distribution of face-directed fixations," Perception, vol. 36, pp. 1535-45, 2007.
21. J. N. Buchan, et al., "Spatial statistics of gaze fixations during dynamic face processing," Soc Neurosci, vol. 2, pp. 1-13, 2007.
22. K. G. Munhall and E. K. Johnson, "Speech perception: when to put your money where the mouth is," Curr Biol, vol. 22, pp. R190-2, Mar 20 2012.
23. M. Patterson, "Matching phonetic information in lips and voice is robust in 4.5-month-old infants," Infant Behavior and Development, vol. 22, pp. 237-247, 1999.
24. M. L. Patterson and J. F. Werker, "Infants' ability to match dynamic phonetic and gender information in the face and voice.," Journal of Experimental Child Psychology, vol. 81, pp. 93-115, 2002.
25. M. L. Patterson and J. F. Werker, "Two-month-old infants match phonetic information in lips and voice," Developmental Science, vol. 6, pp. 191-196, 2003.
26. P. K. Kuhl and A. N. Meltzoff, "The bimodal perception of speech in infancy.," Science, vol. 218, pp. 1138-1141, 1982.
27. D. Burnham and B. Dodd, "Auditory-visual speech perception as a direct process: The McGurk effect in infants and across languages," in Speechreading by humans and machines. vol. 150, D. G. Stork and M. E. Hennecke, Eds., ed: Springer-Verlag, 1996, pp. 103-114.
28. D. Burnham and B. Dodd, "Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect.," Developmental Psychobiology, vol. 45, pp. 204-220, 2004.
29. H. H. Yeung and J. F. Werker, "Lip movements affect infants' audiovisual speech perception," Psychol Sci, vol. 24, pp. 603-12, May 2013.
30. E. Kushnerenko, et al., "Electrophysiological evidence of illusory audiovisual speech percept in human infants," Proc Natl Acad Sci U S A, vol. 105, pp. 11442-5, Aug 12 2008.
31. F. Pons, et al., "Narrowing of intersensory speech perception in infancy," Proc Natl Acad Sci U S A, vol. 106, pp. 10598-602, Jun 30 2009.
32. L. D. Rosenblum, et al., "The McGurk effect in infants," Perception & Psychophysics, vol. 59, pp. 347-357, 1997.
33. T. Teinonen, et al., "Visual speech contributes to phonetic learning in 6-month-old infants," Cognition, vol. 108, pp. 850-5, Sep 2008.
34. D. J. Lewkowicz, et al., "Perception of the multisensory coherence of fluent audiovisual speech in infancy: its emergence and the role of experience," J Exp Child Psychol, vol. 130, pp. 147-62, Feb 2015.
35. D. J. Lewkowicz, "Early experience and multisensory perceptual narrowing," Dev Psychobiol, vol. 56, pp. 292-315, Feb 2014.
36. D. J. Lewkowicz and A. A. Ghazanfar, "The decline of cross-species intersensory perception in human infants," Proc Natl Acad Sci U S A, vol. 103, pp. 6771-4, Apr 25 2006.
37. K. Sekiyama and D. Burnham, "Impact of language on development of auditory-visual speech perception," Developmental Science, vol. 11, pp. 306-20, 2008.
38. S. Dupont, et al., "A study of the McGurk effect in 4-and 5-year-old French Canadian children," ZAS Papers in Linguistics, vol. 40, pp. 1-17, 2005.
39. D. J. Lewkowicz and R. Flom, "The audiovisual temporal binding window narrows in early childhood," Child Dev, vol. 85, pp. 685-94, Mar-Apr 2014.
40. D. W. Massaro, "Children's perception of visual and auditory speech.," Child Development, vol. 55, pp. 1777-1788, 1984.
41. D. W. Massaro, et al., "Developmental changes in visual and auditory contributions to speech perception.," Journal of Experimental Child Psychology, vol. 41, pp. 93-113, 1986.
42. N. S. Hockley and L. Polka, "A developmental study of audiovisual speech perception using the McGurk paradigm," Journal of the Acoustical Society of America, vol. 96, p. 3309, 1994.
43. S. Jerger, et al., "Developmental shifts in children's sensitivity to visual speech: a new multimodal picture-word task.," Journal of Experimental Child Psychology, vol. 102, pp. 40-59, 2009.
44. H. McGurk and J. MacDonald, "Hearing lips and seeing voices," Nature, vol. 264, pp. 746-748, 1976.
45. F. Pons, et al., "Bilingualism modulates infants' selective attention to the mouth of a talking face," Psychol Sci, vol. 26, pp. 490-8, Apr 2015.
46. E. J. Tenenbaum, et al., "Increased focus on the mouth among infants in the first year of life: A longitudinal eye-tracking study," Infancy, vol. 18, pp. 534-553, Jul 2013.
47. E. Kushnerenko, et al., "Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour," Eur J Neurosci, vol. 38, pp. 3363-9, Nov 2013.
48. C. Kubicek, et al., "Cross-modal matching of audio-visual German and French fluent speech in infancy," PLoS One, vol. 9, p. e89275, 2014.
49. T. Farroni, et al., "Eye contact detection in humans from birth," Proc Natl Acad Sci U S A, vol. 99, pp. 9602-5, Jul 9 2002.
50. T. Farroni, et al., "The perception of facial expressions in newborns," Eur J Dev Psychol, vol. 4, pp. 2-13, Mar 2007.
51. A. Senju and G. Csibra, "Gaze following in human infants depends on communicative signals," Curr Biol, vol. 18, pp. 668-71, May 6 2008.
52. A. Senju, et al., "Understanding the referential nature of looking: infants' preference for object-directed gaze," Cognition, vol. 108, pp. 303-19, Aug 2008.
53. D. Stahl, et al., "Eye contact and emotional face processing in 6-month-old infants: advanced statistical methods applied to event-related potentials," Brain Dev, vol. 32, pp. 305-17, Apr 2010.
54. L. Bosch and N. Sebastian-Galles, "Early language differentiation in bilingual infants," in Trends in bilingual acquisition, J. C. F. Genesee, Ed., ed Amsterdam, The Netherlands: John Benjamins, 2001, pp. 71-93.
55. A. Costa and N. Sebastian-Galles, "How does the bilingual experience sculpt the brain?," Nature Reviews Neuroscience, vol. 15, pp. 336–345, 2014.
56. J. F. Werker, "Perceptual foundations of bilingualism acquisition in infancy," Annals of the New-York Academy of Sciences, vol. 0077-8923, pp. 50-61, 2014.
57. E. Bialystok and F. I. Craik, "Cognitive and linguistic processing in the bilingual mind," Courrent Directions In Psychological Science, vol. 70, pp. 636-644, 2010.
58. L. E. Bahrick, et al., "Intersensory redundancy facilitates discrimination of tempo in 3-month-old infants," Dev Psychobiol, vol. 41, pp. 352-63, Dec 2002.
59. L. E. Bahrick and R. Lickliter, "Intersensory redundancy guides attentional selectivity and perceptual learning in infancy," Dev Psychol, vol. 36, pp. 190-201, Mar 2000.
60. W. M. Weikum, et al., "Visual language discrimination in infancy.," Science, vol. 316, p. 1159, 2007.
61. N. Sebastian-Galles, et al., "A bilingual advantage in visual language discrimination in infancy," Psychol Sci, vol. 23, pp. 994-9, Sep 1 2012.
62. N. Esteve-Gibert, et al., "Nine-month-old infants are sensitive to the temporal alignment of prosodic and gesture prominences," Infant Behav Dev, vol. 38, pp. 126-9, Feb 2015.