Perception of audiovisual speech synchrony for native and non-native language

Brain Research

Volumn 1323, Issue , 2010, Pages 84-93

  Navarra, Jordi ^a,b   Alsius, Agnès ^c   Velasco, Ignacio ^a   Soto Faraco, Salvador ^c,d,e   Spence, Charles ^b  

^a Martorell   (Spain)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF BARCELONA   (Spain)

^d INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

^e UNIVERSITAT POMPEU FABRA   (Spain)

Author keywords

Audiovisual integration; Native language; Non native language; Prior experience; Speech perception; Synchrony perception

Indexed keywords

ADULT; ARTICLE; AUDITORY SYSTEM FUNCTION; CONTROLLED STUDY; HUMAN; LANGUAGE; LANGUAGE ABILITY; MALE; PRIORITY JOURNAL; SIMULATION; SPAIN; SPEECH PERCEPTION; TASK PERFORMANCE; VISUAL INFORMATION; VISUAL SYSTEM FUNCTION;

ACOUSTIC STIMULATION; FEMALE; HUMANS; MALE; PHOTIC STIMULATION; SPEECH; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 77953137229     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2010.01.059     Document Type: Article

References (53)

1. Alsius A., Navarra J., Campbell R., and Soto-Faraco S. Audiovisual integration of speech falters under high attention demands. Curr. Biol. 15 (2005) 839-843
2. Alsius A., Navarra J., and Soto-Faraco S. Attention to touch weakens audiovisual speech integration. Exp. Brain Res. 183 (2007) 399-404
3. Arrighi R., Alais D., and Burr D. Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. J. Vis. 6 (2006) 260-268
4. Best C., and McRoberts G.W. Infant perception of non-native consonant contrasts that adults assimilate in different ways. Lang. Speech 46 (2003) 183-216
5. Dixon N.F., and Spitz L. The detection of auditory visual desynchrony. Perception 9 (1980) 719-721
6. Fujisaki W., Shimojo S., Kashino M., and Nishida S. Recalibration of audiovisual simultaneity. Nat. Neurosci. 7 (2004) 773-778
7. Grant K.W., van Wassenhove V., and Poeppel D. Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony. J. Acoust. Soc. Am. 108 (2004) 1197-1208
8. Hanson J.V.M., Heron J., and Whitaker D. Recalibration of perceived time across sensory modalities. Exp. Brain Res. 185 (2008) 347-352
9. Koppen C., Alsius A., and Spence C. Semantic congruency and the Colavita visual dominance effect. Exp. Brain Res. 184 (2008) 533-546
10. Kuhl P.K., Williams K.A., Lacerda F., Stevens K.N., and Lindblom B. Linguistic experience alters phonetic perception in infants six months of age. Science 255 (1992) 606-608
11. Liberman A.M., Cooper F.S., Shankweiler D.P., and Studdert-Kennedy M. Why are speech spectrograms hard to read?. Am. Ann. Deaf 113 (1968) 127-133
12. Marslen-Wilson W.D., and Tyler L.K. The temporal structure of spoken language understanding. Cognition 8 (1980) 1-71
13. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 265 (1976) 746-748
14. Mitterer H., and Blomert L. Coping with phonological assimilation in speech perception: evidence for early compensation. Percept. Psychophys. 65 (2003) 956-969
15. Munhall K.G., Gribble P., Sacco L., and Ward M. Temporal constraints on the McGurk effect. Percept. Psychophys. 58 (1996) 351-362
16. Murray M.M., Foxe J.J., and Wylie G.R. The brain uses single-trial multisensory memories to discriminate without awareness. NeuroImage 27 (2005) 473-478
17. Myung I.J. Tutorial on maximum likelihood estimation. J. Math. Psychol. 47 (2003) 90-100
18. Navarra J., and Soto-Faraco S. Hearing lips in a second language: visual articulatory information enables the perception of L2 sounds. Psychol. Res. 71 (2007) 4-12
19. Navarra J., Sebastián-Gallés N., and Soto-Faraco S. The perception of second language sounds in early bilinguals: new evidence from an implicit measure. J. Exp. Psychol. Hum. Percept. Perform. 31 (2005) 912-918
20. Navarra J., Vatakis A., Zampini M., Soto-Faraco S., Humphreys W., and Spence C. Temporal recalibration during the perception of audiovisual asynchrony in speech and music. Brain Res. Cogn. Brain Res. 25 (2005) 499-507
21. Navarra J., Soto-Faraco S., and Spence C. Adaptation to audiotactile asynchrony. Neurosci. Lett. 413 (2007) 72-76
22. Navarra J., Hartcher-O'Brien J., Piazza E., and Spence C. Adaptation to audiovisual asynchrony modulates the speeded detection of sound. Proc. Natl. Acad. Sci. U. S. A. 106 (2009) 9169-9173
23. Navarra J., Alsius A., Soto-Faraco S., and Spence C. Is attention involved in the audiovisual integration of speech?. Inform. Fusion 11 (2010) 4-11
24. Parise C., and Spence C. 'When the birds of a feather flock together': synesthetic associations modulate multisensory integration in non-synesthetes. PLoS ONE 4 (2009) e5664
25. Petrini K., Russell M., and Pollick F. When knowing can replace seeing in audiovisual integration of actions. Cognition 110 (2009) 432-439
26. Polka L., Colantonio C., and Sundara M. A cross-language comparison of /d/-/th/ perception: evidence for a new developmental pattern. J. Acoust. Soc. Am. 9 (2001) 2190-2201
27. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., and Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 17 (2007) 1147-1153
28. Schutz, M., Kubovy, M., 2009. Causality and cross-modal integration. J. Exp. Psychol. Hum. Percept. Perform. 35 (6), 1791-1810.
29. Sebastián-Gallés N., and Soto-Faraco S. Online processing of native and non-native phonemic contrasts in early bilinguals. Cognition 72 (1999) 111-123
30. Sebastián-Gallés N., Echeverría S., and Bosch L. The influence of initial exposure on lexical representation: comparing early and simultaneous bilinguals. J. Memory Lang. 52 (2005) 240-255
31. Sekiyama K. Cultural and Linguistic factors in audiovisual speech processing: the McGurk effect in Chinese subjects. Percept. Psychophys. 59 (1997) 73-80
32. Sekiyama K., and Tohkura Y. Inter-language differences in the influence of visual cues in speech perception. J. Phonetics 21 (1993) 427-444
33. Sekiyama K., and Burnham D. Impact of language on development of auditory-visual speech perception. Dev. Sci. 11 (2008) 306-320
34. Soto-Faraco S., and Alsius A. Conscious access to the unisensory components of a cross-modal illusion. NeuroReport 18 (2007) 347-350
35. Soto-Faraco S., and Alsius A. Deconstructing the McGurk-MacDonald illusion. J. Exp. Psychol. Hum. Percept. Perform. 35 (2009) 580-587
36. Soto-Faraco S., Navarra J., Weikum W., Vouloumanos A., Sebastián-Gallés N., and Werker J. Discriminating languages by speechreading. Percept. Psychophys.. 69 (2007) 218-231
37. Spence C. Multisensory perception. In: Blumenthal H. (Ed). The Big Fat Duck Cook Book (2008), Bloomsbury, London 484-485
38. Spence C., Shore D.I., and Klein R.M. Multisensory prior entry. J. Exp. Psychol. Gen. 130 (2001) 799-832
39. Stekelenburg J.J., and Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. J. Cogn. Neurosci. 19 (2007) 1964-1973
40. Sumby W., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
41. van Wassenhove V., Grant K.W., and Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 1181-1186
42. Vatakis A., and Spence C. Audiovisual synchrony perception for music, speech, and object actions. Brain Res. 1111 (2006) 134-142
43. Vatakis A., and Spence C. Crossmodal binding: evaluating the "unity assumption" using audiovisual speech stimuli. Percept. Psychophys. 69 (2007) 744-756
44. Vatakis A., and Spence C. Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. Acta Psychol. 127 (2008) 12-23
45. Vatakis, A., Spence, C., unpublished. Audiovisual temporal integration for complex speech, object-action, animal call, and musical stimuli. To appear in M. J. Naumer and J. Kaiser (Eds.), Multisensory object perception in the primate brain. Springer.
46. Vatakis A., Navarra J., Soto-Faraco S., and Spence C. Temporal recalibration during asynchronous audiovisual speech perception. Exp. Brain Res. 181 (2007) 173-181
47. Vroomen, J., Stekelenburg, J.J., in press. Visual anticipatory information modulates multisensory interactions of artificial audiovisual stimuli. Journal of Cognitive Neuroscience.
48. Vroomen J., Keetels M., de Gelder B., and Bertelson P. Recalibration of temporal order perception by exposure to audiovisual asynchrony. Cogn. Brain Res. 22 (2004) 32-35
49. Wallace M.T., and Stein B.E. Early experience determines how the senses will interact. J. Neurophysiol. 97 (2007) 921-926
50. Weikum W.M., Vouloumanos A., Navarra J., Soto-Faraco S., Sebastián-Gallés N., and Werker J.F. Visual language discrimination in infancy. Science 316 (2007) 1159
51. Werker J.F., and Tees R.C. Cross-language speech perception: evidence for perceptual reorganization during the first year of life. Infant Behav. Dev. 7 (1984) 49-63
52. Witten I.B., Knudsen E.I., and Sompolinsky H. A Hebbian learning rule mediates asymmetric plasticity in aligning sensory representations. J. Neurophysiol. 100 (2008) 1067-1079
53. Zampini M., Guest S., Shore D.I., and Spence C. Audio-visual simultaneity judgments. Percept. Psychophys. 67 (2005) 531-544