Speech segmentation is facilitated by visual cues

Quarterly Journal of Experimental Psychology

Volumn 63, Issue 2, 2010, Pages 260-274

  Cunillera, Toni ^a,b   Càmara, Estela ^a,c   Laine, Matti ^b   Rodríguez Fornells, Antoni ^a,d  

^a UNIVERSITY OF BARCELONA   (Spain)

^b ÅBO AKADEMI UNIVERSITY   (Finland)

^c OTTO VON GUERICKE UNIVERSITY   (Germany)

^d INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

Author keywords

Audio visual speech; Language learning; Speech segmentation; Statistical learning; Temporal contiguity

Indexed keywords

ARTICLE; ASSOCIATION; HUMAN; LANGUAGE; LEARNING; LINGUISTICS; PHOTOSTIMULATION; PHYSIOLOGY; SPEECH; SPEECH PERCEPTION; STUDENT; VISION;

ASSOCIATION LEARNING; CUES; HUMANS; LANGUAGE; PHOTIC STIMULATION; PSYCHOLINGUISTICS; SPEECH; SPEECH PERCEPTION; STUDENTS; VERBAL LEARNING; VISUAL PERCEPTION; VOCABULARY;

EID: 77649154903     PISSN: 17470218     EISSN: 17470226     Source Type: Journal    
DOI: 10.1080/17470210902888809     Document Type: Article

References (66)

1. Abercrombie, D. (1967). Elements of general phonetics. Chicago: Aldine.
2. Aronson, E., & Rosenbloom, S. (1971). Space perception in early infancy: Perception within common auditory-visual space. Science, 172, 1161-1163.
3. Aslin, R. N., Saffran, J. R., & Newport, E. L. (1998). Computation of conditional probability statistics by 8-month-old infants. Psychological Science, 9, 321-324.
4. Bahrick, L. E., & Lickliter, R. (2000). Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Developmental Psychology, 36,190-201.
5. Bahrick, L. E., & Lickliter, R. (2002). Intersensory redundancy guides early perceptual and cognitive development. In R. Kail (Eds.), Advances in child development and behavior (pp. 153-187). New York: Academic Press.
6. Baldwin, D. A. (1991). Infants' contribution to the achievement of joint reference. Child Development, 62,875-890.
7. Baldwin, D. A. (1993). Infants' ability to consult the speaker for clues to word reference. Journal of Child Language, 20, 395-418.
8. Baldwin, D. A. (1995). Understanding the link between joint attention and language. In C. Moore & P. J. Dunham (Eds.), Joint attention: Its origins and role in development (pp. 131-158). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
9. Bloom, L. (1998). Language acquisition in its developmental context. In W. Damon, D. Kuhn, & R. S. Siegler (Eds.), Handbook of child psychology (pp. 309-370). New York: Wiley & Sons.
10. Bloom, P. (2002). Mindreading, communication and the learning of names for things. Mind & Language, 17, 37-54.
11. Brooks, R., & Meltzoff, A. N. (2002). The importance of eyes: How infants interpret adult looking behavior. Developmental Psychology, 38, 958-966.
12. Conway, C. M., & Christiansen, M. H. (2006). Statistical learning within and between modalities: Pitting abstract against stimulus-specific representations. Psychological Science, 17, 905-912.
13. Cunillera, T., Gomila, A., & Rodriguez-Fornells, A. (2008). Beneficial effects of word final stress in segmenting a new language: Evidence from ERPs. BMC Neuroscience, 9, 23.
14. Cutler, A., Mehler, J., Norris, D., & Segui, J. (1986). The syllable's differing role in the segmentation of French and English. Journal of Memory and Language, 25, 385-400.
15. Davis, C., & Kim, J. (2001). Repeating and remembering foreign language words: Implications for language teaching systems. Artificial Intelligence Review, 16, 37-47.
16. Dixon, N. F., & Spitz, L. (1980). The detection of auditory visual desynchrony. Perception, 9, 719-721.
17. Dodd, B. (1977). The role of vision in the perception of speech. Perception, 6, 31-40.
18. Dodd, B. (1979). Lip reading in infants: Attention to speech presented in-synchrony and outof-synchrony. Cognitive Psychology, 11, 478-484.
19. Dutoit, T., Pagel, N., Pierret, F., Bataille, O., & van der Vreken, O. (1996). The MBROLA project: Towards a set of high-quality speech synthesizers free of use for non-commercial purposes. In Proceedings of the Fourth International Conference on Spoken Language Processing (pp. 1393-1396). Philadelphia, PA, USA.
20. Echols, C. H. (1993). A perceptually-based model of children's earliest productions.Cognition,46,245-296.
21. Echols, C. H., & Newport, E. L. (1992). The role of stress and position in determining first words. Language Acquisition, 2, 189-220.
22. Fernald, A., & Simon, T. (1984). Expanded intonation contours in mothers' speech to newborns. Developmental Psychology, 20, 104-113.
23. Gibson, E. J., & Pick, A. D. (2000). An ecological approach to perceptual learning and development. New York: Oxford University Press.
24. Gogate, L. J., & Bahrick, L. E. (1998). Intersensory redundancy facilitates learning of arbitrary relations between vowel sounds and objects in seven-month-old infants. Journal of Experimental Child Psychology, 69,133-149.
25. Gogate, L. J., Bahrick, L. E., & Watson, J. D. (2000). A study of multimodal motherese: The role of temporal synchrony between verbal labels and gestures. Child Development, 71, 878-894.
26. Golinkoff, R. M., & Hirsh-Pasek, K. (2006). Baby wordsmith: From associationist to social sophisticate. Current Directions in Psychological Science, 15, 30-33.
27. Graf, E. K., Evans, J. L., Alibali, M. W., & Saffran, J. R. (2007). Can infants map meaning to newly segmented words? Statistical segmentation and word learning. Psychological Science, 18, 254-260.
28. Hashimoto, M., & Kumashiro, M. (2004). Intermodal timing cues for audio-visual speech recognition. Journal of UOEH, 26, 215-225.
29. Hirsh-Pasek, K., Kemler-Nelson, D. G., Jusczyk, P. W., Cassidy, K. W., Druss, B., & Kennedy, L. (1987). Clauses are perceptual units for young infants. Cognition, 26, 269-286.
30. Hollich, G., Hirsh-Pasek, K., Golinkoff, R. M., Brand, R. J., Brown, E., Chung, H. L., et al. (2000). Breaking the language barrier: An emergentist coalition model for the origins of word learning. Monographs of the Society for Research in Child Development, 65, i-123.
31. Hollich, G., Newman, R. S., & Jusczyk, P. W. (2005). Infants' use of synchronized visual information to separate streams of speech. Child Development, 76, 598-613.
32. Jusczyk, P. W., & Luce, P. A. (1994). Infants sensitivity to phonotactic patterns in the native language. Journal of Memory and Language, 33, 630-645.
33. Kuhl, P. K. (2004). Early language acquisition: Cracking the speech code. Nature Reviews Neuroscience, 5, 831-843.
34. Kuhl, P. K., & Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138-1141.
35. Kuhl, P. K., & Meltzoff, A. N. (1984). The intermodal representation of speech in infants. Infant Behavior & Development, 7, 361-381.
36. Lewkowicz, D. J. (1996). Perception of auditory-visual temporal synchrony in human infants. Journal of Experimental Psychology: Human Perception and Performance, 22, 1094-1106.
37. Macleod, A., & Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21, 131-141.
38. McGrath, M., & Summerfield, Q. (1985). Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. Journal of the Acoustical Society of America, 77, 678-685.
39. Mehler, J., Jusczyk, P., Lambertz, G., Halsted, N., Bertoncini, J., & Amiel-Tison, C. (1988). A precursor of language acquisition in young infants. Cognition, 29, 143-178.
40. Mills, A. E. (1987). The development of phonology in the blind child. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip reading (pp. 145-162). Hove, UK: Lawrence Erlbaum Associates.
41. Morgan, J. L., & Saffran, J. R. (1995). Emerging integration of sequential and suprasegmental information in preverbal speech segmentation. Child Development, 66, 911-936.
42. Morton, J., Marcus, S. M., & Frankish, C. (1976). Perceptual centres (P-centres). Psychological Review, 83,405-408.
43. Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996). Temporal constraints on the McGurk effect. Perception & Psychophysics, 58, 351-362.
44. Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility: Head movement improves auditory speech perception. Psychological Science, 15, 133-137.
45. Otake, T., Hatano, G., Cutler, A., & Mehler, J. (1993). Mora or syllable? Speech segmentation in Japanese. Journal of Memory and Language, 32, 258-278.
46. Pike, K. L. (1945). The intonation of American English. Ann Arbor, MI: University of Michigan Press.
47. Ramus, F., Hauser, M. D., Miller, C., Morris, D., & Mehler, J. (2000). Language discrimination by human newborn and cotton-top tamarin monkeys. Science, 288, 349-351.
48. Ramus, F., & Mehler, J. (1999). Language identification with suprasegmental cues: A study based on speech resynthesis. Journal of the Acoustical Society of America, 105, 512-521.
49. Ramus, F., Nespor, M., & Mehler, J. (1999). Correlates of linguistic rhythm in the speech signal. Cognition, 73,265-292.
50. Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97-114). Hove, UK: Lawrence Erlbaum Associates.
51. Rosenblum, L. D., & Saldana, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-331.
52. Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996a). Statistical learning by 8-month-old infants. Science, 274,1926-1928.
53. Saffran, J. R., Newport, E. L., & Aslin, R. N. (1996b). Word segmentation: The role of distributional cues. Journal of Memory and Language, 35, 606-621.
54. Sanders, D. A., & Goodrich, S. J. (1971). The relative contribution of visual and auditory components of speech to speech intelligibility as a function of three conditions of frequency distortion. Journal of Speech, Language, and Hearing Research, 14, 154-159.
55. Scott, S. K. (1998). The point of P-centres. Psychological Research, 61, 4-11.
56. Snodgrass, J. G., & Vanderwart, M. (1980). Standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6, 174-215.
57. Soto-Faraco, S, & Alsius, A. (2009). Deconstructing the McGurk-MacDonald illusion. Journal of Experimental Psychology: Human Perception and Performance, 35, 580-587.
58. Spelke, E. S. (1979). Perceiving bimodally specified events in infancy. Developmental Psychology, 15, 626-636.
59. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
60. Thompson, L. A., & Ogden, W. C. (1995). Visible speech improves human language understanding: Implications for speech processing systems. Artificial Intelligence Review, 9, 347-358.
61. Tincoff, R., Hauser, M., Tsao, F., Spaepen, G., Ramus, F., & Mehler, J. (2005). The role of speech rhythm in language discrimination: Further test with a non-human primate. Developmental Science, 8, 26-35.
62. Tomasello, M. (2003). Constructing a language. Cambridge, MA: Harvard University Press.
63. Tomasello, M., & Barton, M. (1994). Learning words in non-ostensive contexts. Developmental Psychology, 30,639-650.
64. Van Wassenhove,V.,Grant,K.W.,& Poeppel,D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45, 598-607.
65. Yu, C., & Ballard, D. H. (2004). A multimodal learning interface for grounding spoken language in sensory perceptions. ACM Transactions on Applied Perceptions, 1, 57-80.
66. Zukow-Goldring, P. (1997). A social ecological realist approach to the emergence of the lexicon: Educating attention to amodal invariants in gesture and speech. In C. Dent-Read & P. Zukow-Goldring (Eds.), Evolving explanations of development: Ecological approaches to organism-environment systems (pp. 199-252). Washington, DC: American Psychological Association.