The whole is more than the sum of its parts - Audiovisual processing of phonemes investigated with ERPs

Brain and Language

Volumn 124, Issue 3, 2013, Pages 213-224

  Hessler, Dörte ^a   Jonkers, Roel ^a   Stowe, Laurie ^a   Bastiaanse, Roelien ^a  

^a UNIVERSITY OF GRONINGEN   (Netherlands)

Author keywords

Active oddball task; ERP; MMN; P3; Phonemic dimensions; Speechreading

Indexed keywords

ADULT; AGED; ARTICLE; ATTENTION; AUDIOVISUAL EQUIPMENT; AUDIOVISUAL PROCESSING; AUDITORY STIMULATION; CONTROLLED STUDY; EVENT RELATED POTENTIAL; EVOKED RESPONSE; FEMALE; HUMAN; HUMAN EXPERIMENT; LANGUAGE PROCESSING; MALE; NORMAL HUMAN; PHONEME; PURE TONE AUDIOMETRY; VISUAL STIMULATION;

ACOUSTIC STIMULATION; AGED; AUDITORY PERCEPTION; BRAIN; COMPREHENSION; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS, AUDITORY; EVOKED POTENTIALS, VISUAL; FEMALE; HUMANS; MALE; MIDDLE AGED; REACTION TIME; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 84873435529     PISSN: 0093934X     EISSN: 10902155     Source Type: Journal    
DOI: 10.1016/j.bandl.2012.12.006     Document Type: Article

References (33)

1. Aaltonen O., Niemi P., Nyrke T., Tuhkanen M. Event-related brain potentials and the perception of a phonetic continuum. Biological Psychology 1987, 24:197-207.
2. Boersma, P., & Weenink, D. (2009). Praat: Doing phonetics by computer [computer progam]. Version 5.2.17. Retrieved 06.02.09. http://www.praat.org/.
3. Campbell R. Tracing lip movements: Making speech visible. Visible Language 1988, 22:32-57.
4. Campbell R. Lipreading, neuropsychology, and immediate memory. Neuropsychological impairments of short-term memory 1990, 268-286. Cambridge University Press, Cambridge. G. Vallar, T. Shallice (Eds.).
5. Colin C., Radeau M., Soquet A., Deltenre P. Generalization of the generation of an MMN by illusory McGurk percepts: Voiceless consonants. Clinical Neurophysiology 2004, 115:1989-2000.
6. Colin C., Radeau M., Soquet A., Demolin D., Colin F., Deltenre P. Mismatch negativity evoked by the McGurk-MacDonald effect: A phonetic representation within short-term memory. Clinical Neurophysiology 2002, 113:495-506.
7. Courchesne E., Hillyard S.A., Galambos R. Stimulus novelty, task relevance and the visual evoked potential in man. Electroencephalography and Clinical Neurophysiology 1975, 39:131-143.
8. Deacon D., Nousak J.M., Pilotti M., Ritter W., Yang C.-M. Automatic change detection: Does the auditory system use representations of individual stimulus features or gestalts?. Psychophysiology 1998, 35:413-419.
9. Duncan-Johnson C.C., Donchin E. On quantifying surprise: The variation of event-related potentials with subjective probability. Psychophysiology 1977, 14:456-467.
10. Isreal J.B., Chesney G.L., Wickens C.D., Donchin E. P300 and tracking difficulty: Evidence for multiple resources in dual-task performance. Psychophysiology 1980, 17:259-273.
11. Johnson R. P300: A model of the variables controlling its amplitude. Annals of the New York Academy of Sciences 1984, 425:223-229.
12. Johnson R. A triarchic model of P300 amplitude. Psychophysiology 1986, 23:367-384.
13. Kraus N., McGee T., Sharma A.M.A., Carrell T., Nicol T.B.S. Mismatch negativity event-related potential elicited by speech stimuli. Ear and Hearing 1992, 13:158-164. [using smart source parsing].
14. Ladefoged P. Vowels and consonants: An introduction to the sounds of languages 2001, Blackwell Publishers Ltd., Oxford.
15. Lawson E.A., Gaillard A.W.K. Mismatch negativity in a phonetic discrimination task. Biological Psychology 1981, 13:281-288.
16. Luck S.J. An introduction to the event-related potential technique 2005, MIT Press, Cambridge, MA.
17. Maiste A.C., Wiens A.S., Hunt M.J., Scherg M., Picton T.W. Event-related potentials and the categorical perception of speech sounds. Ear and Hearing 1995, 16:68-89. [article using smart source parsing].
18. McClelland J.L., Elman J.L. The TRACE model of speech perception. Cognitive Psychology 1986, 18:1-86.
19. McGee T.J., King C., Tremblay K., Nicol T.G., Cunningham J., Kraus N. Long-term habituation of the speech-elicited mismatch negativity. Psychophysiology 2001, 38:653-658.
20. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
21. Morin R.E., Forrin B. Response equivocation and reaction time. Journal of Experimental Psychology 1963, 66:30-36.
22. Möttönen R., Krause C.M., Tiippana K., Sams M. Processing of changes in visual speech in the human auditory cortex. Cognitive Brain Research 2002, 13:417-425.
23. Näätänen R., Gaillard A.W.K., Mäntysalo S. Early selective-attention effect on evoked potential reinterpreted. Acta Psychologica 1978, 42:313-329.
24. Näätänen R., Simpson M., Loveless N.E. Stimulus deviance and evoked potentials. Biological Psychology 1982, 14:53-98.
25. Novak G., Ritter W., Vaughan H.G. Mismatch detection and the latency of temporal judgments. Psychophysiology 1992, 29:398-411.
26. Pazo-Alvarez P., Cadaveira F., Amenedo E. MMN in the visual modality: A review. Biological Psychology 2003, 63:199-236.
27. Reisberg D., McLean J., Goldfield A. Easy to hear, but hard to understand: A lipreading advantage with intact auditory stimuli. Hearing by eye: The psychology of lipreading 1987, 97-114. Lawrence Erlbaum, London. B. Dodd, R. Campbell (Eds.).
28. Saint-Amour D., De Sanctis P., Molholm S., Ritter W., Foxe J.J. Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion. Neuropsychologia 2007, 45:587-597.
29. Sams M., Aulanko R., Aaltonen O., Näätänen R. Event-related potentials to infrequent changes in synthesized phonetic stimuli. Journal of Cognitive Neuroscience 1990, 2:344-357.
30. Sams M., Paavilainen P., Alho K., Näätänen R. Auditory frequency discrimination and event-related potentials. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 1985, 62:437-448.
31. Sharma A., Dorman M.F. Cortical auditory evoked potential correlates of categorical perception of voice-onset time. The Journal of the Acoustical Society of America 1999, 106:1078-1083.
32. Sharma A., Kraus N., McGee T., Carrell T., Nicol T. Acoustic versus phonetic representation of speech as reflected by the mismatch negativity event-related potential. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 1993, 88:64-71.
33. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America 1954, 26:212-215.