Haptic and visual information speed up the neural processing of auditory speech in live dyadic interactions

Neuropsychologia

Volumn 57, Issue 1, 2014, Pages 71-77

  Treille, Avril ^a   Cordeboeuf, Camille ^a   Vilain, Coriandre ^a   Sato, Marc ^a  

^a UNIV GRENOBLE ALPES   (France)

Author keywords

Audio haptic speech perception; Audio visual speech perception; EEG; Multisensory interactions

Indexed keywords

ADULT; ARTICLE; AUDIOHAPTIC SPEECH PERCEPTION; AUDIOVISUAL SPEECH PERCEPTION; BRAIN FUNCTION; CONTROLLED STUDY; DYADIC INTERACTION; ELECTROENCEPHALOGRAM; EVOKED AUDITORY RESPONSE; FEMALE; HUMAN; MALE; NORMAL HUMAN; PERCEPTIVE DISCRIMINATION; SENSORY STIMULATION; SIGNAL PROCESSING; SPEECH PERCEPTION; TOUCH; VISION; ACOUSTICS; ANALYSIS OF VARIANCE; AUDITORY STIMULATION; BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED RESPONSE; MIDDLE AGED; PHOTOSTIMULATION; PHYSIOLOGY; REACTION TIME; YOUNG ADULT;

ACOUSTIC STIMULATION; ACOUSTICS; ADULT; ANALYSIS OF VARIANCE; BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; FEMALE; HUMANS; MALE; MIDDLE AGED; PHOTIC STIMULATION; REACTION TIME; SPEECH PERCEPTION; TOUCH; TOUCH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84896978375     PISSN: 00283932     EISSN: 18733514     Source Type: Journal    
DOI: 10.1016/j.neuropsychologia.2014.02.004     Document Type: Article

References (32)

1. Alcorn S. The Tadoma method. Volta Review 1932, 34:195-198.
2. Arnal L.H., Giraud A.L. Cortical oscillations and sensory predictions. Trends in Cognitive Science 2012, 16(7):390-398.
3. Arnal L.H., Morillon B., Kell C.A., Giraud A.L. Dual neural routing of visual facilitation in speech processing. The Journal of Neuroscience 2009, 29(43):13445-13453.
4. Arnal L.H., Wyart V., Giraud A.L. Transitions in neural oscillations reflect prediction errors generated in audiovisual speech. Nature Neuroscience 2011, 14(6):797-801.
5. Benoît C., Mohamadi T., Kandel S.D. Effects on phonetic context on audio-visual intelligibility of French. Journal of Speech and Hearing Research 1994, 37:1195-1203.
6. Besle J., Fort A., Delpuech C., Giard M.H. Bimodal speech: Early suppressive visual effects in human auditory cortex. European journal of Neuroscience 2004, 20:2225-2234.
7. Boersma, P. & Weenink, D. (2013). Praat: doing phonetics by computer. Computer program, Version 5.3.42, retrieved 2 March 2013 from http://www.praat.org/.
8. Delorme A., Makeig S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics. Journal of Neuroscience Methods 2004, 134:9-21.
9. Desmedt J.E., Tomberg C. Mapping early somatosensory evoked potentials in selective attention: Critical evaluation of control conditions used for titrating by difference the cognitive P30, P40, P100 and N140. Electroencephalography and Clinical Neurophysiology 1989, 74(5):321-346.
10. Fowler C., Dekle D. Listening with eye and hand: Crossmodal contributions to speech perception. Journal of Experimental Psychology: Human Perception and Performance 1991, 17:816-828.
11. Giard M.H., Fort A., Mouchetant-Rostaing Y., Pernier J. Neurophysiological mechanisms of auditory selective attention in humans. Front. Bioscience 2000, 5:84-94.
12. Gick B., Jóhannsdóttir K.M., Gibraiel D., Mühlbauer M. Tactile enhancement of auditory and visual speech perception in untrained perceivers. Journal of Acoustical Society of America 2008, 123:72-76.
13. Jousmäki V., Hari R. Parchment-skin illusion: Sound-biased touch. Current Biology 1998, 8(6):190.
14. Klucharev V., Möttönen R., Sams M. Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Brain Research. Cognitive Brain Research 2003, 18:65-75.
15. Leotta D.F., Rabinowitz W.M., Reed C.M., Drulach N.I. Preliminary results of speech-reception tests obtained with the synthetic Tadoma system. Journal of Rehabilitation Research and Development 1988, 25(4):45-52.
16. Lütkenhöner B., Lammertmann C., Simões C., Hari R. Magnetoencephalographic correlates of audiotactile interaction. NeuroImage 2002, 15(3):509-522.
17. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
18. Näätänen R. Attention and Brain Function 1992, Lawrence Erlbaum Associates, Hillsdale, NJ.
19. Näätänen R., Picton T.W. The N1 wave of the human electric and magnetic response to sound: A review and an analysis of the component structure. Psychophysiology 1987, 24:375-425.
20. Navarra J., Soto-Faraco S. Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research 2005, 71(1):4-12.
21. Norton S.J., Schultz M.C., Reed C.M., Braida L.D., Durlach N.I., Rabinowitz W.M., et al. Analytic study of the Tadoma method: Background and preliminary results. Journal of Speech and Hearing Research 1977, 20:574-595.
22. Pilling M. Auditory event-related potentials (ERPs) in audiovisual speech perception. Journal of Speech, Language, and Hearing Research 2010, 52(4):1073-1081.
23. Reisberg D., McLean J., Goldfield A. Easy to hear but hard tounderstand: A lipreading advantage with intact auditory stimuli. Hearing by eye: The psychology of lipreading 1987, 97-113. Lawrence Erlbaum Associates, London (UK). R. Campbell, B. Dodd (Eds.).
24. Sato M., Cavé C., Ménard L., Brasseur L. Auditory-tactile speech perception in congenitally blind and sighted adults. Neuropsychologia 2010, 48(12):3683-3686.
25. Scherg M., Von Cramon D. Evoked dipole source potentials of the human auditory cortex. Electroencephalography and Clinical Neurology 1986, 65:344-360.
26. Schwartz J.L., Ménard L., Basirat A., Sato M. The perception for action control theory (PACT): A perceptuo-motor theory of speech perception. Journal of Neurolinguistics 2012, 25(5):336-354.
27. Schwartz J.-L., Robert-Ribes J., Escudier P. Ten years after summerfield: A taxonomy of models for audio-visual fusion in speech perception. Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech 1998, 85-108. Psychology Press, Hove, UK. R. Campbell, B. Dodd, D. Burnham (Eds.).
28. Stekelenburg J.J., Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience 2007, 19:1964-1973.
29. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of Acoustical Society of America 1954, 26:212-215.
30. van Wassenhove V., Grant K.W., Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America 2005, 102:1181-1186.
31. Vroomen J., Stekelenburg J.J. Visual anticipatory information modulates multisensory interactions of artificial audiovisual stimuli. Journal of Cognitive Neuroscience 2010, 22:1583-1596.
32. Wolpert D.M., Ghahramani Z., Jordan M.I. An internal model for sensorimotor integration. Science 1995, 269(5232):1880-1882.