Lipreading and covert speech production similarly modulate human auditory-cortex responses to pure tones

Journal of Neuroscience

Volumn 30, Issue 4, 2010, Pages 1314-1321

  Kauramäki, Jaakko ^a   Jääskeläinen, Iiro P ^a   Hari, Riitta ^a   Möttönen, Riikka ^b   Rauschecker, Josef P ^a,c   Sams, Mikko ^a  

^a AALTO UNIVERSITY   (Finland)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c GEORGETOWN UNIVERSITY MEDICAL CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY RESPONSE; CONTROLLED STUDY; FEMALE; GESTURE; HUMAN; HUMAN EXPERIMENT; LEFT HEMISPHERE; LIP READING; MAGNETOENCEPHALOGRAPHY; MALE; NORMAL HUMAN; PRIORITY JOURNAL; PURE TONE AUDIOMETRY; SPEECH; SPEECH PERCEPTION; VIDEORECORDING; VOWEL;

ACOUSTIC STIMULATION; ADULT; AUDITORY CORTEX; AUDITORY PATHWAYS; AUDITORY PERCEPTION; BRAIN MAPPING; EVOKED POTENTIALS, AUDITORY; FEMALE; FUNCTIONAL LATERALITY; HUMANS; LIPREADING; MAGNETOENCEPHALOGRAPHY; MALE; NERVE NET; NEURAL INHIBITION; NEUROPSYCHOLOGICAL TESTS; PERCEPTUAL MASKING; PHOTIC STIMULATION; PITCH DISCRIMINATION; REACTION TIME; SPEECH ACOUSTICS; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 75349097161     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.1950-09.2010     Document Type: Article

References (66)

1. Besle J, Fort A, Delpuech C, Giard MH (2004) Bimodal speech: early suppressive visual effects in human auditory cortex. Eur J Neurosci 20:2225-2234.
2. Busse L, Roberts KC, Crist RE, Weissman DH, Woldorff MG (2005) The spread of attention across modalities and space in a multisensory object. Proc Natl Acad Sci U S A 102:18751-18756.
3. Calvert GA, Bullmore ET, Brammer MJ, Campbell R, Williams SCR, McGuire PK, Woodruff PWR, Iversen SD, David AS (1997) Activation of auditory cortex during silent lipreading. Science 276:593-596.
4. Cappe C, Barone P (2005) Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur J Neurosci 22:2886-2902.
5. Cappe C, Morel A, Barone P, Rouiller EM (2009a) The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cereb Cortex 19:2025-2037.
6. Cappe C, Rouiller EM, Barone P (2009b) Multisensory anatomical pathways. Hear Res 258:28-36.
7. Christoffels IK, Formisano E, Schiller NO (2007) Neural correlates of verbal feedback processing: an fMRI study employing overt speech. Hum Brain Mapp 28:868-879.
8. Curio G, Neuloh G, Numminen J, Jousmäki V, Hari R (2000) Speaking modifies voice-evoked activity in the human auditory cortex. Hum Brain Mapp 9:183-191.
9. Falchier A, Clavagnier S, Barone P, Kennedy H (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749-5759.
10. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25:5004-5012.
11. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
12. Hackett TA, De La Mothe LA, Ulbert I, Karmos G, Smiley J, Schroeder CE (2007) Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. J Comp Neurol 502:924-952.
13. Hämäläinen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounasmaa OV (1993) Magnetoencephalography - theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev Mod Phys 65:413-497.
14. Hari R (1990) The neuromagnetic method in the study of the human auditory cortex. In: Auditory evoked magnetic fields and potentials. Advances in audiology, Vol 6 (Grandori F, Hoke M, Romani G, eds), pp 222-282. Basel: Karger.
15. Heinks-Maldonado TH, Mathalon DH, Gray M, Ford JM (2005) Fine-tuning of auditory cortex during speech production. Psychophysiology 42:180-190.
16. Heinks-Maldonado TH, Nagarajan SS, Houde JF (2006) Magnetoencephalographic evidence for a precise forward model in speech production. Neuroreport 17:1375-1379.
17. Hertrich I, Mathiak K, Lutzenberger W, Menning H, Ackermann H (2007) Sequential audiovisual interactions during speech perception: a whole-head MEG study. Neuropsychologia 45:1342-1354.
18. Hertrich I, Mathiak K, Lutzenberger W, Ackermann H (2009) Time course of early audiovisual interactions during speech and nonspeech central auditory processing: a magnetoencephalography study. J Cogn Neurosci 21:259-274.
19. Houde JF, Nagarajan SS, Sekihara K, Merzenich MM (2002) Modulation of the auditory cortex during speech: an MEG study. J Cogn Neurosci 14:1125-1138.
20. Hromádka T, DeWeese MR, Zador AM (2008) Sparse representation of sounds in the unanesthetized auditory cortex. PLoS Biol 6:e16.
21. Jääskeläinen IP, Ojanen V, Ahveninen J, Auranen T, Levänen S, Möttönen R, Tarnanen I, Sams M (2004) Adaptation of neuromagnetic N1 responses to phonetic stimuli by visual speech in humans. Neuroreport 15:2741-2744.
22. Jääskeläinen IP, Ahveninen J, Belliveau JW, Raij T, Sams M (2007) Short-term plasticity in auditory cognition. Trends Neurosci 30:653-661.
23. Jääskeläinen IP, Kauramäki J, Tujunen J, Sams M (2008) Formant transition-specific adaptation by lipreading of left auditory cortex N1m. Neuroreport 19:93-97.
24. Kaas JH, Hackett TA (2000) Subdivisions of auditory cortex and processing streams in primates. Proc Natl Acad Sci U S A 97:11793-11799.
25. Kayser C, Logothetis NK (2009) Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Front Integr Neurosci 3:7.
26. Klucharev V, Möttönen R, Sams M (2003) Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Brain Res Cogn Brain Res 18:65-75.
27. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279-292.
28. Lewis JW, Van Essen DC (2000) Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J Comp Neurol 428:112-137.
29. Loveless N, Hari R, Hämäläinen M, Tiihonen J (1989) Evoked responses of human auditory cortex may be enhanced by preceding stimuli. Electroencephalogr Clin Neurophysiol 74:217-227.
30. Lütkenhöner B, Krumbholz K, Seither-Preisler A (2003) Studies of tonotopy based on wave N100 of the auditory evoked field are problematic. Neuroimage 19:935-949.
31. MacSweeney M, Amaro E, Calvert GA, Campbell R, David AS, McGuire P, Williams SC, Woll B, Brammer MJ (2000) Silent speechreading in the absence of scanner noise: an event-related fMRI study. Neuroreport 11:1729-1733.
32. Martikainen MH, Kaneko K, Hari R (2005) Suppressed responses to self-triggered sounds in the human auditory cortex. Cereb Cortex 15:299-302.
33. Massaro DW, Cohen MM, Smeele PMT (1996) Perception of asynchronous and conflicting visual and auditory speech. J Acoust Soc Am 100:1777-1786.
34. Miki K, Watanabe S, Kakigi R (2004) Interaction between auditory and visual stimulus relating to the vowel sounds in the auditory cortex in humans: a magnetoencephalographic study. Neurosci Lett 357:199-202.
35. Musacchia G, Sams M, Nicol T, Kraus N (2006) Seeing speech affects acoustic information processing in the human brainstem. Exp Brain Res 168:1-10.
36. Nishitani N, Hari R (2002) Viewing lip forms: cortical dynamics. Neuron 36:1211-1220.
37. Nishitani N, Schürmann M, Amunts K, Hari R (2005) Broca's region: from action to language. Physiology 20:60-69.
38. Numminen J, Curio G (1999) Differential effects of overt, covert and replayed speech on vowel-evoked responses of the human auditory cortex. Neurosci Lett 272:29-32.
39. Otazu GH, Tai LH, Yang Y, Zador AM (2009) Engaging in an auditory task suppresses responses in auditory cortex. Nat Neurosci 12:646-654.
40. Pantev C, Bertrand O, Eulitz C, Verkindt C, Hampson S, Schuierer G, Elbert T (1995) Specific tonotopic organizations of different areas of the human auditory cortex revealed by simultaneous magnetic and electric recordings. Electroencephalogr Clin Neurophysiol 94:26-40.
41. Papanicolaou AC, Raz N, Loring DW, Eisenberg HM (1986) Brain stem evoked response suppression during speech production. Brain Lang 27:50-55.
42. Paus T, Perry DW, Zatorre RJ, Worsley KJ, Evans AC (1996) Modulation of cerebral blood flow in the human auditory cortex during speech: role of motor-to-sensory discharges. Eur J Neurosci 8:2236-2246.
43. Pekkola J, Ojanen V, Autti T, Jääskeläinen IP, Möttönen R, Tarkiainen A, Sams M (2005) Primary auditory cortex activation by visual speech: an fMRI study at 3 T. Neuroreport 16:125-128.
44. Rauschecker JP, Tian B, Hauser M (1995) Processing of complex sounds in the macaque nonprimary auditory cortex. Science 268:111-114.
45. Rizzolatti G, Arbib MA (1998) Language within our grasp. Trends Neurosci 21:188-194.
46. Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169-192.
47. Rockland KS, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50:19-26.
48. Sams M, Hari R, Rif J, Knuutila J (1993) The human auditory sensory memory trace persists about 10 sec: neuromagnetic evidence. J Cogn Neurosci 5:363-370.
49. Sams M, Möttönen R, Sihvonen T (2005) Seeing and hearing others and oneself talk. Brain Res Cogn Brain Res 23:429-435.
50. Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14:187-198.
51. Schroeder CE, Smiley J, Fu KG, McGinnis T, O'Connell MN, Hackett TA (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5-17.
52. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A (2008) Neuronal oscillations and visual amplification of speech. Trends Cogn Sci 12:106-113.
53. Sherman SM, Guillery RW (2002) The role of the thalamus in the flow of information to the cortex. Philos Trans R Soc Lond B Biol Sci 357:1695-1708.
54. Skipper JI, van Wassenhove V, Nusbaum HC, Small SL (2007) Hearing lips and seeing voices: how cortical areas supporting speech production mediate audiovisual speech perception. Cereb Cortex 17:2387-2399.
55. Steinstraeter O, Teismann IK, Wollbrink A, Suntrup S, Stoeckigt K, Dziewas R, Pantev C (2009) Local sphere-based co-registration for SAM group analysis in subjects without individual MRI. Exp Brain Res 193:387-396.
56. Stekelenburg JJ, Vroomen J (2005) An event-related potential investigation of the time-course of temporal ventriloquism. Neuroreport 16:641-644.
57. Stekelenburg JJ, Vroomen J (2007) Neural correlates of multisensory integration of ecologically valid audiovisual events. J Cogn Neurosci 19:1964-1973.
58. Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212-215.
59. Talavage TM, Sereno MI, Melcher JR, Ledden PJ, Rosen BR, Dale AM (2004) Tonotopic organization in human auditory cortex revealed by progressions of frequency sensitivity. J Neurophysiol 91:1282-1296.
60. Valtonen J, May P, Mäkinen V, Tiitinen H (2003) Visual short-term memory load affects sensory processing of irrelevant sounds in human auditory cortex. Brain Res Cogn Brain Res 17:358-367.
61. van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962-974.
62. van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci U S A 102:1181-1186.
63. van Wassenhove V, Grant KW, Poeppel D (2007) Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45:598-607.
64. Wang X, Lu T, Snider RK, Liang L (2005) Sustained firing in auditory cortex evoked by preferred stimuli. Nature 435:341-346.
65. Warren RM, Riener KR, Bashford JA Jr, Brubaker BS (1995) Spectral redundancy: intelligibility of sentences heard through narrow spectral slits. Percept Psychophys 57:175-182.
66. Woods DL, Alho K, Algazi A (1992) Intermodal selective attention. I. Effects on event-related potentials to lateralized auditory and visual stimuli. Electroencephalogr Clin Neurophysiol 82:341-355.