Task-irrelevant visual letters interact with the processing of speech sounds in heteromodal and unimodal cortex

European Journal of Neuroscience

Volumn 28, Issue 3, 2008, Pages 500-509

  Blau, Vera ^a   Van Atteveldt, Nienke ^a   Formisano, Elia ^a   Goebel, Rainer ^a   Blomert, Leo ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

Author keywords

fMRI; Integration; Multisensory; Noise

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BEHAVIOR; BRAIN REGION; CONTROLLED STUDY; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; MALE; PRIORITY JOURNAL; SPEECH; TASK PERFORMANCE; TEMPORAL CORTEX; VISION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; BEHAVIOR; CEREBRAL CORTEX; HUMANS; LANGUAGE; MAGNETIC RESONANCE IMAGING; MALE; NETHERLANDS; NEUROPSYCHOLOGICAL TESTS; PATTERN RECOGNITION, VISUAL; PHONETICS; PHOTIC STIMULATION; READING; SPEECH; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 47849117162     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/j.1460-9568.2008.06350.x     Document Type: Article

References (55)

1. van Atteveldt, N., Formisano, E., Goebel, R. Blomert, L. (2004) Integration of letters and speech sounds in the human brain. Neuron, 43, 271 282.
2. van Atteveldt, N.M., Formisano, E., Blomert, L. Goebel, R. (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb. Cortex, 17, 962 974.
3. Beauchamp, M.S., Argall, B.D., Bodurka, J., Duyn, J.H. Martin, A. (2004) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci., 7, 1190 1192.
4. Binder, J.R., Frost, J.A., Hammeke, T.A., Rao, S.M. Cox, R.W. (1996) Function of the left planum temporale in auditory and linguistic processing. Brain, 119 (Pt 4 1239 1247.
5. Booth, J.R., Cho, S., Burman, D.D. Bitan, T. (2007) Neural correlates of mapping from phonology to orthography in children performing an auditory spelling task. Dev. Sci., 10, 441 451.
6. Booth, J.R., Burman, D.D., Meyer, J.R., Gitelman, D.R., Parrish, T.B. Mesulam, M.M. (2002) Functional anatomy of intra- and cross-modal lexical tasks. Neuroimage, 16, 7 22.
7. Borowsky, R., Owen, W.J. Fonos, N. (1999) Reading speech and hearing print: constraining models of visual word recognition by exploring connections with speech perception. Can. J. Exp. Psychol., 53, 294 305.
8. Buchsbaum, B.R., Olsen, R.K., Koch, P.F., Kohn, P., Kippenhan, J.S. Berman, K.F. (2005) Reading, hearing, and the planum temporale. Neuroimage, 24, 444 454.
9. Bushara, K.O., Hanakawa, T., Immisch, I., Toma, K., Kansaku, K. Hallett, M. (2003) Neural correlates of cross-modal binding. Nat. Neurosci., 6, 190 195.
10. Busse, L., Roberts, K.C., Crist, R.E., Weissman, D.H. Woldorff, M.G. (2005) The spread of attention across modalities and space in a multisensory object. Proc. Natl Acad. Sci. USA, 102, 18751 18756.
11. Callan, D.E., Jones, J.A., Munhall, K., Callan, A.M., Kroos, C. Vatikiotis-Bateson, E. (2003) Neural processes underlying perceptual enhancement by visual speech gestures. Neuroreport, 14, 2213 2218.
12. Calvert, G.A., Campbell, R. Brammer, M.J. (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol., 10, 649 657.
13. Cohen, L. Dehaene, S. (2004) Specialization within the ventral stream: the case for the visual word form area. Neuroimage, 22, 466 476.
14. Cohen, L., Lehericy, S., Chochon, F., Lemer, C., Rivaud, S. Dehaene, S. (2002) Language-specific tuning of visual cortex? Functional properties of the Visual Word Form Area. Brain, 125, 1054 1069.
15. Dijkstra, T., Schreuder, R. Frauenfelder, U.H. (1989) Grapheme contex effects on phonemic processing. Lang. Speech, 32, 98 108.
16. Dijkstra, T., Frauenfelder, U.H. Schreuder, R. (1993) Bidirectional grapheme-phoneme activation in a bimodal detection task. J. Exp. Psychol. Hum. Percept. Perform., 19, 931 950.
17. Ehri, L.C. (2005) Development of sight word reading:phases and findings. In Snowling, M.J. Hulme, C. (Eds The Science of Reading: A Handbook. Blackwell Publishing, Oxford, pp. 135 145.
18. Fiebach, C.J., Friederici, A.D., Muller, K. von Cramon, D.Y. (2002) fMRI evidence for dual routes to the mental lexicon in visual word recognition. J. Cogn. Neurosci., 14, 11 23.
19. Fiez, J.A., Balota, D.A., Raichle, M.E. Petersen, S.E. (1999) Effects of lexicality, frequency, and spelling-to-sound consistency on the functional anatomy of reading. Neuron, 24, 205 218.
20. Flowers, D.L., Jones, K., Noble, K., VanMeter, J., Zeffiro, T.A., Wood, F.B. Eden, G.F. (2004) Attention to single letters activates left extrastriate cortex. Neuroimage, 21, 829 839.
21. Forman, S.D., Cohen, J.D., Fitzgerald, M., Eddy, W.F., Mintun, M.A. Noll, D.C. (1995) Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med., 33, 636 647.
22. Frost, R. Katz, L. (1989) Orthographic depth and the interaction of visual and auditory processing in word recognition. Mem. Cognit., 17, 302 310.
23. Froyen, D., van Atteveldt, N., Bonte, M. Blomert, L. (2008) Cross-modal enhancement of the MMN to speech-sounds indicates early and automatic integration of letters and speech-sounds. Neurosci. Lett., 430, 23 28.
24. Gauthier, I., Tarr, M.J., Anderson, A.W., Skudlarski, P. Gore, J.C. (1999) Activation of the middle fusiform 'face area' increases with expertise in recognizing novel objects. Nat. Neurosci., 2, 568 573.
25. Goebel, R., Esposito, F. Formisano, E. (2006) Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: from single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum. Brain Mapp., 27, 392 401.
26. Hasegawa, T., Matsuki, K.-I., Ueno, T., Maeda, Y., Matsue, Y., Konishi, Y. Sadato, N. (2004) Learned audio-visual cross-modal associations in observed piano playing activate the left planum temporale. An fMRI study. Cogn. Brain Res., 20, 510 518.
27. Hashimoto, R. Sakai, K.L. (2004) Learning letters in adulthood: direct visualization of cortical plasticity for forming a new link between orthography and phonology. Neuron, 42, 311 322.
28. Hein, G., Doehrmann, O., Muller, N.G., Kaiser, J., Muckli, L. Naumer, M.J. (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J. Neurosci., 27, 7881 7887.
29. Hwang, J.H., Wu, C.W., Chen, J.H. Liu, T.C. (2006) The effects of masking on the activation of auditory-associated cortex during speech listening in white noise. Acta Otolaryngol., 126, 916 920.
30. Jäncke, L., Wüstenberg, T., Scheich, H. Heinze, H.J. (2002) Phonetic perception and the temporal cortex. NeuroImage, 15, 733 746.
31. Kronbichler, M., Hutzler, F., Wimmer, H., Mair, A., Staffen, W. Ladurner, G. (2004) The visual word form area and the frequency with which words are encountered: evidence from a parametric fMRI study. Neuroimage, 21, 946 953.
32. Longcamp, M., Anton, J.L., Roth, M. Velay, J.L. (2003) Visual presentation of single letters activates a premotor area involved in writing. Neuroimage, 19, 1492 1500.
33. Macaluso, E., George, N., Dolan, R., Spence, C. Driver, J. (2004) Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage, 21, 725 732.
34. McCandliss, B.D., Cohen, L. Dehaene, S. (2003) The visual word form area: expertise for reading in the fusiform gyrus. Trends Cogn. Sci., 7, 293 299.
35. McDonald, J.J., Teder-Salejarvi, W.A., Di Russo, F. Hillyard, S.A. (2003) Neural substrates of perceptual enhancement by cross-modal spatial attention. J. Cogn. Neurosci., 15, 10 19.
36. Nakada, T., Fujii, Y., Yoneoka, Y. Kwee, I.L. (2001) Planum temporale: where spoken and written language meet. Eur. Neurol., 46, 121 125.
37. Obleser, J., Boecker, H., Drzezga, A., Haslinger, B., Hennenlotter, A., Roettinger, M., Eulitz, C. Rauschecker, J.P. (2006) Vowel sound extraction in anterior superior temporal cortex. Hum. Brain Mapp., 27, 562 571.
38. Olson, I.R., Gatenby, J.C. Gore, J.C. (2002) A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res. Cogn. Brain Res., 14, 129 138.
39. Perre, L. Ziegler, J.C. (2008) On-line activation of orthography in spoken word recognition. Brain Res., 1188, 132 138.
40. Proverbio, A.M., Vecchi, L. Zani, A. (2004) From orthography to phonetics: ERP measures of grapheme-to-phoneme conversion mechanisms in reading. J. Cogn. Neurosci., 16, 301 317.
41. Raij, T., Uutela, K. Hari, R. (2000) Audiovisual integration of letters in the human brain. Neuron, 28, 617 625.
42. Schroeder, C.E. Foxe, J.J. (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res., 14, 187 198.
43. Scott, S.K. (2005) Auditory processing - speech, space and auditory objects. Curr. Opin. Neurobiol., 15, 197 201.
44. Scott, S.K. Johnsrude, I.S. (2003) The neuroanatomical and functional organization of speech perception. Trends Neurosci., 26, 100 107.
45. Scott, S.K., Blank, C.C., Rosen, S. Wise, R.J. (2000) Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123 (Pt 12 2400 2406.
46. Seifritz, E., Esposito, F., Hennel, F., Mustovic, H., Neuhoff, J.G., Bilecen, D., Tedeschi, G., Scheffler, K. Di Salle, F. (2002) Spatiotemporal pattern of neural processing in the human auditory cortex. Science, 297, 1706 1708.
47. Sekiyama, K., Kanno, I., Miura, S. Sugita, Y. (2003) Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res., 47, 277 287.
48. Sestieri, C., Di Matteo, R., Ferretti, A., Del Gratta, C., Caulo, M., Tartaro, A., Olivetti Belardinelli, M. Romani, G.L. (2006) "What" versus "where" in the audiovisual domain: an fMRI study. Neuroimage, 33, 672 680.
49. Simos, P.G., Breier, J.I., Fletcher, J.M., Foorman, B.R., Castillo, E.M. Papanicolaou, A.C. (2002) Brain mechanisms for reading words and pseudowords: an integrated approach. Cereb. Cortex, 12, 297 305.
50. Stanford, T.R., Quessy, S. Stein, B.E. (2005) Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci., 25, 6499 6508.
51. Tagamets, M.A., Novick, J.M., Chalmers, M.L. Friedman, R.B. (2000) A parametric approach to orthographic processing in the brain: an fMRI study. J. Cogn. Neurosci., 12, 281 297.
52. Talairach, J. Tournoux, P. (1988) Co-Planar Stereotactic Atlas of the Human Brain. Thieme, Stuttgart.
53. Vellutino, F.R., Fletcher, J.M., Snowling, M.J. Scanlon, D.M. (2004) Specific reading disability (dyslexia): what have we learned in the past four decades? J. Child Psychol. Psychiatry, 45, 2 40.
54. Wilson, S.M., Saygin, A.P., Sereno, M.I. Iacoboni, M. (2004) Listening to speech activates motor areas involved in speech production. Nat. Neurosci., 7, 701 702.
55. Wright, T.M., Pelphrey, K.A., Allison, T., McKeown, M.J. McCarthy, G. (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex, 13, 1034 1043.