Cortical operational synchrony during audio-visual speech integration

Brain and Language

Volumn 85, Issue 2, 2003, Pages 297-312

  Fingelkurts, Andrew A ^a,b,e   Fingelkurts, Alexander A ^a,b   Krause, Christina M ^c   Möttönen, Riikka ^d   Sams, Mikko ^d  

^a MOSCOW STATE UNIVERSITY   (Russian Federation)

^b BM Science   (Finland)

^c UNIVERSITY OF HELSINKI   (Finland)

^d AALTO UNIVERSITY   (Finland)

^e NONE   (Finland)

Author keywords

Audio visual; Crossmodal; Large scale networks; MEG; Multisensory integration; Operations; Synchronization

Indexed keywords

ADULT; ARTICLE; BRAIN DEPTH STIMULATION; BRAIN REGION; BRAIN SYNAPTOSOME; CONTROLLED STUDY; CORTICAL SYNCHRONIZATION; DATA ANALYSIS; ELECTROENCEPHALOGRAM; FEMALE; HEARING; HUMAN; HUMAN EXPERIMENT; MAGNETOENCEPHALOGRAPHY; MALE; NORMAL HUMAN; OSCILLATORY POTENTIAL; SPEECH; VISION; VOLUNTEER;

EID: 0038301182     PISSN: 0093934X     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0093-934X(03)00059-2     Document Type: Article

References (79)

1. Bressler S.L., Kelso J.A.S. Cortical coordination dynamics and cognition. Trends in Cognitive Sciences. 5:2001;26-36.
2. Bushara K.O., Grafman J., Hallett M. Neural correlates of auditory-visual stimulus onset asynchrony detection. The Journal of Neuroscience. 21:2001;300-304.
3. Callan D.E., Callan A.M., Kroos C., Vatikiotis-Bateson E. Multimodal contribution to speech perception revealed by independent component analysis: A single-sweep EEG study. Brain Research. Cognitive Brain Research. 10:2001;349-353.
4. Calvert G.A. Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex. 11:2001;1110-1123.
5. Calvert G.A., Hansen P.C., Iversen S.D., Brammer M.J. Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage. 14:2001;427-438.
6. Dogil G., Ackermann H., Grodd W., Haider H., Kamp H., Mayer J., Riecker A., Wildgruber D. The speaking brain: A tutorial introduction to fMRI experiments in the production of speech, prosody and syntax. Journal of Neurolinguistics. 15:2002;59-90.
7. Driver J., Spence C. Attention and the crossmodal construction of space. Trends in Cognitive Sciences. 2:1998;254-262.
8. Engel A.K., Fries P., Singer W. Dynamic predictions: Oscillations and synchrony in top-down processing. Nature Reviews Neuroscience. 2:2001;704-716.
9. Erdi P. On the 'Dynamic Brain' metaphor. Brain and Mind. 1:2000;119-145.
10. Escera C., Alho K., Winkler I., Näätänen R. Neural mechanisms of involuntary attention to acoustic novelty and change. Journal of Cognitive Neuroscience. 10:1998;590-604.
11. Ettlinger G., Wilson W.A. Cross-modal performance: Behavioral processes, phylogenetic considerations, and neural mechanisms. Behavioral Brain research. 40:1990;169-192.
12. Fingelkurts, An. A. (1998). Time-spatial organization of human EEG segment's structure. Ph.D. Dissertation, Moscow State University, Moscow, p. 415 (in Russian).
13. Fingelkurts An.A., Fingelkurts Al.A. Operational architectonics of the human brain biopotential field: Towards solving the mind-brain problem. Brain and Mind. 2(3):2001;261-296.
14. Fodor J. The modularity of mind. 1983;MIT Press, Cambridge.
15. Frens M.A., Vanopstal A.J., Vanderwilligen R.F. Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Perception and Psychophysics. 57:1995;802-816.
16. Fries W. Cortical projections to the superior colliculus in the macaque monkey: A retrograde study using horseradish peroxidase. The Journal of Comparative Neurology. 230:1984;55-76.
17. Friston K.J., Price C.J. Dynamic representations and generative models of brain function. Brain Research Bulletin. 54:2001;275-285.
18. Giard M.H., Peronnet F. Audio-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive neuroscience. 11:1999;473-490.
19. Green K.P., Gerdeman A. Cross-modal discrepancies in coarticulation and the integration of speech information: The McGurk effect with mismatched vowels. Journal of Experimental Psychology. Human Perception & Performance. 21:1995;1409-1426.
20. Green K.P., Kuhl P.K. The role of visual information in the processing of place and manner features in speech perception. Perception and Psychophysics. 45:1989;34-42.
21. Hari R., Salmelin R. Human cortical oscillations: A neuromagnetic view through the skull. Trends in Neuroscience. 20:1997;44-49.
22. Ingber L. Statistical mechanics of multiple scales of neocortical interactions. Nunez P.L. Neocortical dynamics and human EEG rhythms. 1995;628-681 Oxford University Press, New York.
23. Jääskeläinen I., Alho K., Escera K., Winkler I., Sillanaukee P., Näätänen R. Effects of ethanol and auditory distraction on forced choice reaction time. Alcohol. 13:1996;153-156.
24. John E.R., Prichep L.S., Kox W., Valdés-Sosa P., Bosch-Bayard J., Aubert E.et al. Invariant reversible QEEG effects of anesthetics. Consciousness and Cognition. 10:2001;165-183.
25. Kaplan A.Ya. Nonstationary EEG: Methodological and experimental analysis. Uspehi Physiologicheskih Nayk (Success in Physiological Sciences). 29(3):1998;35-55. (in Russian).
26. Kaplan A.Ya., Shishkin S.L. Application of the change-point analysis to the investigation of the brain's electrical activity. Problems and methods. Brodsky B.E., Darkhovsky B.S. Non-parametric statistical diagnosis. 2000;333-388 Kluwer Academic Publishers, Dordrecht.
27. Kaplan A.Ya., Fingelkurts Al.A., Fingelkurts An.A., Darkhovsky B.S. Topological mapping of sharp reorganization synchrony in multichannel EEG. American Journal of Electroneurodiagnostic Technology. 37:1997;265-275.
28. Kaplan A.Ya., Fingelkurts An.A., Fingelkurts Al.A., Ivashko R.M. Temporal consistency of phasic changes in the EEG basic frequency components. Zhurnal Vysshei Nervnoi Deiatelnosty Im. I.P. Pavlova (Journal of Higher Nervous Activity). 48:1998;816-826. (in Russian).
29. Kim J. Downward causation in emergentism and nonreductive physicalism. Beckermann A., Flohr H., Kim J. Emergence of reduction? Essays on the prospects of nonreductive physicalism. 1992;119-138 de Gruyter, Berlin.
30. Klingberg T. Concurrent performance of two working memory tasks: Potential mechanisms of interference. Cerebral Cortex. 8:1998;593-601.
31. Krause, C. M., Möttönen, R., Jensen, O., Lampinen, J., & Sams, M. (2001). Brain oscillations during audio-visual speech perception - a MEG study. Paper was presented at the Human Brain Mapping Conference, June 10-14, England.
32. Lewis J.W., Beauchamp M.S., DeYoe E.A. A comparison of visual and auditory motion processing in human cerebral cortex. Cerebral Cortex. 10:2000;873-888.
33. Lewkowicz D.J. Perception of auditory-visual temporal synchrony in human infants. Journal of Experimental Psychology. Human Perception and Performance. 22:1996;1094-1106.
34. Loveless N., Levänen S., Jousmäki V., Sams M., Hari R. Temporal integration in auditory sensory memory: Neuromagnetic evidence. Evoked Potential. 100:1996;220-228.
35. Macaluso E., Frith C., Driver J. Modulation of human visual cortex by crossmodal spatial attention. Science. 253:2000;85-87.
36. MacLeod A., Summerfield Q. Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology. 21:1987;131-141.
37. Massaro D.W. Speech perception by ear and eye. Dodd B., Campbell R. Hearing by eye: The psychology of lip-reading. 1987;53-83 Erlbaum, Hillsdale, NJ.
38. Massaro D.W., Cohen M.M. Perceiving speech from inverted faces. Perception and Psychophysics. 58:1996;1047-1065.
39. McGurk H., MacDonald J.W. Hearing lips and seeing voices. Nature. 264:1976;746-748.
40. McIntosh A.R., Fitzpatrick S.M., Friston K.J. On the marriage of cognition and neuroscience. Neuroimage. 14:2001;1231-1237.
41. Mesulam M.-M. Large-scale neurocognitive networks and distributed processing for attention, language, and memory. Annual Neurology. 28:1990;597-613.
42. Mesulam M.-M. Neurocognitive networks and selectivity distributed processing. Review of Neurology (Paris). 150:1994;564-569.
43. Mesulam M.-M. From sensation to cognition. Brain. 121:1998;1013-1052.
44. Miller J.O. Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology. 14:1982;247-279.
45. Miller J.O. Time course of coactivation in bimodal divided attention. Perception and Psychophysics. 40:1986;331-343.
46. Möttönen R., Krause C.M., Tiipana K., Sams M. Processing of changes in visual speech in the human auditory cortex. Brain Research. Cognitive Brain Research. 13:2002;417-425.
47. Murray E.A., Malkova I., Goulet S. Milner A.D. Comparative neuropsychology. 1998;51-69 Oxford University Press, Oxford.
48. Näätänen R., Winkler I. The concept of auditory stimulus representation in cognitive neuroscience. Psychological Bulletin. 125:1999;826-859.
49. Neurobihavioral Systems (2001). Presentation. Available http://www.neurobehavioralsystems.com.
50. Nunez P.L. Neocortical dynamics and human EEG rhythms. 1995;Oxford University Press, New York.
51. Nunez P.L. Toward a quantitative description of large-scale neocortical dynamic function and EEG. Behavioral and Brain Sciences. 23:2000;371-437.
52. O'Hare J.J. Perceptual integration. Journal of the Washington Academy of Sciences. 81:1991;44-59.
53. Petrides M. Frontal lobes and behavior. Current Opinion in Neurobiology. 4:1994;207-211.
54. Pfurtscheller G., Neuper C., Schlogl A., Lugger K. Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. IEEE Transactions on Rehabilitation Engineering. 6:1998;316-325.
55. Phillips W.A., Singer W. In search of common foundations for cortical computation. Behavioral and Brain Sciences. 20:1997;657-683.
56. Rappelsberger P., Petsche H. Probability mapping: power and coherence analysis of cognitive processes. Brain Topography. 1:1988;46-54.
57. Razoumnikova O.M. Functional organization of different brain areas during convergent and divergent thinking: An EEG investigation. Cognitive Brain Research. 10:2000;11-18.
58. Reisberg D., McLean J., Goldfield A. Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. Dodd B., Campbell R. Hearing by eye: The psychology of lip reading. 1987;Erlbaum, Hillsdale, NJ.
59. Repp, B. H., Manuel, S. Y., Liberman, A. M., & Studdert-Kennedy, M. (1983). Exploring the "McGurk effect". Paper Presented at the 24th annual meeting of the Psychonomic Society in San Diego.
60. Rosenblum L.D., Saldana H.M. An audio-visual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology. Human Perception and Performance. 22:1996;318-331.
61. Rosenblum L.D., Yakel D.A., Green K.P. Face and mouth inversion affects on visual and audio-visual speech perception. Journal of Experimental Psychology. Human Perception and Performance. 26:2000;806-819.
62. Salinas E., Sejnowsli T.J. Correlated neuronal activity and the flow of neural information. Nature Reviews Neuroscience. 2:2001;539-550.
63. Sams M., Imada T. Integration of auditory and visual information in the human brain: Neuromagnetic evidence. Society for Neuroscience. Abstracts. 23:1997;1305.
64. Sams M., Aulanko R., Hämäläinen M., Hari R., Lounasmaa O.V., Lu S.T., Simola J. Seeing speech: Visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters. 127:1991;141-145.
65. Sekiyama K., Tohkura Y. McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America. 90:1991;1797-1805.
66. Singer W., Gray C. Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience. 18:1995;555-586.
67. Stein A.V., Rappelsberger P., Sarnthein J., Petsche H. Synchronization between temporal and parietal cortex during multimodal object processing in man. Cerebral Cortex. 9:1999;137-150.
68. Stein B.E., Meredith M.A. The merging of the senses. 1993;MIT Press, Cambridge, MA.
69. Stein B.E., Meredith M.A., Wallace M.T. Development and neural basis of multisensory integration. Lewkowicz D.J., Lickliter R. The development of intersensory perception: Comparative perspectives. 1994;81-105 Erlbaum, Hillsdale, NJ.
70. Stein B.E., Meredith M.A., Huneycutt W.S., McDade L. Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. Journal of Cognitive Neuroscience. 1:1989;12-24.
71. Treisman A. The binding problem. Current Opinion in Neurobiology. 6:1996;171-178.
72. Weiss S., Rappelsberger P. Long-range EEG synchronization during word encoding correlates with successful memory performance. Cognitive Brain Research. 9:2000;299-312.
73. Zeki S. Localization and globalisation in conscious vision. Annual Review of Neuroscience. 24:2001;57-86.
74. Haken H. What can synergetics contribute to the understanding of brain functioning? Uhl C. Analysis of neurophysiological brain functioning. 1999;7-40 Springer, Berlin.
75. Hämäläinen M.S., Ilmoniemi R. Interpreting magnetic fields of the brain: Minimum norm estimates. Medical and Biology Engineering and Computing. 32:1994;35-42.
76. John E.R. A Field theory of consciousness. Consciousness and Cognition. 10:2001;184-213.
77. Lewkowics D.J. The development of intersensory temporal perception: An epigenetic systems/limitations view. Psychological Bulletin. 126:2000;281-308.
78. Numminen J., Ahlfors S., Ilmoniemi R., Montonen J., Nenonen J. Transformation of multichannel megnetocardiographic signals to standard grid form. IEEE Transactions on Biomedical Engineering. 40:1995;72-77.
79. Seltzer B., Pandya D.N. Converging visual and somatic sensory cortical input to the intraparietal sulcus of the rhesus monkey. Brain Research. 192:1980;339-351.