A systematic review of multisensory cognitive-affective integration in schizophrenia

Neuroscience and Biobehavioral Reviews

Volumn 55, Issue , 2015, Pages 444-452

  Tseng, Huai Hsuan ^a,b,c   Bossong, Matthijs G ^a   Modinos, Gemma ^a   Chen, Kuan Ming ^d   McGuire, Philip ^a   Allen, Paul ^a  

^a KING'S COLLEGE LONDON   (United Kingdom)

^b CENTRE FOR ADDICTION AND MENTAL HEALTH   (Canada)

^c Ju Shan Hospital   (Taiwan)

^d NATIONAL TAIWAN NORMAL UNIVERSITY   (Taiwan)

Author keywords

Congruent facilitation; Emotion; Functional magnetic resonance imaging; Incongruent interference; Multi sensory integration; Schizophrenia

Indexed keywords

AFFECT; AUDITORY STIMULATION; BEHAVIOR; COGNITION; FRONTAL CORTEX; HUMAN; INFERIOR FRONTAL CORTEX; LINGUISTICS; MAJOR CLINICAL STUDY; MULTISENSORY INTEGRATION; PARIETAL CORTEX; PRIORITY JOURNAL; REVIEW; SCHIZOPHRENIA; SUPERIOR PARIETAL CORTEX; SUPERIOR TEMPORAL CORTEX; SYSTEMATIC REVIEW; TEMPORAL CORTEX; VISUAL STIMULATION; BRAIN; BRAIN MAPPING; HEARING; NUCLEAR MAGNETIC RESONANCE IMAGING; PATHOPHYSIOLOGY; PERCEPTION; PHYSIOLOGY; PSYCHOLOGY; VISION;

AFFECT; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; COGNITION; HUMANS; MAGNETIC RESONANCE IMAGING; PERCEPTION; SCHIZOPHRENIA; SCHIZOPHRENIC PSYCHOLOGY; VISUAL PERCEPTION;

EID: 84930942605     PISSN: 01497634     EISSN: 18737528     Source Type: Journal    
DOI: 10.1016/j.neubiorev.2015.04.019     Document Type: Review

References (82)

1. Addington J., Piskulic D., Perkins D., Woods S.W., Liu L., Penn D.L. Affect recognition in people at clinical high risk of psychosis. Schizophr. Res. 2012, 140:87-92.
2. Adolphs R. Neural systems for recognizing emotion. Curr. Opin. Neurobiol. 2002, 12:169-177.
3. Amminger G.P., Schafer M.R., Klier C.M., Schlogelhofer M., Mossaheb N., Thompson A., Bechdolf A., Allott K., McGorry P.D., Nelson B. Facial and vocal affect perception in people at ultra-high risk of psychosis, first-episode schizophrenia and healthy controls. Early Intervention Psychiatry 2012, 6:450-454.
4. Amminger G.P., Schafer M.R., Papageorgiou K., Klier C.M., Schlogelhofer M., Mossaheb N., Werneck-Rohrer S., Nelson B., McGorry P.D. Emotion recognition in individuals at clinical high-risk for schizophrenia. Schizophr. Bull. 2012, 38:1030-1039.
5. Bertelson P., Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 1981, 29:578-584.
6. Brittain P., Ffytche D.H., McKendrick A., Surguladze S. Visual processing, social cognition and functional outcome in schizophrenia. Psychiatry Res. 2010, 178:270-275.
7. Broome M.R., Matthiasson P., Fusar-Poli P., Woolley J.B., Johns L.C., Tabraham P., Bramon E., Valmaggia L., Williams S.C., Brammer M.J., Chitnis X., McGuire P.K. Neural correlates of executive function and working memory in the 'at-risk mental state'. Br. J. Psychiatry 2009, 194:25-33. the journal of mental science.
8. Calvert G.A., Hansen P.C., Iversen S.D., Brammer M.J. Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 2001, 14:427-438.
9. Carretie L. Exogenous (automatic) attention to emotional stimuli: a review. Cogn. Affect. Behav. Neurosci. 2014, 14:1228-1258.
10. Carretie L., Hinojosa J.A., Martin-Loeches M., Mercado F., Tapia M. Automatic attention to emotional stimuli: neural correlates. Hum. Brain Mapp. 2004, 22:290-299.
11. Choe C.S., Welch R.B., Gilford R.M., Juola J.F. Ventriloquist effect-visual dominance or response bias. Percept. Psychophys. 1975, 18:55-60.
12. Collignon O., Girard S., Gosselin F., Roy S., Saint-Amour D., Lassonde M., Lepore F. Audio-visual integration of emotion expression. Brain Res. 2008, 1242:126-135.
13. de Boer-Schellekens L., Stekelenburg J.J., Maes J.P., Van Gool A.R., Vroomen J. Sound improves diminished visual temporal sensitivity in schizophrenia. Acta Psychol. 2014, 147:136-142.
14. de Gelder B., Vroomen J. Bimodal emotion perception: integration across separate modalities, cross-modal perceptual grouping or perception of multimodal events?. Cogn. Emotion 2000, 14:321-324.
15. de Gelder B., Vroomen J., Annen L., Masthof E., Hodiamont P. Audio-visual integration in schizophrenia. Schizophr. Res. 2003, 59:211-218.
16. de Gelder B., Vroomen J., de Jong S.J., Masthoff E.D., Trompenaars F.J., Hodiamont P. Multisensory integration of emotional faces and voices in schizophrenics. Schizophr. Res. 2005, 72:195-203.
17. de Jong J.J., Hodiamont P.P., de Gelder B. Modality-specific attention and multisensory integration of emotions in schizophrenia: reduced regulatory effects. Schizophr. Res. 2010, 122:136-143.
18. de Jong J.J., Hodiamont P.P., Van den Stock J., de Gelder B. Audiovisual emotion recognition in schizophrenia: reduced integration of facial and vocal affect. Schizophr. Res. 2009, 107:286-293.
19. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 2008, 57:11-23.
20. Edwards J., Jackson H.J., Pattison P.E. Emotion recognition via facial expression and affective prosody in schizophrenia: a methodological review. Clin. Psychol. Rev. 2002, 22:789-832.
21. Egan M.F., Goldberg T.E., Gscheidle T., Weirich M., Bigelow L.B., Weinberger D.R. Relative risk of attention deficits in siblings of patients with schizophrenia. Am. J. Psychiatry 2000, 157:1309-1316.
22. Ethofer T., Pourtois G., Wildgruber D. Investigating audiovisual integration of emotional signals in the human brain. Prog. Brain Res. 2006, 156:345-361.
23. Ferri F., Costantini M., Salone A., Di Iorio G., Martinotti G., Chiarelli A., Merla A., Di Giannantonio M., Gallese V. Upcoming tactile events and body ownership in schizophrenia. Schizophr Res. 152 2014, 51-57.
24. Foucher J.R., Lacambre M., Pham B.T., Giersch A., Elliott M.A. Low time resolution in schizophrenia lengthened windows of simultaneity for visual, auditory and bimodal stimuli. Schizophr. Res. 2007, 97:118-127.
25. Fusar-Poli P., Broome M.R., Matthiasson P., Woolley J.B., Mechelli A., Johns L.C., Tabraham P., Bramon E., Valmaggia L., Williams S.C., McGuire P. Prefrontal function at presentation directly related to clinical outcome in people at ultrahigh risk of psychosis. Schizophr. Bull. 2011, 37:189-198.
26. Fusar-Poli P., Perez J., Broome M., Borgwardt S., Placentino A., Caverzasi E., Cortesi M., Veggiotti P., Politi P., Barale F., McGuire P. Neurofunctional correlates of vulnerability to psychosis: a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 2007, 31:465-484.
27. Gur R.E., McGrath C., Chan R.M., Schroeder L., Turner T., Turetsky B.I., Kohler C., Alsop D., Maldjian J., Ragland J.D., Gur R.C. An fMRI study of facial emotion processing in patients with schizophrenia. Am. J. Psychiatry 2002, 159:1992-1999.
28. Hempel A., Hempel E., Schonknecht P., Stippich C., Schroder J. Impairment in basal limbic function in schizophrenia during affect recognition. Psychiatry Res. 2003, 122:115-124.
29. Hickok G., Poeppel D. The cortical organization of speech processing. Nat. Rev. Neurosci. 2007, 8:393-402.
30. Hu Y., Chen X., Gu H., Yang Y. Resting-state glutamate and GABA concentrations predict task-induced deactivation in the default mode network. J. Neurosci. 2013, 33:18566-18573. the official journal of the Society for Neuroscience.
31. Jacklin D.L., Goel A., Clementino K.J., Hall A.W., Talpos J.C., Winters B.D. Severe cross-modal object recognition deficits in rats treated sub-chronically with NMDA receptor antagonists are reversed by systemic nicotine: implications for abnormal multisensory integration in schizophrenia. Neuropsychopharmacology 2012, 37:2322-2331.
32. Jardri R., Pins D., Bubrovszky M., Lucas B., Lethuc V., Delmaire C., Vantyghem V., Despretz P., Thomas P. Neural functional organization of hallucinations in schizophrenia: multisensory dissolution of pathological emergence in consciousness. Conscious. Cogn. 2009, 18:449-457.
33. Koelewijn T., Bronkhorst A., Theeuwes J. Attention and the multiple stages of multisensory integration: a review of audiovisual studies. Acta Psychol. 2010, 134:372-384. (Amst).
34. Kohler C.G., Turner T.H., Bilker W.B., Brensinger C.M., Siegel S.J., Kanes S.J., Gur R.E., Gur R.C. Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am. J. Psychiatry 2003, 160:1768-1774.
35. Laroi F., Fonteneau B., Mourad H., Raballo A. Basic emotion recognition and psychopathology in schizophrenia. J. Nerv. Ment. Dis. 2010, 198:79-81.
36. Leentjens A.F., Wielaert S.M., van Harskamp F., Wilmink F.W. Disturbances of affective prosody in patients with schizophrenia; a cross sectional study. J. Neurol. Neurosurg. Psychiatry 1998, 64:375-378.
37. Leitman D.I., Foxe J.J., Butler P.D., Saperstein A., Revheim N., Javitt D.C. Sensory contributions to impaired prosodic processing in schizophrenia. Biol. Psychiatry 2005, 58:56-61.
38. Liberati A., Altman D.G., Tetzlaff J., Mulrow C., Gotzsche P.C., Ioannidis J.P., Clarke M., Devereaux P.J., Kleijnen J., Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009, 339:b2700.
39. Marsman A., van den Heuvel M.P., Klomp D.W., Kahn R.S., Luijten P.R., Hulshoff Pol H.E. Glutamate in schizophrenia: a focused review and meta-analysis of (1)H-MRS studies. Schizophr. Bull. 2013, 39:120-129.
40. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
41. Mechelli A., Riecher-Rossler A., Meisenzahl E.M., Tognin S., Wood S.J., Borgwardt S.J., Koutsouleris N., Yung A.R., Stone J.M., Phillips L.J., McGorry P.D., Valli I., Velakoulis D., Woolley J., Pantelis C., McGuire P. Neuroanatomical abnormalities that predate the onset of psychosis: a multicenter study. Arch. Gen. Psychiatry 2011, 68:489-495.
42. Merritt K., McGuire P., Egerton A. Relationship between glutamate dysfunction and symptoms and cognitive function in psychosis. Front. Psychiatry 2013, 4:151.
43. Mesholam-Gately R.I., Giuliano A.J., Goff K.P., Faraone S.V., Seidman L.J. Neurocognition in first-episode schizophrenia: a meta-analytic review. Neuropsychology 2009, 23:315-336.
44. Miller J. Divided attention: evidence for coactivation with redundant signals. Cognit. Psychol. 1982, 14:247-279.
45. Miller J. Timecourse of coactivation in bimodal divided attention. Atten. Percept. Psychophys. 1986, 40:331-343.
46. Modinos G., Ormel J., Aleman A. Altered activation and functional connectivity of neural systems supporting cognitive control of emotion in psychosis proneness. Schizophr. Res. 2010, 118:88-97.
47. Moher D., Liberati A., Tetzlaff J., Altman D.G., Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009, 339:b2535.
48. Muller V.I., Cieslik E.C., Kellermann T.S., Eickhoff S.B. Crossmodal emotional integration in major depression. Soc. Cogn. Affect. Neurosci. 2014, 147:136-142.
49. Muller V.I., Cieslik E.C., Laird A.R., Fox P.T., Eickhoff S.B. Dysregulated left inferior parietal activity in schizophrenia and depression: functional connectivity and characterization. Front. Hum. Neurosci. 2013, 7:268.
50. Muller V.I., Habel U., Derntl B., Schneider F., Zilles K., Turetsky B.I., Eickhoff S.B. Incongruence effects in crossmodal emotional integration. Neuroimage 2011, 54:2257-2266.
51. Muller V.I., Kellermann T.S., Seligman S.C., Turetsky B.I., Eickhoff S.B. Modulation of affective face processing deficits in schizophrenia by congruent emotional sounds. Soc. Cogn. Affect. Neurosci. 2014, 9:436-444.
52. Pauly K., Seiferth N.Y., Kellermann T., Ruhrmann S., Daumann B., Backes V., Klosterkotter J., Shah N.J., Schneider F., Kircher T.T., Habel U. The interaction of working memory and emotion in persons clinically at risk for psychosis: an fMRI pilot study. Schizophr. Res. 2010, 120:167-176.
53. Pearl D., Yodashkin-Porat D., Katz N., Valevski A., Aizenberg D., Sigler M., Weizman A., Kikinzon L. Differences in audiovisual integration, as measured by McGurk phenomenon, among adult and adolescent patients with schizophrenia and age-matched healthy control groups. Compr. Psychiatry 2009, 50:186-192.
54. Penn D.L., Corrigan P.W., Bentall R.P., Racenstein J.M., Newman L. Social cognition in schizophrenia. Psychol. Bul. 1997, 121:114-132.
55. Pinkham A., Penn D., Wangelin B., Perkins D., Gerig G., Gu H., Lieberman J. Facial emotion perception and fusiform gyrus volume in first episode schizophrenia. Schizophr. Res. 2005, 79:341-343.
56. Pinkham A.E., Penn D.L., Perkins D.O., Lieberman J. Implications for the neural basis of social cognition for the study of schizophrenia. Am. J. Psychiatry 2003, 160:815-824.
57. Ross L.A., Saint-Amour D., Leavitt V.M., Molholm S., Javitt D.C., Foxe J.J. Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions. Schizophr. Res. 2007, 97:173-183.
58. Sass K., Heim S., Sachs O., Straube B., Schneider F., Habel U., Kircher T. Neural correlates of semantic associations in patients with schizophrenia. Eur. Arch. Psychiatry Clin. Neurosci. 2014, 264:143-154.
59. Schröger E., Widmann A. Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 1998, 35:755-759.
60. Seiferth N.Y., Pauly K., Habel U., Kellermann T., Shah N.J., Ruhrmann S., Klosterkotter J., Schneider F., Kircher T. Increased neural response related to neutral faces in individuals at risk for psychosis. Neuroimage 2008, 40:289-297.
61. Seubert J., Loughead J., Kellermann T., Boers F., Brensinger C.M., Habel U. Multisensory integration of emotionally valenced olfactory-visual information in patients with schizophrenia and healthy controls. J. Psychiatry Neurosci. 2010, 35:185-194.
62. Shenton M.E., Dickey C.C., Frumin M., McCarley R.W. A review of MRI findings in schizophrenia. Schizophr. Res. 2001, 49:1-52.
63. Smieskova R., Fusar-Poli P., Allen P., Bendfeldt K., Stieglitz R.D., Drewe J., Radue E.W., McGuire P.K., Riecher-Rossler A., Borgwardt S.J. Neuroimaging predictors of transition to psychosis-a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 2010, 34:1207-1222.
64. Smiley J.F., Konnova K., Bleiwas C. Cortical thickness, neuron density and size in the inferior parietal lobe in schizophrenia. Schizophr. Res. 2012, 136:43-50.
65. Smiley J.F., Rosoklija G., Mancevski B., Mann J.J., Dwork A.J., Javitt D.C. Altered volume and hemispheric asymmetry of the superficial cortical layers in the schizophrenia planum temporale. Eur. J. Neurosci. 2009, 30:449-463.
66. Stekelenburg J.J., Maes J.P., Van Gool A.R., Sitskoorn M., Vroomen J. Deficient multisensory integration in schizophrenia: an event-related potential study. Schizophr. Res. 2013, 147:253-261.
67. Stephen J.M., Coffman B.A., Jung R.E., Bustillo J.R., Aine C.J., Calhoun V.D. Using joint ICA to link function and structure using MEG and DTI in schizophrenia. Neuroimage 2013, 83:418-430.
68. Stone D.B., Urrea L.J., Aine C.J., Bustillo J.R., Clark V.P., Stephen J.M. Unisensory processing and multisensory integration in schizophrenia: a high-density electrical mapping study. Neuropsychologia 2011, 49:3178-3187.
69. Straube B., Green A., Sass K., Kircher T. Superior temporal sulcus disconnectivity during processing of metaphoric gestures in schizophrenia. Schizophr. Bull. 2013.
70. Surguladze S.A., Calvert G.A., Brammer M.J., Campbell R., Bullmore E.T., Giampietro V., David A.S. Audio-visual speech perception in schizophrenia: an fMRI study. Psychiatry Res. 2001, 106:1-14.
71. Szycik G.R., Munte T.F., Dillo W., Mohammadi B., Samii A., Emrich H.M., Dietrich D.E. Audiovisual integration of speech is disturbed in schizophrenia: an fMRI study. Schizophr. Res. 2009, 110:111-118.
72. Szycik G.R., Munte T.F., Dillo W., Mohammadi B., Samii A., Emrich H.M., Dietrich D.E. Audiovisual integration of speech is disturbed in schizophrenia: an fMRI study. Schizophr. Res. 2009, 110:111-118.
73. Szycik G.R., Ye Z., Mohammadi B., Dillo W., Te Wildt B.T., Samii A., Frieling H., Bleich S., Munte T.F. Maladaptive connectivity of Broca's area in schizophrenia during audiovisual speech perception: an fMRI study. Neuroscience 2013, 253:274-282.
74. Thompson A., Papas A., Bartholomeusz C., Allott K., Amminger G.P., Nelson B., Wood S., Yung A. Social cognition in clinical "at risk" for psychosis and first episode psychosis populations. Schizophr. Res. 2012, 141:204-209.
75. Tognin S., Riecher-Rossler A., Meisenzahl E.M., Wood S.J., Hutton C., Borgwardt S.J., Koutsouleris N., Yung A.R., Allen P., Phillips L.J., McGorry P.D., Valli I., Velakoulis D., Nelson B., Woolley J., Pantelis C., McGuire P., Mechelli A. Reduced parahippocampal cortical thickness in subjects at ultra-high risk for psychosis. Psychol. Med. 2014, 44:489-498.
76. Tseng H.H., Chen S.H., Liu C.M., Howes O., Huang Y.L., Hsieh M.H., Liu C.C., Shan J.C., Lin Y.T., Hwu H.G. Facial and prosodic emotion recognition deficits associate with specific clusters of psychotic symptoms in schizophrenia. PLoS ONE 2013, 8:e66571.
77. Van den Stock J., de Jong S.J., Hodiamont P.P., de Gelder B. Perceiving emotions from bodily expressions and multisensory integration of emotion cues in schizophrenia. Soc. Neurosci. 2011, 6:537-547.
78. Vuilleumier P., Huang Y.M. Emotional attention: uncovering the mechanisms of affective biases in perception. Curr. Dir. Psychol. 2009, 18:148-152.
79. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychol. Bul. 1980, 88:638-667.
80. Wildgruber D., Ethofer T., Grandjean D., Kreifelts B. A cerebral network model of speech prosody comprehension. Int. J. Speech Lang. Pathol. 2009, 11:277-281.
81. Williams L.E., Light G.A., Braff D.L., Ramachandran V.S. Reduced multisensory integration in patients with schizophrenia on a target detection task. Neuropsychologia 2010, 48:3128-3136.
82. Williams L.M., Das P., Harris A.W., Liddell B.B., Brammer M.J., Olivieri G., Skerrett D., Phillips M.L., David A.S., Peduto A., Gordon E. Dysregulation of arousal and amygdala-prefrontal systems in paranoid schizophrenia. Am. J. Psychiatry 2004, 161:480-489.