Audiovisual emotion recognition in schizophrenia: Reduced integration of facial and vocal affect

Schizophrenia Research

Volumn 107, Issue 2-3, 2009, Pages 286-293

  de Jong, J J ^a,b   Hodiamont, P P G ^a,b   Van den Stock, J ^a,c   de Gelder, B ^a,d  

^a TILBURG UNIVERSITY   (Netherlands)

^b GGZ Midden Brabant   (Netherlands)

^c UNIVERSITY HOSPITALS LEUVEN   (Belgium)

^d MASSACHUSETTS GENERAL HOSPITAL   (United States)

Author keywords

Emotion recognition; Multisensory integration; Schizophrenia

Indexed keywords

ADULT; AFFECT; ARTICLE; AUDITORY STIMULATION; CONTROLLED STUDY; EMOTION; FACE; FEAR; FEMALE; HAPPINESS; HUMAN; MAJOR CLINICAL STUDY; MALE; MENTAL DISEASE; POSITIVE AND NEGATIVE SYNDROME SCALE; PRIORITY JOURNAL; SCHIZOPHRENIA; SENSORY SYSTEM ELECTROPHYSIOLOGY; TASK PERFORMANCE; VISUAL STIMULATION; VOICE;

ADULT; COGNITION DISORDERS; DISCRIMINATION (PSYCHOLOGY); EMOTIONS; FACIAL EXPRESSION; FEMALE; HUMANS; MALE; MIDDLE AGED; PATTERN RECOGNITION, VISUAL; PSYCHOTIC DISORDERS; SCHIZOPHRENIA; SCHIZOPHRENIC PSYCHOLOGY; SPEECH PERCEPTION; YOUNG ADULT;

EID: 58249091409     PISSN: 09209964     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.schres.2008.10.001     Document Type: Article

References (55)

1. APA. Diagnostic and Statistical Manual of mental disorders, DSM-IV-TR (2000), American Psychiatric Association, Washington, DC
2. Bertelson P., and de Gelder B. The psychology of multimodal perception. In: Spence C., and Driver J. (Eds). Crossmodal space and crossmodal attention (2004), Oxford University Press, Oxford 151-177
3. Braff D.L., and Light G.A. Preattentional and attentional cognitive deficits as targets for treating schizophrenia. Psychopharmacology (Berl) 174 (2004) 75-85
4. Braff D.L., Geyer M.A., Light G.A., Sprock J., Perry W., Cadenhead K.S., and Swerdlow N.R. Impact of prepulse characteristics on the detection of sensorimotor gating deficits in schizophrenia. Schizophr. Res. 49 (2001) 171-178
5. Brancazio L., and Miller J.L. Use of visual information in speech perception: evidence for a visual rate effect both with and without a McGurk effect. Percept. Psychophys. 67 (2005) 759-769
6. Brunet-Gouet E., and Decety J. Social brain dysfunctions in schizophrenia: a review of neuroimaging studies. Psychiatry Res. 148 (2006) 75-92
7. Calvert G.A. Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 11 (2001) 1110-1123
8. Calvert G.A. Handbook of multisensory processes (2004), MIT, Cambridge, MA
9. Calvert G.A., Campbell R., and Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10 (2000) 649-657
10. Couture S.M., Penn D.L., and Roberts D.L. The functional significance of social cognition in schizophrenia: a review. Schizophr. Bull. 32 Suppl. 1 (2006) S44-S63
11. de Gelder B. Neuroscience. More to seeing than meets the eye. Science 289 (2000) 1148-1149
12. de Gelder B., and Bertelson P. Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7 (2003) 460-467
13. de Gelder B., Bocker K.B., Tuomainen J., Hensen M., and Vroomen J. The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses. Neurosci. Lett. 260 (1999) 133-136
14. de Gelder B., Pourtois G., and Weiskrantz L. Fear recognition in the voice is modulated by unconsciously recognized facial expressions but not by unconsciously recognized affective pictures. Proc. Natl. Acad. Sci. U. S. A. 99 (2002) 4121-4126 %19
15. de Gelder B., Vroomen J., Annen L., Masthof E., and Hodiamont P. Audio-visual integration in schizophrenia. Schizophr. Res. 59 (2003) 211-218
16. de Gelder B., Morris J.S., and Dolan R.J. Unconscious fear influences emotional awareness of faces and voices. Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 18682-18687 %20
17. de Gelder B., Vroomen J., de Jong S.J., Masthoff E.D., Trompenaars F.J., and Hodiamont P. Multisensory integration of emotional faces and voices in schizophrenics. Schizophr. Res. 72 (2005) 195-203
18. Dolan R.J., Morris J.S., and de Gelder B. Crossmodal binding of fear in voice and face. Proc. Natl. Acad. Sci. U. S. A. 98 (2001) 10006-10010
19. Ekman P., and Friessen W.V. Measuring facial movement. J. Environ. Psychol. Non-Verbal Behav. 1 (1976) 56-75
20. Foxe J.J., and Schroeder C.E. The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16 (2005) 419-423
21. Giard M.H., and Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11 (1999) 473-490
22. Green M.F., Kern R.S., Braff D.L., and Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the "right stuff"?. Schizophr. Bull. 26 (2000) 119-136
23. Green M.F., Nuechterlein K.H., Gold J.M., Barch D.M., Cohen J., Essock S., Fenton W.S., Frese F., Goldberg T.E., Heaton R.K., Keefe R.S., Kern R.S., Kraemer H., Stover E., Weinberger D.R., Zalcman S., and Marder S.R. Approaching a consensus cognitive battery for clinical trials in schizophrenia: the NIMH-MATRICS conference to select cognitive domains and test criteria. Biol. Psychiatry 56 (2004) 301-307
24. Green M.F., Penn D.L., Bentall R., Carpenter W.T., Gaebel W., Gur R.C., Kring A.M., Park S., Silverstein S.M., and Heinssen R. Social cognition in schizophrenia: an NIMH workshop on definitions, assessment, and research opportunities. Schizophr. Bull. (2008)
25. Grillon C., Ameli R., Charney D.S., Krystal J., and Braff D. Startle gating deficits occur across prepulse intensities in schizophrenic patients. Biol. Psychiatry. 32 (1992) 939-943
26. Hershenson M. Reaction time as a measure of intersensory facilitation. J. Exp. Psychol. 63 (1962) 289-293
27. Jones J.A., and Callan D.E. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Neuroreport 14 (2003) 1129-1133
28. Kay S.R., Fiszbein A., and Opler L.A. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13 (1987) 261-276
29. Kreifelts B., Ethofer T., Grodd W., Erb M., and Wildgruber D. Audiovisual integration of emotional signals in voice and face: an event-related fMRI study. Neuroimage 37 (2007) 1445-1456
30. Lezak M.D., Howieson D.B., and Loring D.W. Orientation and attention. In: Lezak M.D., Howieson D.B., and Loring D.W. (Eds). Neuropsychological Assessment (2004), Oxford University Press, New York 337-374
31. Light G.A., and Braff D.L. Mismatch negativity deficits are associated with poor functioning in schizophrenia patients. Arch. Gen. Psychiatry 62 (2005) 127-136
32. Macaluso E., George N., Dolan R., Spence C., and Driver J. Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 21 (2004) 725-732
33. MacLeod C.M., and MacDonald P.A. Interdimensional interference in the Stroop effect: uncovering the cognitive and neural anatomy of attention. Trends Cogn. Sci. 4 (2000) 383-391
34. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976) 746-748
35. Meeren H.K., van Heijnsbergen C.C., and de Gelder B. Rapid perceptual integration of facial expression and emotional body language. Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 16518-16523
36. Meredith M.A., and Stein B.E. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 56 (1986) 640-662
37. Nose M., Tansella M., Thornicroft G., Schene A., Becker T., Veronese A., Leese M., Koeter M., Angermeyer M., and Barbui C. Is the defined daily dose system a reliable tool for standardizing antipsychotic dosages?. Int. Clin. Psychopharmacol. 23 (2008) 287-290
38. Pichot P., Lebeaux M.O., Penhouët C., and Simon M. Comparaison entre la méthode de Barona et l'Index de détérioration de Wechsler. Revue Européenne de Psychologie Appliquée 43 (1993) 291-301
39. Pourtois G., de Gelder B., Vroomen J., Rossion B., and Crommelinck M. The time-course of intermodal binding between seeing and hearing affective information. Neuroreport 11 (2000) 1329-1333
40. Pourtois G., de Gelder B., Bol A., and Crommelinck M. Perception of facial expressions and voices and of their combination in the human brain. Cortex 41 (2005) 49-59
41. Radeau M. Auditory-visual spatial interaction and modularity. Curr. Psychol. Cogn. 13 (1994) 117-123
42. Ross L.A., Saint-Amour D., Leavitt V.M., Molholm S., Javitt D.C., and Foxe J.J. Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions. Schizophr. Res. 97 (2007) 173-183
43. Schall U., Schon A., Zerbin D., Eggers C., and Oades R.D. Event-related potentials during an auditory discrimination with prepulse inhibition in patients with schizophrenia, obsessive-compulsive disorder and healthy subjects. Int. J. Neurosci. 84 (1996) 15-33
44. Schroeder C.E., and Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res. 14 (2002) 187-198
45. Schroeder C.E., Lindsley R.W., Specht C., Marcovici A., Smiley J.F., and Javitt D.C. Somatosensory input to auditory association cortex in the macaque monkey. J. Neurophysiol. 85 (2001) 1322-1327
46. Stein B.E., Huneycutt W.S., and Meredith M.A. Neurons and behavior: the same rules of multisensory integration apply. Brain Res. 448 (1988) 355-358
47. Surguladze S.A., Calvert G.A., Brammer M.J., Campbell R., Bullmore E.T., Giampietro V., and David A.S. Audio-visual speech perception in schizophrenia: an fMRI study. Psychiatry Res. 106 (2001) 1-14
48. Swerdlow N.R., Light G.A., Cadenhead K.S., Sprock J., Hsieh M.H., and Braff D.L. Startle gating deficits in a large cohort of patients with schizophrenia: relationship to medications, symptoms, neurocognition, and level of function. Arch. Gen. Psychiatry 63 (2006) 1325-1335
49. Van den Stock J., Righart R., and de Gelder B. Body expressions influence recognition of emotions in the face and voice. Emotion 7 (2007) 487-494
50. Velligan D.I., Mahurin R.K., Diamond P.L., Hazleton B.C., Eckert S.L., and Miller A.L. The functional significance of symptomatology and cognitive function in schizophrenia. Schizophr. Res. 25 (1997) 21-31
51. Velligan D.I., Bow-Thomas C.C., Mahurin R.K., Miller A.L., and Halgunseth L.C. Do specific neurocognitive deficits predict specific domains of community function in schizophrenia?. J. Nerv. Ment. Dis. 188 (2000) 518-524
52. Voort L.V., Senf G.M., and Benton A.L. Development of audiovisual integration in normal and retarded readers. Child Dev. 43 (1972) 1260-1272
53. Vroomen J., Driver J., and de Gelder B. Is cross-modal integration of emotional expressions independent of attentional resources?. Cogn. Affect. Behav. Neurosci. 1 (2001) 382-387
54. WHO, 2008. http://www.searo.who.int/en/section1174/section1199/section1567_6744.htm. internet. Ref Type: Electronic Citation.
55. WHO Collaborating Center for Drug Statistic Methodology. Guidelines for ATC Classification and DDD Assignment (2003) Oslo