Top-down and bottom-up modulation in processing bimodal face/voice stimuli

BMC Neuroscience

Volumn 11, Issue , 2010, Pages

  Latinus, Marianne ^a,b   VanRullen, Rufin ^a   Taylor, Margot J ^c  

^a Brain and Cognition Research Center in Toulouse   (France)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

^c UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; ATTENTION; AUDITORY STIMULATION; BEHAVIOR; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; FACE; FEMALE; FRONTAL CORTEX; GENDER; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PERCEPTION; SENSORY CORTEX; VISUAL STIMULATION; VOICE; BRAIN; DECISION MAKING; ELECTROENCEPHALOGRAPHY; EVOKED RESPONSE; FRONTAL LOBE; HEARING; NEUROPSYCHOLOGICAL TEST; PHOTOSTIMULATION; PHYSIOLOGY; REACTION TIME; SEXUAL DEVELOPMENT; TIME; VISION;

ACOUSTIC STIMULATION; ADULT; ATTENTION; AUDITORY PERCEPTION; BRAIN; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; FACE; FEMALE; FRONTAL LOBE; HUMANS; JUDGMENT; MALE; NEUROPSYCHOLOGICAL TESTS; PHOTIC STIMULATION; REACTION TIME; SEX CHARACTERISTICS; TIME FACTORS; VISUAL PERCEPTION; VOICE; YOUNG ADULT;

EID: 77949474503     PISSN: None     EISSN: 14712202     Source Type: Journal    
DOI: 10.1186/1471-2202-11-36     Document Type: Article

References (76)

1. Giard MH, Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. Journal of Cognitive Neuroscience 1999, 11(5):473-490. 10.1162/089892999563544, 10511637.
2. Giard MH, Fort A, Mouchetant-Rostaing Y, Pernier J. Neurophysiological mechanisms of auditory selective attention in humans. Front Biosci 2000, 5:D84-94. 10.2741/Giard, 10702372.
3. Vroomen J, Driver J, de Gelder B. Is cross-modal integration of emotional expressions independent of attentional resources?. Cogn Affect Behav Neuroscience 2001, 1(4):382-387.
4. Bertelson P, Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept Psychophys 1981, 29(6):578-584.
5. Driver J. Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature 1996, 381(6577):66-68. 10.1038/381066a0, 8609989.
6. Spence C, Driver J. Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism. Neuroreport 2000, 11(9):2057-2061. 10.1097/00001756-200006260-00049, 10884070.
7. Wada Y, Kitagawa N, Noguchi K. Audio-visual integration in temporal perception. Int J Psychophysiol 2003, 50(1-2):117-124. 10.1016/S0167-8760(03)00128-4, 14511840.
8. Shimojo S, Shams L. Sensory modalities are not separate modalities: plasticity and interactions. Curr Opin Neurobiol 2001, 11(4):505-509. 10.1016/S0959-4388(00)00241-5, 11502399.
9. Bertelson P, Aschersleben G. Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. Int J Psychophysiol 2003, 50(1-2):147-155. 10.1016/S0167-8760(03)00130-2, 14511842.
10. Aschersleben G, Bertelson P. Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization. Int J Psychophysiol 2003, 50(1-2):157-163. 10.1016/S0167-8760(03)00131-4, 14511843.
11. de Gelder B, Pourtois G, Weiskrantz L. Fear recognition in the voice is modulated by unconsciously recognized facial expressions but not by unconsciously recognized affective pictures. Proc Natl Acad Sci USA 2002, 99(6):4121-4126. 10.1073/pnas.062018499, 122658, 11904455.
12. Fort A, Giard MH. Multi electrophysiological mechanisms of audio-visual integration in human perception. The Handbook of Multisensory Processes 2004, 503-514. Cambridge: MIT Press, Calvert GA, Spence C, Stein BE.
13. Calvert GA, Brammer MJ, Bullmore ET, Campbell R, Iversen SD, David AS. Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport 1999, 10(12):2619-2623. 10.1097/00001756-199908200-00033, 10574380.
14. Eimer M. Crossmodal links in spatial attention between vision, audition, and touch: evidence from event-related brain potentials. Neuropsychologia 2001, 39(12):1292-1303. 10.1016/S0028-3932(01)00118-X, 11566312.
15. Calvert GA, Campbell R, Brammer MJ. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 2000, 10(11):649-657. 10.1016/S0960-9822(00)00513-3, 10837246.
16. Fort A, Delpuech C, Pernier J, Giard MH. Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Research Cogn Brain Research 2002, 14(1):20-30.
17. Teder-Salejarvi WA, McDonald JJ, Di Russo F, Hillyard SA. An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Research Cogn Brain Research 2002, 14(1):106-114.
18. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ. Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Research Cogn Brain Research 2002, 14(1):115-128.
19. Molholm S, Ritter W, Javitt DC, Foxe JJ. Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 2004, 14(4):452-465. 10.1093/cercor/bhh007, 15028649.
20. McGurk H, MacDonald J. Hearing lips and seeing voices. Nature 1976, 264(5588):746-748. 10.1038/264746a0, 1012311.
21. Besle J, Fort A, Delpuech C, Giard MH. Bimodal speech: early suppressive visual effects in human auditory cortex. Eur J Neuroscience 2004, 20(8):2225-2234.
22. Izumi A, Kojima S. Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes). Anim Cogn 2004, 7(3):179-184. 10.1007/s10071-004-0212-4, 15015035.
23. Dolan RJ, Morris JS, de Gelder B. Crossmodal binding of fear in voice and face. Proc Natl Acad Sci USA 2001, 98(17):10006-10010. 10.1073/pnas.171288598, 55568, 11493699.
24. Pourtois G, de Gelder B, Bol A, Crommelinck M. Perception of facial expressions and voices and of their combination in the human brain. Cortex 2005, 41(1):49-59. 10.1016/S0010-9452(08)70177-1, 15633706.
25. Pourtois G, de Gelder B, Vroomen J, Rossion B, Crommelinck M. The time-course of intermodal binding between seeing and hearing affective information. Neuroreport 2000, 11(6):1329-1333. 10.1097/00001756-200004270-00036, 10817616.
26. Parr LA. Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition. Anim Cogn 2004, 7(3):171-178. 10.1007/s10071-004-0207-1, 14997361.
27. De Gelder B, Vroomen J. The perception of emotions by ear and by eye. Cognition and Emotion 2000, 14(3):289-311.
28. Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends Cogn Sci 2000, 4(6):223-233. 10.1016/S1364-6613(00)01482-0, 10827445.
29. Bentin S, Allison T, Puce A, Perez E, McCarthy G. Electrophysiological Studies of Face Perception in Humans. Journal of Cognitive Neuroscience 1996, 8:551-565.
30. George N, Evans J, Fiori N, Davidoff J, Renault B. Brain events related to normal and moderately scrambled faces. Cognitive Brain Research 1996, 4:65-76. 10.1016/0926-6410(95)00045-3, 8883920.
31. Itier RJ, Latinus M, Taylor MJ. Face, eye and object early processing: what is the face specificity?. Neuroimage 2006, 29(2):667-676. 10.1016/j.neuroimage.2005.07.041, 16169749.
32. Rossion B, Gauthier I, Tarr MJ, Despland P, Bruyer R, Linotte S, Crommelinck M. The N170 occipito-temporal component is delayed and enhanced to inverted faces but not to inverted objects: an electrophysiological account of face-specific processes in the human brain. Neuroreport 2000, 11(1):69-74. 10.1097/00001756-200001170-00014, 10683832.
33. Puce A, Allison T, McCarthy G. Electrophysiological studies of human face perception. III: Effects of top-down processing on face-specific potentials. Cereb Cortex 1999, 9(5):445-458. 10.1093/cercor/9.5.445, 10450890.
34. Severac Cauquil A, Edmonds GE, Taylor MJ. Is the face-sensitive N170 the only ERP not affected by selective attention?. Neuroreport 2000, 11(10):2167-2171.
35. Taylor MJ, Edmonds GE, McCarthy G, Allison T. Eyes first! Eye processing develops before face processing in children. Neuroreport 2001, 12(8):1671-1676. 10.1097/00001756-200106130-00031, 11409737.
36. Itier RJ, Taylor MJ. Effects of repetition and configural changes on the development of face recognition processes. Dev Sci 2004, 7(4):469-487. 10.1111/j.1467-7687.2004.00367.x, 15484595.
37. Bedard C, Belin P. A "voice inversion effect?". Brain Cogn 2004, 55(2):247-249. 10.1016/j.bandc.2004.02.008, 15177788.
38. Belin P, Fecteau S, Bedard C. Thinking the voice: neural correlates of voice perception. Trends Cogn Sci 2004, 8(3):129-135. 10.1016/j.tics.2004.01.008, 15301753.
39. Levy DA, Granot R, Bentin S. Processing specificity for human voice stimuli: electrophysiological evidence. Neuroreport 2001, 12(12):2653-2657. 10.1097/00001756-200108280-00013, 11522942.
40. Levy DA, Granot R, Bentin S. Neural sensitivity to human voices: ERP evidence of task and attentional influences. Psychophysiology 2003, 40(2):291-305. 10.1111/1469-8986.00031, 12820870.
41. Charest I, Pernet CR, Rousselet GA, Quinones I, Latinus M, Fillion-Bilodeau S, Chartrand JP, Belin P. Electrophysiological evidence for an early processing of human voices. BMC Neurosci 2009, 10:127. 10.1186/1471-2202-10-127, 2770575, 19843323.
42. Lattner S, Maess B, Wang Y, Schauer M, Alter K, Friederici AD. Dissociation of human and computer voices in the brain: evidence for a preattentive gestalt-like perception. Hum Brain Mapp 2003, 20(1):13-21. 10.1002/hbm.10118, 12953302.
43. Joassin F, Maurage P, Bruyer R, Crommelinck M, Campanella S. When audition alters vision: an event-related potential study of the cross-modal interactions between faces and voices. Neuroscience Lett 2004, 369(2):132-137.
44. Smith EL, Grabowecky M, Suzuki S. Auditory-visual crossmodal integration in perception of face gender. Curr Biol 2007, 17(19):1680-1685. 10.1016/j.cub.2007.08.043, 17825561.
45. Andersen TS, Tiippana K, Sams M. Factors influencing audiovisual fission and fusion illusions. Brain Research Cogn Brain Research 2004, 21(3):301-308.
46. Reddy L, Wilken P, Koch C. Face-gender discrimination is possible in the near-absence of attention. J Vis 2004, 4(2):106-117. 10.1167/4.2.4, 15005651.
47. Bindemann M, Burton AM, Hooge IT, Jenkins R, de Haan EH. Faces retain attention. Psychon Bull Rev 2005, 12(6):1048-1053.
48. Fort A, Delpuech C, Pernier J, Giard MH. Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 2002, 12(10):1031-1039. 10.1093/cercor/12.10.1031, 12217966.
49. Degerman A, Rinne T, Pekkola J, Autti T, Jaaskelainen IP, Sams M, Alho K. Human brain activity associated with audiovisual perception and attention. Neuroimage 2007, 34(4):1683-1691. 10.1016/j.neuroimage.2006.11.019, 17204433.
50. Vuilleumier P. Faces call for attention: evidence from patients with visual extinction. Neuropsychologia 2000, 38(5):693-700. 10.1016/S0028-3932(99)00107-4, 10689045.
51. Shams L, Kamitani Y, Thompson S, Shimojo S. Sound alters visual evoked potentials in humans. Neuroreport 2001, 12(17):3849-3852. 10.1097/00001756-200112040-00049, 11726807.
52. Talsma D, Doty TJ, Woldorff MG. Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration?. Cereb Cortex 2007, 17(3):679-690. 10.1093/cercor/bhk016, 16707740.
53. Talsma D, Woldorff MG. Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience 2005, 17(7):1098-1114. 10.1162/0898929054475172, 16102239.
54. Rossion B, Campanella S, Gomez CM, Delinte A, Debatisse D, Liard L, Dubois S, Bruyer R, Crommelinck M, Guerit JM. Task modulation of brain activity related to familiar and unfamiliar face processing: an ERP study. Clin Neurophysiol 1999, 110(3):449-462. 10.1016/S1388-2457(98)00037-6, 10363769.
55. Näätänen R, Picton T. The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. Psychophysiology 1987, 24(4):375-425. 10.1111/j.1469-8986.1987.tb00311.x, 3615753.
56. Alho K, Sams M, Paavilainen P, Naatanen R. Small pitch separation and the selective-attention effect on the ERP. Psychophysiology 1986, 23(2):189-197. 10.1111/j.1469-8986.1986.tb00617.x, 3704075.
57. Jacques C, Rossion B. Concurrent processing reveals competition between visual representations of faces. Neuroreport 2004, 15(15):2417-2421. 10.1097/00001756-200410250-00023, 15640767.
58. Michalewski HJ, Prasher DK, Starr A. Latency variability and temporal interrelationships of the auditory event-related potentials (N1, P2, N2, and P3) in normal subjects. Electroencephalogr Clin Neurophysiol 1986, 65(1):59-71. 10.1016/0168-5597(86)90037-7, 2416547.
59. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ. Functional imaging of human crossmodal identification and object recognition. Exp Brain Research 2005, 166(3-4):559-571.
60. Callan DE, Jones JA, Munhall K, Callan AM, Kroos C, Vatikiotis-Bateson E. Neural processes underlying perceptual enhancement by visual speech gestures. Neuroreport 2003, 14(17):2213-2218. 10.1097/00001756-200312020-00016, 14625450.
61. Calvert GA, Hansen PC, Iversen SD, Brammer MJ. Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 2001, 14(2):427-438. 10.1006/nimg.2001.0812, 11467916.
62. Sestieri C, Di Matteo R, Ferretti A, Del Gratta C, Caulo M, Tartaro A, Olivetti Belardinelli M, Romani GL. "What" versus "where" in the audiovisual domain: an fMRI study. Neuroimage 2006, 33(2):672-680. 10.1016/j.neuroimage.2006.06.045, 16934491.
63. Talsma D, Kok A, Slagter HA, Cipriani G. Attentional orienting across the sensory modalities. Brain Cogn 2008, 66(1):1-10. 10.1016/j.bandc.2007.04.005, 17553604.
64. Wagner AD. Working memory contributions to human learning and remembering. Neuron 1999, 22(1):19-22. 10.1016/S0896-6273(00)80674-1, 10027285.
65. Poldrack RA, Wagner AD, Prull MW, Desmond JE, Glover GH, Gabrieli JD. Functional specialization for semantic and phonological processing in the left inferior prefrontal cortex. Neuroimage 1999, 10(1):15-35. 10.1006/nimg.1999.0441, 10385578.
66. Bushara KO, Grafman J, Hallett M. Neural correlates of auditory-visual stimulus onset asynchrony detection. J Neuroscience 2001, 21(1):300-304.
67. Itier RJ, Taylor MJ. N170 or N1? Spatiotemporal differences between object and face processing using ERPs. Cereb Cortex 2004, 14(2):132-142. 10.1093/cercor/bhg111, 14704210.
68. Belin P, Zatorre RJ, Lafaille P, Ahad P, Pike B. Voice-selective areas in human auditory cortex. Nature 2000, 403(6767):309-312. 10.1038/35002078, 10659849.
69. Beauchemin M, De Beaumont L, Vannasing P, Turcotte A, Arcand C, Belin P, Lassonde M. Electrophysiological markers of voice familiarity. Eur J Neuroscience 2006, 23(11):3081-3086.
70. Campanella S, Belin P. Integrating face and voice in person perception. Trends Cogn Sci 2007, 11(12):535-543. 10.1016/j.tics.2007.10.001, 17997124.
71. Jeffreys DA. The influence of stimulus orientation on the vertex positive scalp potential evoked by faces. Experimental Brain Research 1993, 96(1):163-172.
72. Picton TW, Bentin S, Berg P, Donchin E, Hillyard SA, Johnson R, Miller GA, Ritter W, Ruchkin DS, Rugg MD, et al. Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 2000, 37(2):127-152. 10.1017/S0048577200000305, 10731765.
73. McDonald JJ, Teder-Salejarvi WA, Hillyard SA. Involuntary orienting to sound improves visual perception. Nature 2000, 407(6806):906-908. 10.1038/35038085, 11057669.
74. Berkovits I, Hancock GR, Nevitt J. Bootstrap resampling approaches for repeated measure designs: relative robustness to sphericity and normality violations. Educational and Psychological Measurement 2000, 60(6):877-892.
75. Wilcox RR. Introduction to Robust Estimation and Hypothesis Testing. 2005, Second.
76. Rousselet GA, Husk JS, Pernet CR, Gaspar CM, Bennett PJ, Sekuler AB. Age-related delay in information accrual for faces: evidence from a parametric, single-trial EEG approach. BMC Neurosci 2009, 10:114. 10.1186/1471-2202-10-114, 2746225, 19740414.