Integration of auditory and visual information in the recognition of realistic objects

Experimental Brain Research

Volumn 194, Issue 1, 2009, Pages 91-102

  Suied, Clara ^b   Bonneel, Nicolas ^c   Viaud Delmon, Isabelle ^a,b  

^a PITIÉ SALPÊTRIÈRE HOSPITAL   (France)

^b CNRS   (France)

^c INRIA   (France)

Author keywords

Audiovisual; Human; Multisensory integration; Object recognition; Reaction time; Simon effect; Spatial disparity

Indexed keywords

ADULT; ARTICLE; AUDITORY SYSTEM; CONTROLLED STUDY; FEMALE; HAND; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; MALE; NORMAL HUMAN; PRIORITY JOURNAL; REACTION TIME; RECOGNITION; SEMANTICS; SENSORIMOTOR INTEGRATION; SENSORY SYSTEM; SPATIAL ORIENTATION; STIMULUS RESPONSE; VIRTUAL REALITY; VISUAL INFORMATION;

ACOUSTIC STIMULATION; ADULT; ANALYSIS OF VARIANCE; AUDITORY PERCEPTION; FEMALE; HAND; HUMANS; MALE; PHOTIC STIMULATION; REACTION TIME; RECOGNITION (PSYCHOLOGY); SPACE PERCEPTION; VISUAL PERCEPTION;

EID: 62149145171     PISSN: 00144819     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00221-008-1672-6     Document Type: Article

References (54)

1. D Alais C Morrone D Burr 2006 Separate attentional resources for vision and audition Proc Biol Sci 273 1339 1345
2. FL Bedford 2001 Toward a general law of numerical/object identity Cahiers de Psychologie Cognitive/Curr Psychol Cogn 20 113 176
3. F Bedford 2004 Analysis of a constraint on perception, cognition, and development: one object, one place, one time J Exp Psychol Hum Percept Perform 30 907 912
4. Bertelson P, Vroomen J, Wiegeraad G, de Gelder B (1994) Exploring the relation between McGurk interference and ventriloquism. In: International conference on spoken language processing, Yokohama, Japan, pp 556-562
5. GA Calvert T Thesen 2004 Multisensory integration: methodological approaches and emerging principles in the human brain J Physiol Paris 98 191 205
6. GA Calvert MJ Brammer SD Iversen 1998 Crossmodal identification Trends Cogn Sci 2 247 253
7. A Caramazza AE Hillis BC Rapp C Romani 1990 The multiple semantic hypothesis: multiple confusions? Cogn Neuropsychol 7 161 189
8. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, Hillsdale
9. F Frassinetti N Bolognini E Ladavas 2002 Enhancement of visual perception by crossmodal visuo-auditory interaction Exp Brain Res 147 332 343
10. MA Frens AJ Van Opstal RF Van der Willigen 1995 Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements Percept Psychophys 57 802 816
11. MH Giard F Peronnet 1999 Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study J Cogn Neurosci 11 473 490
12. M Giray R Ulrich 1993 Motor coactivation revealed by response force in divided and focused attention J Exp Psychol Hum Percept Perform 19 1278 1291
13. M Gondan B Niederhaus F Rosler B Roder 2005 Multisensory processing in the redundant-target effect: a behavioral and event-related potential study Percept Psychophys 67 713 726
14. GR Grice L Canham 1990 Redundancy phenomena are affected by response requirements Percept Psychophys 48 209 213
15. GR Grice JW Gwynne 1987 Dependence of target redundancy effects on noise conditions and number of targets Percept Psychophys 42 29 36
16. GR Grice JM Reed 1992 What makes targets redundant? Percept Psychophys 51 437 442
17. GR Grice L Canham JW Gwynne 1984 Absence of a redundant-signals effect in a reaction time task with divided attention Percept Psychophys 36 565 570
18. LK Harrington CK Peck 1998 Spatial disparity affects visual-auditory interactions in human sensorimotor processing Exp Brain Res 122 247 252
19. M Hershenson 1962 Reaction time as a measure of intersensory facilitation J Exp Psychol 63 289 293
20. NP Holmes C Spence 2005 Multisensory integration: space, time and superadditivity Curr Biol 15 R762 R764
21. HC Hughes PA Reuter-Lorenz G Nozawa R Fendrich 1994 Visual-auditory interactions in sensorimotor processing: saccades versus manual responses J Exp Psychol Hum Percept Perform 20 131 153
22. RA Kinchla 1974 Detecting target elements in multielement displays: a confusability model Percept Psychophys 15 149 158
23. PJ Laurienti RA Kraft JA Maldjian JH Burdette MT Wallace 2004 Semantic congruence is a critical factor in multisensory behavioral performance Exp Brain Res 158 405 414
24. S Lehmann MM Murray 2005 The role of multisensory memories in unisensory object discrimination Brain Res Cogn Brain Res 24 326 334
25. CH Lu RW Proctor 1995 The influence of irrelevant location information on performance: a review of the Simon and spatial Stroop effects Psychon Bull Rev 2 174 207
26. Luce RD (1986) Response times: their role in inferring elementary mental organization. Oxford University Press, New York
27. A Martin 2007 The representation of object concepts in the brain Annu Rev Psychol 58 25 45
28. J Miller 1982 Divided attention: evidence for coactivation with redundant signals Cogn Psychol 14 247 279
29. J Miller 1986 Timecourse of coactivation in bimodal divided attention Percept Psychophys 40 331 343
30. J Miller 1991 Channel interaction and the redundant-targets effect in bimodal divided attention J Exp Psychol Hum Percept Perform 17 160 169
31. Moeck T, Bonneel N, Tsingos N, Drettakis G, Viaud-Delmon I, Alloza D (2007) Progressive perceptual audio rendering of complex scenes. In: ACM SIGGRAPH symposium on interactive 3D graphics and games
32. S Molholm W Ritter DC Javitt JJ Foxe 2004 Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study Cereb Cortex 14 452 465
33. JT Mordkoff S Yantis 1991 An interactive race model of divided attention J Exp Psychol Hum Percept Perform 17 520 538
34. MM Murray S Molholm CM Michel DJ Heslenfeld W Ritter DC Javitt CE Schroeder JJ Foxe 2005 Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment Cereb Cortex 15 963 974
35. K Patterson PJ Nestor TT Rogers 2007 Where do you know what you know? The representation of semantic knowledge in the human brain Nat Rev Neurosci 8 976 987
36. DH Raab 1962 Statistical facilitation of simple reaction times Trans N Y Acad Sci 24 574 590
37. M Radeau P Bertelson 1977 Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations Percept Psychophys 22 137 146
38. M Radeau P Bertelson 1978 Cognitive factors and adaptation to auditory-visual discordance Percept Psychophys 23 341 343
39. MJ Riddoch GW Humphreys M Coltheart E Funnell 1988 Semantic systems or system? Neuropsychological evidence re-examined Cogn Neuropsychol 5 3 25
40. S Savazzi CA Marzi 2008 Does the redundant signal effect occur at an early visual stage? Exp Brain Res 184 275 281
41. M Schmitt A Postma E de Haan 2000 Interactions between exogenous auditory and visual spatial attention Q J Exp Psychol A 53 105 130
42. E Schroger A Widmann 1998 Speeded responses to audiovisual signal changes result from bimodal integration Psychophysiology 35 755 759
43. W Schwarz 1996 Further tests of the interactive race model of divided attention: the effects of negative bias and varying stimulus-onset asynchronies Psychol Res 58 233 245
44. JR Simon JL Craft 1970 Effects of an irrelevant auditory stimulus on visual choice reaction time J Exp Psychol 86 272 274
45. JR Simon PE Sly S Vilapakkam 1981 Effect of compatibility of SR mapping on reactions toward the stimulus source Acta Psychol 47 63 81
46. EL Smith M Grabowecky S Suzuki 2007 Auditory-visual crossmodal integration in perception of face gender Curr Biol 17 1680 1685
47. Stein BE, Meredith MA (1993) The merging of the senses. MIT, Cambridge
48. BE Stein N London LK Wilkinson DD Price 1996 Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis J Cogn Neurosci 8 497 506
49. WA Teder-Salejarvi F Di Russo JJ McDonald SA Hillyard 2005 Effects of spatial congruity on audio-visual multimodal integration J Cogn Neurosci 17 1396 1409
50. R Ulrich J Miller 1993 Information processing models generating lognormally distributed reaction times J Math Psychol 37 513 525
51. R Ulrich J Miller H Schroter 2007 Testing the race model inequality: an algorithm and computer programs Behav Res Methods 39 291 302
52. S Yuval-Greenberg LY Deouell 2007 What you see is not (always) what you hear: induced gamma band responses reflect cross-modal interactions in familiar object recognition J Neurosci 27 1090 1096
53. M Zampini D Torresan C Spence MM Murray 2007 Auditory-somatosensory multisensory interactions in front and rear space Neuropsychologia 45 1869 1877
54. M Zorzi C Umilta 1995 A computational model of the Simon effect Psychol Res 58 193 205