Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events

Brain Research Bulletin

Volumn 75, Issue 6, 2008, Pages 753-760

  Bresciani, Jean Pierre ^a   Dammeier, Franziska ^a   Ernst, Marc O ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

Audition; Multimodal integration; Touch; Virtual environments; Vision

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; HEARING; HUMAN; HUMAN EXPERIMENT; NORMAL HUMAN; PRIORITY JOURNAL; RELIABILITY; TACTILE STIMULATION; TOUCH; VISION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; AFFERENT PATHWAYS; AUDITORY PERCEPTION; BRAIN; COGNITION; DISCRIMINATION (PSYCHOLOGY); HUMANS; ILLUSIONS; JUDGMENT; NEUROPSYCHOLOGICAL TESTS; OBSERVER VARIATION; PHOTIC STIMULATION; PHYSICAL STIMULATION; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REPRODUCIBILITY OF RESULTS; TIME PERCEPTION; TOUCH; USER-COMPUTER INTERFACE; VISUAL PERCEPTION;

EID: 41249085379     PISSN: 03619230     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainresbull.2008.01.009     Document Type: Article

References (50)

1. Alais D., and Burr D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14 (2004) 257-262
2. Andersen T.S., Tiippana K., and Sams M. Maximum Likelihood Integration of rapid flashes and beeps. Neurosci. Lett. 380 (2005) 155-160
3. Avanzini F., and Crosato P. Integrating physically based sound models in a multimodal rendering architecture. Comput. Anim. Vir. Worlds 17 (2006) 411-419
4. Bermant R.I., and Welch R.B. Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition. Percept. Mot. Skills 42 (1976) 487-493
5. Bernstein I.H., Clark M.H., and Edelstein B.A. Effects of an auditory signal on visual reaction time. J. Exp. Psychol. 80 (1969) 567-569
6. Bertelson P., and Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 29 (1981) 578-584
7. Bresciani J.P., Dammeier F., and Ernst M.O. Vision and touch are automatically integrated for the perception of sequences of events. J. Vis. 6 (2006) 554-564
8. Bresciani J.-P., and Ernst M.O. Signal reliability modulates auditory-tactile integration for event counting. NeuroReport 18 11 (2007) 1157-1161
9. Bresciani J.P., Ernst M.O., Drewing K., Bouyer G., Maury V., and Kheddar A. Feeling what you hear: auditory signals can modulate tactile tap perception. Exp. Brain. Res. 162 (2005) 172-180
10. Collins J.K., and Singer G. Interaction between sensory spatial after-effects and persistence of response following behavioral compensation. J. Exp. Psychol. 77 (1968) 301-307
11. Day R.H., and Singer G. Sensory adaptation and behavioral compensation with spatially transformed vision and hearing. Psychol. Bull. 67 (1967) 307-322
12. De Gelder B., and Bertelson P. Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7 (2003) 460-467
13. Deneve S., Latham P.E., and Pouget A. Efficient computation and cue integration with noisy population codes. Nat. Neurosci. 4 (2001) 826-831
14. Diederich A., Colonius H., Bockhorst D., and Tabeling S. Visual-tactile spatial interaction in saccade generation. Exp. Brain Res. 148 (2003) 328-337
15. DiFranco D.E., Beauregard G.L., and Srinivasan M.A. The effect of auditory cues on the haptic perception of stiffness in virtual environments. Proc. ASME Dyn. Syst. Contr. Div. 61 (1997) 17-22
16. Ernst M.O., and Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415 (2002) 429-433
17. Ernst M.O., and Bulthoff H.H. Merging the senses into a robust percept. Trends Cogn. Sci. 8 (2004) 162-169
18. Fendrich R., and Corballis P.M. The temporal cross-capture of audition and vision. Percept. Psychophys. 63 (2001) 719-725
19. Fishkin S.M., Pishkin V., and Stahl M.L. Factors involved in visual capture. Percept. Mot. Skills 40 (1975) 427-434
20. Foxe J.J., and Schroeder C.E. The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16 (2005) 419-423
21. Gebhard J.W., and Mowbray G.H. On discriminating the rate of visual flicker and auditory flutter. Am. J. Psychol. 72 (1959) 521-529
22. Gepshtein S., and Banks M.S. Viewing geometry determines how vision and haptics combine in size perception. Curr. Biol. 13 (2003) 483-488
23. Gielen S.C., Schmidt R.A., and Van den Heuvel P.J. On the nature of intersensory facilitation of reaction time. Percept. Psychophys. 34 (1983) 161-168
24. Guest S., Catmur C., Lloyd D., and Spence C. Audiotactile interactions in roughness perception. Exp. Brain. Res. 146 (2002) 161-171
25. Guest S., and Spence C. Tactile dominance in speeded discrimination of textures. Exp. Brain Res. 150 (2003) 201-207
26. Guest S., and Spence C. What role does multisensory integration play in the visuotactile perception of texture?. Int J. Psychophysiol. 50 (2003) 63-80
27. Hershenson M. Reaction time as a measure of intersensory facilitation. J. Exp. Psychol. 63 (1962) 289-293
28. Hikosaka K., Iwai E., Saito H., and Tanaka K. Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J. Neurophysiol. 60 (1988) 1615-1637
29. Hotting K., and Roder B. Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychol. Sci. 15 (2004) 60-64
30. Hyvarinen J., and Poranen A. Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys. Brain 97 (1974) 673-692
31. Jousmaki V., and Hari R. Parchment-skin illusion: sound-biased touch. Curr. Biol. 8 (1998) R190
32. Kitagawa N., and Ichihara S. Hearing visual motion in depth. Nature 416 (2002) 172-174
33. Klatzky R.L., Pai D.K., and Krotkov E.P. Perception of material from contact sounds. Presence: Teleoperat. Vir. Environ. 9 (2000) 399-410
34. Leinonen L., Hyvarinen J., and Sovijarvi A.R. Functional properties of neurons in the temporo-parietal association cortex of awake monkey. Exp. Brain Res. 39 (1980) 203-215
35. Ma W.J., Beck J.M., Latham P.E., and Pouget A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9 (2006) 1432-1438
36. Morein-Zamir S., Soto-Faraco S., and Kingstone A. Auditory capture of vision: examining temporal ventriloquism. Brain Res. Cogn. Brain Res. 17 (2003) 154-163
37. Morrell L.K. Temporal characteristics of sensory interaction in choice reaction times. J. Exp. Psychol. 77 (1968) 14-18
38. Nickerson R.S. Intersensory facilitation of reaction time: energy summation or preparation enhancement?. Psychol. Rev. 80 (1973) 489-509
39. Recanzone G.H. Auditory influences on visual temporal rate perception. J. Neurophysiol. 89 (2003) 1078-1093
40. Rock I., and Victor J. Vision and touch: An experimentally created conflict between the two senses. Science 143 (1964) 594-596
41. Sekuler R., Sekuler A.B., and Lau R. Sound alters visual motion perception. Nature 385 (1997) 308
42. Shams L., Kamitani Y., and Shimojo S. Illusions. What you see is what you hear. Nature 408 (2000) 788
43. Shipley T. Auditory flutter-driving of visual flicker. Science 145 (1964) 1328-1330
44. Singer G., and Day R.H. Spatial adaptation and aftereffect with optically transformed vision: effects of active and passive responding and the relationship between test and exposure responses. J. Exp. Psychol. 71 (1966) 725-731
45. Stein B.E., Wallace M.W., Stanford T.R., and Jiang W. Cortex governs multisensory integration in the midbrain. Neuroscientist 8 (2002) 306-314
46. van Beers R.J., Sittig A.C., and Gon J.J. Integration of proprioceptive and visual position-information: an experimentally supported model. J. Neurophysiol. 81 (1999) 1355-1364
47. Violentyev A., Shimojo S., and Shams L. Touch-induced visual illusion. Neuroreport 16 (2005) 1107-1110
48. Wada Y., Kitagawa N., and Noguchi K. Audio-visual integration in temporal perception. Int. J. Psychophysiol. 50 (2003) 117-124
49. Wagenaar W.A. Note on the construction of diagram-balanced latin squares. Psychol. Bull. 72 (1969) 384-386
50. Watanabe K., and Shimojo S. When sound affects vision: effects of auditory grouping on visual motion perception. Psychol. Sci. 12 (2001) 109-116