Resolving multisensory conflict: A strategy for balancing the costs and benefits of audio-visual integration

Proceedings of the Royal Society B: Biological Sciences

Volumn 273, Issue 1598, 2006, Pages 2159-2168

  Roach, Neil W ^a   Heron, James ^b   McGraw, Paul V ^a  

^a UNIVERSITY OF NOTTINGHAM   (United Kingdom)

^b UNIVERSITY OF BRADFORD   (United Kingdom)

Author keywords

Audio visual conflict; Bayesian modelling; Multisensory integration

Indexed keywords

AUDITORY CUE; BAYESIAN ANALYSIS; COST-BENEFIT ANALYSIS; PERCEPTION; VISUAL ANALYSIS;

ARTICLE; AUDIOVISUAL EQUIPMENT; BAYES THEOREM; COST BENEFIT ANALYSIS; HEARING ACUITY; HUMAN; HUMAN EXPERIMENT; PERCEPTION; PRIORITY JOURNAL; SENSORY ANALYSIS; VISUAL ACUITY; VISUAL STIMULATION;

EID: 33748272486     PISSN: 09628452     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rspb.2006.3578     Document Type: Article

References (41)

1. Adams, W. J., Graf, E. W. & Ernst, M. O. 2004 Experience can change the 'light-from-above' prior. Nat. Neurosci. 7, 1057-1058. (doi:10.1038/nn1312)
2. Alais, D. & Burr, D. 2004 The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257-262. (doi:10.1016/S0960- 9822(04)00043-0)
3. Battaglia, P. W., Jacobs, R. A. & Aslin, R. N. 2003 Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A 20, 1391-1397.
4. Bertelson, P. & Radeau, M. 1981 Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 29, 578-584.
5. Bresciani, J. P., Ernst, M. O., Drewing, K., Bouyer, G., Maury, V. & Kheddar, A. 2005 Feeling what you hear: auditory signals can modulate tactile tap perception. Exp. Brain Res. 162, 172-180. (doi:10.1007/s00221-0042128-2)
6. Calvert, G. A., Bullmore, E., Brammer, M. J., Campbell, R., Iversen, S. D., Woodruff, P., McGuire, P., Williams, S. & David, A. S. 1997 Activation of auditory cortex during silent lipreading. Science 276, 593-596. (doi:10.1126/science.276.5312.593)
7. Clark, J. J. & Yuille, A. L. 1990 Data fusion for sensory information processing systems. Norwell, MA: Kluwer Academic.
8. Driver, J. & Spence, C. 2000 Multisensory perception: beyond modularity and convergence. Curr. Biol. 10, R731-R735. (doi:10.1016/80960- 9822(00)00740-5)
9. Ersnt, M. O. 2005 A Bayesian view on multimodal cue integration. In Human body perception from the inside out (ed. G. Knoblich, I. Thornton, M. Grosjean & M. Shiffrar). New York, NY: Oxford University Press.
10. Ernst, M. O. & Banks, M. S. 2002 Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429-433. (doi:10.1038/415429a)
11. Ersnt, M. O. & Bülthoff, H. H. 2004 Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162-169. (doi:10. 1016/j.tics.2004.02.002)
12. Falchier, A., Clavagnier, S., Barone, P. & Kennedy, H. 2002 Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749-5759.
13. Fu, K., Johnston, T., Shah, A., Arnold, L., Smiley, J., Hackett, T., Garraghty, P. & Schroeder, C. 2003 Auditory cortical neurons respond to somatosensory stimulation. J. Neurosci. 23, 7510-7515.
14. Gebhard, J. W. & Mowbray, G. H. 1959 On discriminating the rate of visual flicker and auditory flutter. Am. J. Psychol. 72, 521-528. (doi:10.2307/1419493)
15. Gepshtein, S. & Banks, M. S. 2003 Viewing geometry determines how vision and haptics combine in size perception. Curr. Biol. 13, 483-488. (doi:10.1016/S0960-9822(03)00133-7)
16. Gepshtein, S., Burge, J., Ernst, M. O. & Banks, M. S. 2005 The combination of vision and touch depends on spatial proximity. J. Vis. 5, 1013-1023. (doi:10.1167/5.11.7)
17. Hillis, J. M., Ernst, M. O., Banks, M. S. & Landy, M. S. 2002 Combining sensory information: mandatory fusion within, but not between, senses. Science 298, 1627-1630. (doi:10.1126/science.1075396)
18. Hillis, J. M., Watt, S. J., Landy, M. S. & Banks, M. S. 2004 Slant from texture and disparity cues: optimal cue combination. J. Vis. 4, 967-992. (doi:10.1167/4.12.1)
19. Jack, C. E. & Thurlow, W. R. 1973 Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. Percept. Mot. Skills 37, 967-979.
20. Jacobs, R. A. 1999 Optimal integration of texture and motion cues to depth. Vis. Res. 39, 3621-3629. (doi:10.1016/80042-6989(99)00088-7)
21. Knill, D. C. & Pouget, A. 2004 The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712-719. (doi:10.1016/j.tins.2004.10.007)
22. Knill, D. C. & Saunders, J. A. 2003 Do humans optimally integrate stereo and texture information for judgments of surface slant? Vis. Res. 43, 2539-2558. (doi:10.1016/S0042-6989(03)00458-9)
23. Landy, M. S. & Kojima, H. 2001 Ideal cue combination for localizing texture-defined edges. J. Opt. Soc. Am. A 18, 2307-2320.
24. Landy, M. S., Maloney, L. T., Johnston, E. B. & Young, M. 1995 Measurement and modeling of depth cue combination: in defense of weak fusion. Vis. Res. 35, 389-412. (doi:10.1016/0042-6989(94)00176-M)
25. Macaluso, E., Frith, C. D. & Driver, J. 2000 Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206-1208. (doi:10.1126/science.289.5482.1206)
26. Meredith, M. A. 2002 On the neuronal basis for multisensory convergence: a brief overview. Cogn. Brain Res. 14, 31-40. (doi:10.1016/S0926-6410(02)00059- 9)
27. Myers, A. K., Cotton, B. & Hilp, H. A. 1981 Matching the rate of concurrent tone bursts and light flashes as a function of flash surround luminance. Percept. Psychophys. 30, 33-38.
28. Pick Jr, H. L., Warren, D. H. & Hay, J. C. 1969 Sensory conflict in judgments of spatial direction. Percept. Psychophys. 6, 203-205.
29. Recanzone, G. H. 2003 Auditory influences on visual temporal rate perception. J. Neurophysiol. 89, 1078-1093.
30. Rockland, K. S. & Ojima, H. 2003 Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophys. 50, 19-26. (doi:10.1016/S0167-8760(03)00121-1)
31. Rushton, S. K. & Wann, J. P. 1999 Weighted combination of size and disparity: a computational model for timing a ball catch. Nat. Neurosci. 2, 186-190. (doi:10.1038/5750)
32. Schroeder, C. E., Lindsley, R. W., Specht, C., Marcovici, A., Smiley, J. F. & Javitt, D. C. 2001 Somatosensory input to auditory association cortex in macaque monkey. J. Neurophysiol. 85, 1322-1327.
33. Shams, L., Kamitani, Y. & Shimojo, S. 2002 What you see is what you hear. Nature 408, 788. (doi:10.1038/35048669)
34. Shams, L., Ma, W. J. & Beierholm, U. 2005 Sound-induced flash illusion as an optimal percept. Neuroreport 16, 1923-1927. (doi:10.1097/01.wnr. 0000187634.68504.bb)
35. Shipley, T. 1964 Auditory flutter-driving of visual flicker. Science 145, 1328-1330.
36. van Beers, R. J., Sittig, A. C. & Denier van der Gon, J. J. 1999 Integration of proprioceptive and visual position-information: an experimentally supported model. J. Neurophysiol. 81, 1355-1364.
37. van Beers, R. J., Wolpert, D. M. & Haggard, P. 2002 When feeling is more important than seeing in sensorimotor adaptation. Curr. Biol. 12, 834-837. (doi:10.1016/S0960-9822(02)00836-9)
38. Warren, D. H. & Cleaves, W. T. 1971 Visual-proprioceptive interaction under large amounts of conflict. J. Exp. Psychol. 90, 206-214.
39. Welch, R. B. & Warren, D. H. 1980 Immediate perceptual response to inter-sensory discrepancy. Psychol. Bull. 88, 638-667. (doi:10.1037//0033-2909. 88.3.638)
40. Welch, R. B., Duttonhurt, L. D. & Warren, D. H. 1986 Contributions of audition and vision to temporal rate perception. Percept. Psychophys. 39, 294-300.
41. Witton, I. B. & Knudsen, E. I. 2005 Why seeing is believing: merging auditory and visual worlds. Neuron 48, 489-496. (doi:10.1016/j.neuron.2005.10. 020)