Interactions of auditory and visual stimuli in space and time

Hearing Research

Volumn 258, Issue 1-2, 2009, Pages 89-99

  Recanzone, Gregg H ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Auditory cortex; Multisensory; Perception; Vision

Indexed keywords

ARTICLE; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; ENERGY TRANSFER; HUMAN; ILLUSION; INFORMATION PROCESSING; NERVOUS SYSTEM FUNCTION; NONHUMAN; PRIMATE; PRIORITY JOURNAL; PSYCHOPHYSIOLOGY; SENSORY EVALUATION; SIGNAL PROCESSING; TIME; VISUAL DISCRIMINATION; VISUAL STIMULATION;

ACOUSTICS; ANIMALS; AUDITORY PERCEPTION; HEARING; HUMANS; MODELS, BIOLOGICAL; MODELS, NEUROLOGICAL; NERVOUS SYSTEM; PSYCHOPHYSICS; REACTION TIME; SOUND LOCALIZATION; SPACE PERCEPTION; TIME FACTORS; VISION, OCULAR; VISUAL PERCEPTION;

EID: 70449644799     PISSN: 03785955     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.heares.2009.04.009     Document Type: Article

References (65)

1. Abel S.M., Giguere C., Consoli A., and Papsin B.C. The effect of aging on horizontal plane sound localization. J. Acoust. Soc. Am. 108 (2000) 743-752
2. Alais D., and Burr D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14 (2004) 257-262
3. Altshuler M.W., and Comalli P.E. Effect of stimulus intensity and frequency on median horizontal plane sound localization. J. Auditory Res. 15 (1975) 262-265
4. Barraclough N.E., Xiao D., Baker C.I., Oram M.W., and Perrett D.I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 17 (2005) 377-391
5. Beauchamp M.S. See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr. Opin. Neurobiol. 15 (2005) 145-153
6. Bizley J.K., and King A.J. Visual-auditory spatial processing in auditory cortical neurons. Brain Res. 1242 (2008) 24-36
7. Bonath B., Noesselt T., Martinez A., Mishra J., Schwiecker K., Heinze H.J., and Hillyard S.A. Neural basis of the ventriloquist illusion. Curr. Biol. 17 (2007) 1697-1703
8. Brown C.H., Beecher M.D., Moody D.B., and Stebbins W.C. Localization of noise bands by Old World monkeys. J. Acoust. Soc. Am. 68 (1980) 127-132
9. Burr D., and Alais D. Combining visual and auditory information. Prog. Brain Res. 155 (2006) 243-258
10. Cavonius C.R., and Robbins D.O. Relationships between luminance and visual acuity in the rhesus monkey. J. Physiol. 232 (1973) 239-246
11. Cohen L.E., Batista A.P., and Andersen R.A. Comparison of neural activity preceding reaches to auditory and visual stimuli in the parietal reach region. Neuroreport 13 (2002) 891-894
12. Comalli P.E., and Altshuler M.W. Effect of stimulus intensity, frequency, and unilateral hearing loss on sound localization. J. Auditory Res. 16 (1976) 275-279
13. Freeman E., and Driver J. Direction of visual apparent motion driven solely by timing of a static sound. Curr. Biol. 18 (2008) 1262-1266
14. Frissen I., Vroomen J., de Gelder B., and Bertelson P. The aftereffects of ventriloquism: generalization across sound-frequencies. Acta Psychol. 118 (2005) 93-100
15. Gebhard J.W., and Mowbray G.H. On discriminating the rate of visual flicker and auditory flutter. Am. J. Psychol. 72 (1959) 521-528
16. Getzmann S. The effect of brief auditory stimuli on visual apparent motion. Perception 36 (2007) 1089-1103
17. Ghazanfar A.A., and Schroeder C.E. Is neocortex essentially multisensory?. Trends Cogn. Sci. 10 (2006) 278-285
18. Hay J.C., Pick Jr. H.L., and Ikeda K. Visual capture produced by prism spectacles. Psychon. Sci. 2 (1965) 215-216
19. Howard I.P., and Templeton W.B. Human Spatial Orientation (1966), Wiley, New York, NY
20. Jack C.E., and Thurlow W.R. Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. Percept. Mot. Skills 37 (1973) 967-979
21. Kaas J.H., and Hackett T.A. Subdivisions of auditory cortex and processing streams in primates. Proc. Natl. Acad. Sci. USA 97 (2000) 11793-11799
22. Kayser, C., Petkov, C.I., Logothetis, N.K., 2009. Multisensory interactions in primary auditory cortex: fMRI and electrophysiology. Hear. Res., doi:10.1016/j.heares.2009.02.011.
23. Knox G.W. Investigations of flicker and fusion: IV. The effect of auditory flicker on the pronouncedness of visual flicker. J. Gen. Psychol. 33 (1945) 145-154
24. Lakatos P., Chen C.M., O'Connel M.N., Mills A., and Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53 (2007) 279-292
25. Lakatos P., Karmos G., Mehta A.D., Ulbert I., and Schroeder C.E. Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320 (2008) 110-113
26. LeWald J. Rapid adaptation to auditory-visual spatial disparity. Learn. Mem. 9 (2002) 268-278
27. Linden J.F., Grunewald A., and Andersen R.A. Responses to auditory stimuli in macaque lateral intraparietal area. II. Behavioral modulation. J. Neurophysiol. 82 (1999) 343-358
28. Maunsell J.H., and Gibson J.R. Visual response latencies in striate cortex of the macaque monkey. J. Neurophysiol. 68 (1992) 1332-1344
29. Mazzoni P., Bracewell R.M., Barash S., and Andersen R.A. Spatially tuned auditory responses in area LIP of macaques performing delayed memory saccades to acoustic targets. J. Neurophysiol. 75 (1996) 1233-1241
30. Miller L.M., and Recanzone G.H. Firing rate across the population of auditory cortical neurons can accurately represent acoustic space in the alert macaque. Proc. Natl. Acad. Sci. USA 106 (2009) 5931-5935
31. Ogilvie J.C. Effect of auditory flutter on the visual critical flicker frequency. Can. J. Psychol. 10 (1956) 61-68
32. Olgivie J.C. The interaction of auditory flutter and cff: the effect of brightness. Can. J. Psychol. 10 (1956) 207-210
33. Parise C., and Spence C. Synesthetic congruency modulates the temporal ventriloquism effect. Neurosci. Lett. 442 (2008) 257-261
34. Phan M.L., Schendel K.L., Recanzone G.H., and Robertson L.R. Auditory and visual spatial localization deficits following bilateral parietal lobe lesions in a patient with Balint's syndrome. J. Cogn. Neurosci. 12 (2000) 583-600
35. Poremba A., Saunders R.C., Crane A.M., Cook M., Sokoloff L., and Mishkin M. Functional mapping of the primate auditory system. Science 299 (2003) 568-572
36. Radeau M., and Bertelson P. Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations. Percept. Psychophys. 22 (1977) 137-146
37. Radeau M., and Bertelson P. Cognitive factors and adaptation to auditory-visual discordance. Percept. Psychophys. 23 (1978) 341-343
38. Rauschecker J.P., Tian B., Pons T., and Mishkin M. Serial and parallel processing in rhesus monkey auditory cortex. J. Comp. Neurol. 26 (1997) 89-103
39. Rauschecker J.P., and Tian B. Mechanisms and streams for processing of "what" and "where" in auditory cortex. Proc. Natl. Acad. Sci. USA 97 (2000) 11800-11806
40. Recanzone G.H. Rapidly induced auditory plasticity: the ventriloquism aftereffect. Proc. Natl. Acad. Sci. USA 95 (1998) 869-875
41. Recanzone G.H., Makhamra S.D., and Guard D.C. Comparison of relative and absolute sound localization ability in humans. J. Acoust. Soc. Am. 103 (1998) 1085-1097
42. Recanzone G.H., Guard D.C., and Phan M.L. Frequency and intensity response properties of single neurons in the auditory cortex of the behaving macaque monkey. J. Neurophysiol. 83 (2000) 2315-2331
43. Recanzone G.H., Guard D.C., Phan M.L., and Su T.K. Correlation between the activity of single auditory cortical neurons and sound localization behavior in the macaque monkey. J. Neurophysiol. 83 (2000) 2723-2739
44. Recanzone G.H. Auditory influences on visual temporal rate perception. J. Neurophysiol. 89 (2003) 1078-1093
45. Recanzone G.H., and Beckerman N.S. Effects of intensity and location on sound location discrimination in macaque monkeys. Hear. Res. 198 (2004) 116-124
46. Sabin A.T., Macpherson E.A., and Middlebrooks J.C. Human sound localization at near-threshold levels. Hear. Res. 199 (2005) 124-134
47. Sato Y., Toyoizumi T., and Aihara K. Bayesian inference explains perception of unity and ventriloquism aftereffect: identification of common sources of audiovisual stimuli. Neural Comput. 19 (2007) 3335-3355
48. Schroeder C.E., Lindsley R.W., Specht C., Marcovici A., Smiley J.F., and Javitt D.C. Somatosensory input to auditory association cortex in the macaque monkey. J. Neurophysiol. 85 (2001) 1322-1327
49. Schroeder C.E., and Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cogn. Brain Res. 14 (2002) 187-198
50. Schroeder C.E., and Foxe J. Multisensory contributions to low-level, 'unisensory' processing. Curr. Opin. Neurobiol. 15 (2005) 454-458
51. Sekuler R., Sekuler A.B., and Lau R. Sound alters visual motion perception. Nature 385 (1997) 308
52. Shipley T. Auditory flutter-driving of visual flicker. Science 145 (1964) 1328-1330
53. Slutsky D.A., and Recanzone G.H. Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12 (2001) 7-10
54. Stein B.E., London N., Wilkinson L.K., and Price D.D. Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J. Cogn. Neurosci. 8 (1996) 497-506
55. Stevens S.S., and Newman E.B. The localization of actual sources of sound. Am. J. Psychol. 48 (1936) 297-306
56. Su T.K., and Recanzone G.H. Differential effect of near-threshold stimulus intensities on sound localization performance in azimuth and elevation in normal human subjects. J. Assoc. Res. Otolaryngol. 2 (2001) 246-256
57. Vroomen J., de Gelder B., and Vroomen J. Temporal ventriloquism: sound modulates the flash-lag effect. J. Exp. Psychol. Hum. Percept. Perform. 30 (2004) 513-518
58. Wade N.J., and Ziefle M. The surface and deep structure of the waterfall illusion. Psychol. Res. 72 (2008) 593-600
59. Welch R.B., and Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 88 (1980) 638-667
60. Welch R.B., Duttonhurt L.D., and Warren D.H. Contributions of audition and vision to temporal rate perception. Percept. Psychophys. 39 (1986) 294-300
61. Welch R.B. Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions. In: Aschersleben G., Bachmann T., and Musseler J. (Eds). Cognitive Contributions to the Perception of Spatial and Temporal Events (1999), Elsevier, Amsterdam 371-387
62. Werner-Reiss U., and Groh J.M. A rate code for sound azimuth in monkey auditory cortex: implications for human neuroimaging studies. J. Neurosci. 28 (2008) 3747-3758
63. Woods T.M., and Recanzone G.H. Visually-induced plasticity of auditory spatial perception in macaques. Curr. Biol. 15 (2004) 1559-1564
64. Woods T.M., and Recanzone G.H. Multimodal interactions evidenced by the ventriloquism effect in humans and monkeys. In: Spence C., Calvert G., and Stein B.E. (Eds). Handbook of Multisensory Processes (2004), MIT Press 35-48
65. Woods T.M., Lopez S.E., Long J.H., Rahman J.E., and Recanzone G.H. Effects of stimulus azimuth and intensity on the single neuron activity in the auditory cortex of the alert macaque monkey. J. Neurophysiol. 96 (2006) 3323-3337