Crossmodal interactions and multisensory integration in the perception of audio-visual motion - A free-field study

Brain Research

Volumn 1466, Issue , 2012, Pages 99-111

  Schmiedchen, Kristina ^a   Freigang, Claudia ^a   Nitsche, Ines ^a   Rübsamen, Rudolf ^a  

^a UNIVERSITY OF LEIPZIG   (Germany)

Author keywords

Attention; Audio visual; Dynamic capture; Motion perception; Perceived unity; Trajectory

Indexed keywords

ACOUSTICS; ARTICLE; ATTENTION; AUDITORY STIMULATION; HUMAN; MOVEMENT (PHYSIOLOGY); MOVEMENT PERCEPTION; PRIORITY JOURNAL; SIGNAL PROCESSING; TASK PERFORMANCE; VISION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; ATTENTION; AUDITORY PERCEPTION; DISCRIMINATION (PSYCHOLOGY); FEMALE; HUMANS; MALE; MOTION PERCEPTION; PHOTIC STIMULATION; VISUAL PERCEPTION;

EID: 84862993175     PISSN: 00068993     EISSN: 18726240     Source Type: Journal    
DOI: 10.1016/j.brainres.2012.05.015     Document Type: Article

References (64)

1. D. Alais, and D. Burr The ventriloquist effect results from near-optimal bimodal integration Curr. Biol. 14 3 2004 257 262 (Pubitemid 38215356)
2. T.D. Albright, and G.R. Stoner Visual motion perception Proc. Natl. Acad. Sci. U. S. A. 92 7 1995 2433 2440
3. A. Alink, W. Singer, and L. Muckli Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex J. Neurosci. 28 11 2008 2690 2697 (Pubitemid 351398870)
4. A. Alink, F. Euler, E. Galeano, A. Krugliak, W. Singer, and A. Kohler Auditory motion capturing ambiguous visual motion Front. Psychol. 2 2012 391
5. P.G. Allen, and P.A. Kolers Sensory specificity of apparent motion J. Exp. Psychol. Hum. Percept. Perform. 7 6 1981 1318 1328
6. L.H. Arnal, V. Wyart, and A. Giraud Transitions in neural oscillations reflect prediction errors generated in audiovisual speech Nat. Neurosci. 14 6 2011 797 801
7. P.W. Battaglia, R.A. Jacobs, and R.N. Aslin Bayesian integration of visual and auditory signals for spatial localization J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20 7 2003 1391 1397
8. P. Bertelson, and M. Radeau Cross-modal bias and perceptual fusion with auditory-visual spatial discordance Percept. Psychophys. 29 6 1981 578 584
9. P. Bertelson, J. Vroomen, B. de Gelder, and J. Driver The ventriloquist effect does not depend on the direction of deliberate visual attention Percept. Psychophys. 62 2 2000 321 332
10. N. Bolognini, F. Frassinetti, A. Serino, and E. Làdavas "Acoustical vision" of below threshold stimuli: interaction among spatially converging audiovisual inputs Exp. Brain Res. 160 3 2005 273 282 (Pubitemid 40169419)
11. A. Brooks, R. van der Zwan, A. Billard, B. Petreska, S. Clarke, and O. Blanke Auditory motion affects visual biological motion processing Neuropsychologia 45 3 2007 523 530 (Pubitemid 44894454)
12. D. Burr, and D. Alais Combining visual and auditory information Prog. Brain Res. 155 2006 243 258
13. G.A. Calvert, P.C. Hansen, S.D. Iversen, and M.J. Brammer Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect Neuroimage 14 2 2001 427 438 (Pubitemid 32709557)
14. H. Colonius, and A. Diederich The optimal time window of visual-auditory integration: a reaction time analysis Front. Integr. Neurosci. 4 2010 11
15. M.O. Ernst, and M.S. Banks Humans integrate visual and haptic information in a statistically optimal fashion Nature 415 6870 2002 429 433 (Pubitemid 34100953)
16. M.O. Ernst, and H.H. Bülthoff Merging the senses into a robust percept Trends Cogn. Sci. 8 4 2004 162 169 (Pubitemid 38429926)
17. D. Finlay Motion perception in the peripheral visual field Perception 11 4 1982 457 462 (Pubitemid 13087792)
18. E. Freeman, and J. Driver Direction of visual apparent motion driven solely by timing of a static sound Curr. Biol. 18 16 2008 1262 1266
19. S. Getzmann The effect of brief auditory stimuli on visual apparent motion Perception 36 7 2007 1089 1103
20. D.W. Grantham Detection and discrimination of simulated motion of auditory targets in the horizontal plane J. Acoust. Soc. Am. 79 6 1986 1939 1949 (Pubitemid 16034865)
21. W.D. Hairston, M.T. Wallace, J.W. Vaughan, B.E. Stein, J.L. Norris, and J.A. Schirillo Visual localization ability influences cross-modal bias J. Cogn. Neurosci. 15 1 2003 20 29 (Pubitemid 36106260)
22. S. Hidaka, W. Teramoto, Y. Sugita, Y. Manaka, S. Sakamoto, and Y. Suzuki Auditory motion information drives visual motion perception PLoS One 6 3 2011 e17499
23. M. Hofbauer, S.M. Wuerger, G.F. Meyer, F. Roehrbein, K. Schill, and C. Zetzsche Catching audiovisual mice: predicting the arrival time of auditory-visual motion signals Cogn. Affect. Behav. Neurosci. 4 2 2004 241 250 (Pubitemid 39167166)
24. T.L. Hubbard Representational momentum and related displacements in spatial memory: a review of the findings Psychon. Bull. Rev. 12 5 2005 822 851 (Pubitemid 43341538)
25. A. Jain, S.L. Sally, and T.V. Papathomas Audiovisual short-term influences and aftereffects in motion: examination across three sets of directional pairings J. Vis. 8 15 2008 7.1 7.13
26. H. Kafaligonul, and G.R. Stoner Auditory modulation of visual apparent motion with short spatial and temporal intervals J. Vis. 10 12 2010 31
27. A.J. King Multisensory integration: strategies for synchronization Curr. Biol. 15 9 2005 R339 R341
28. N. Kitajima, and Y. Yamashita Dynamic capture of sound motion by light stimuli moving in three-dimensional space Percept. Mot. Skills 89 3 Pt 2 1999 1139 1158
29. J. Lewald, and R. Guski Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli Brain Res. Cogn. Brain Res. 16 3 2003 468 478 (Pubitemid 36426932)
30. R. Lewis, and U. Noppeney Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas J. Neurosci. 30 37 2010 12329 12339
31. S. Mateeff, J. Hohnsbein, and T. Noack Dynamic visual capture: apparent auditory motion induced by a moving visual target Perception 14 6 1985 721 727
32. H. McGurk, and J. MacDonald Hearing lips and seeing voices Nature 264 5588 1976 746 748
33. S.P. McKee, and K. Nakayama The detection of motion in the peripheral visual field Vision Res. 24 1 1984 25 32 (Pubitemid 14221098)
34. M.A. Meredith, and B.E. Stein Interactions among converging sensory inputs in the superior colliculus Science 221 4608 1983 389 391 (Pubitemid 13067842)
35. M.A. Meredith, J.W. Nemitz, and B.E. Stein Determinants of multisensory integration in superior colliculus neurons. I. temporal factors J. Neurosci. 7 10 1987 3215 3229
36. G.F. Meyer, and S.M. Wuerger Cross-modal integration of auditory and visual motion signals Neuroreport 12 11 2001 2557 2560 (Pubitemid 32682221)
37. G.F. Meyer, S.M. Wuerger, F. Röhrbein, and C. Zetzsche Low-level integration of auditory and visual motion signals requires spatial co-localisation Exp. Brain Res. 166 3-4 2005 538 547 (Pubitemid 41502097)
38. G.F. Meyer, M. Greenlee, and S. Wuerger Interactions between auditory and visual semantic stimulus classes: evidence for common processing networks for speech and body actions J. Cogn. Neurosci. 23 9 2011 2291 2308
39. J.C. Middlebrooks, and D.M. Green Sound localization by human listeners Annu. Rev. Psychol. 42 1991 135 159
40. S. Morein-Zamir, S. Soto-Faraco, and A. Kingstone Auditory capture of vision: examining temporal ventriloquism Brain Res. Cogn. Brain Res. 17 1 2003 154 163 (Pubitemid 36577550)
41. J. Navarra, A. Vatakis, M. Zampini, S. Soto-Faraco, W. Humphreys, and C. Spence Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration Brain Res. Cogn. Brain Res. 25 2 2005 499 507 (Pubitemid 41443353)
42. D.R. Perrott, H. Ambarsoom, and J. Tucker Changes in head position as a measure of auditory localization performance: auditory psychomotor coordination under monaural and binaural listening conditions J. Acoust. Soc. Am. 82 5 1987 1637 1645
43. D.R. Perrott, K. Saberi, K. Brown, and T.Z. Strybel Auditory psychomotor coordination and visual search performance Percept. Psychophys. 48 3 1990 214 226 (Pubitemid 20286501)
44. G.H. Recanzone Interactions of auditory and visual stimuli in space and time Hear. Res. 258 1-2 2009 89 99
45. D. Sanabria, J. Lupiañez, and C. Spence Auditory motion affects visual motion perception in a speeded discrimination task Exp. Brain Res. 178 3 2007 415 421 (Pubitemid 46440587)
46. D. Sanabria, C. Spence, and S. Soto-Faraco Perceptual and decisional contributions to audiovisual interactions in the perception of apparent motion: a signal detection study Cognition 102 2 2007 299 310 (Pubitemid 44917949)
47. D. Senkowski, D. Talsma, M. Grigutsch, C.S. Herrmann, and M.G. Woldorff Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations Neuropsychologia 45 3 2007 561 571 (Pubitemid 44894455)
48. D.A. Slutsky, and G.H. Recanzone Temporal and spatial dependency of the ventriloquism effect Neuroreport 12 1 2001 7 10 (Pubitemid 32099017)
49. S. Soto-Faraco, J. Lyons, M. Gazzaniga, C. Spence, and A. Kingstone The ventriloquist in motion: illusory capture of dynamic information across sensory modalities Brain Res. Cogn. Brain Res. 14 1 2002 139 146 (Pubitemid 34626804)
50. S. Soto-Faraco, C. Spence, D. Lloyd, and A. Kingstone Moving multisensory research along: motion perception across sensory modalities Curr. Dir. Psychol. Sci. 13 2004 29 32 (Pubitemid 38389667)
51. C. Spence, D. Senkowski, and B. Röder Crossmodal processing Exp. Brain Res. 198 2-3 2009 107 111
52. B.E. Stein, and M.A. Meredith The Merging of the Senses 1993 MIT Press Cambridge, MA
53. J.J. Stekelenburg, and J. Vroomen Neural correlates of audiovisual motion capture Exp. Brain Res. 198 2-3 2009 383 390
54. T.Z. Strybel, and A. Vatakis A comparison of auditory and visual apparent motion presented individually and with crossmodal moving distractors Perception 33 9 2004 1033 1048 (Pubitemid 39558193)
55. W.A. Teder-Sälejärvi, F. Di Russo, J.J. McDonald, and S.A. Hillyard Effects of spatial congruity on audio-visual multimodal integration J. Cogn. Neurosci. 17 9 2005 1396 1409 (Pubitemid 41291963)
56. W.R. Thurlow, and C.E. Jack Certain determinants of the "ventriloquism effect" Percept. Mot. Skills 36 3 1973 1171 1184
57. M.P.S. To, B.C. Regan, D. Wood, and J.D. Mollon Vision out of the corner of the eye Vision Res. 51 1 2011 203 214
58. M.M. Van Wanrooij, P. Bremen, and A. John Van Opstal Acquired prior knowledge modulates audiovisual integration Eur. J. Neurosci. 31 10 2010 1763 1771
59. M.T. Wallace, G.E. Roberson, W.D. Hairston, B.E. Stein, J.W. Vaughan, and J.A. Schirillo Unifying multisensory signals across time and space Exp. Brain Res. 158 2 2004 252 258 (Pubitemid 39281121)
60. R.B. Welch, and D.H. Warren Immediate perceptual response to intersensory discrepancy Psychol. Bull. 88 3 1980 638 667
61. S. Werner, and U. Noppeney Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization Cereb. Cortex 20 8 2010 1829 1842
62. S. Werner, and U. Noppeney The contributions of transient and sustained response codes to audiovisual integration Cereb. Cortex 21 4 2011 920 931
63. W.W. Wilson, and W.E. O'Neill Auditory motion induces directionally dependent receptive field shifts in inferior colliculus neurons J. Neurophysiol. 79 4 1998 2040 2062 (Pubitemid 28183337)
64. S.M. Wuerger, M. Hofbauer, and G.F. Meyer The integration of auditory and visual motion signals at threshold Percept. Psychophys. 65 8 2003 1188 1196 (Pubitemid 38126209)