Direction of visual apparent motion driven by perceptual organization of cross-modal signals

Journal of Vision

Volumn 13, Issue 1, 2013, Pages

  Roseboom, Warrick ^a   Kawabe, Takahiro ^a   Nishida, Shin'ya ^a  

^a Nippon Telegraph and Telephone Corporation   (Japan)

Author keywords

Apparent motion; Cross modal hierarchy; Cross modal timing; Multisensory timing; Perceptual grouping; Perceptual organization; Perceptual segmentation; Temporal ventriloquism

Indexed keywords

ARTICLE; COMPARATIVE STUDY; HEARING; HUMAN; MOTION; MOVEMENT PERCEPTION; PHYSIOLOGY; TOUCH; VISION;

AUDITORY PERCEPTION; HUMANS; MOTION; MOTION PERCEPTION; TOUCH; VISUAL PERCEPTION;

MLCS; MLOWN;

EID: 84878367491     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/13.1.6     Document Type: Article

References (61)

1. Arnold, D. H., Clifford, C. W. G. & Wenderoth, P. (2001). Asynchronous processing in vision. Current Biology, 11, 596-600.
2. Arnold, D. H., Ong, Y., & Roseboom, W. (2009). Simple differential latencies modulate, but do not cause the flash-lag effect. Journal of Vision, 9(5):4, 1-8, http://www.journalofvision.org/content/9/5/4, doi:10.1167/9.5.4. [PubMed] [Article]
3. Arnold, D. H., Tear, M., Schindel, R., & Roseboom, W. (2010). Audio-visual speech cue combination. PLoS One, 5, e10217.
4. Battaglia, P. W., Jacobs, R. A., & Aslin, R. N. (2003). Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America, 20(7), 1391-1397.
5. Calvert, G. A., Spence, C., & Stein, B. E. (Eds.). (2004). The handbook of multisensory processing. Cambridge, MA: MIT Press.
6. Cook, L. A., & Van Valkenburg, D. L. (2009). Audiovisual organization and the temporal ventriloquism effect between grouped sequences: Evidence that unimodal grouping precedes cross-modal integration. Perception, 38(8), 1220-33.
7. Dawson, M. R. (1991). The how and why of what went where in apparent motion: Modeling solutions to the motion correspondence problem. Psychological Review, 98, 569-603.
8. Dennett, D. C., & Kinsbourne, M. (1992). Time and the observer: the where and when of consciousness in the brain. Behavioural Brain Science, 15, 183- 247.
9. Di Luca, M., Machulla, T. K., & Ernst, M. O. (2009). Recalibration of multisensory simultaneity: Crossmodal transfer coincides with a change in perceptual latency. Journal of Vision, 9(12):7, 1-16, http://www.journalofvision.org/content/9/12/7, doi:10. 1167/9.12.7. [PubMed] [Article]
10. Dixon, N. F., & Spitz, L. (1980). The detection of auditory visual desynchrony. Perception, 9, 719- 721.
11. Fendrich, R., & Corballis, P. M. (2001). The temporal cross-capture of audition and vision. Perception and Psychophysics, 63, 719-725.
12. Freeman, E., & Driver, J. (2008). Direction of visual apparent motion driven solely by timing of a static sound. Current Biology, 18(16), 1262-1266.
13. Fujisaki, W., Kitazawa, S., & Nishida, S. (2012). Multisensory timing. In B. E. Stein (Ed.), The new handbook of multisensory processes (pp. 301-317). Cambridge, MA: MIT Press.
14. Fujisaki, W., Shimojo, S., Kashino, M., & Nishida, S. (2004). Recalibration of audiovisual simultaneity. Nature Neuroscience, 7, 773-778.
15. Gepshtein, S., & Kubovy, M. (2007). The lawful perception of apparent motion. Journal of Vision, 7(8):9, 1-15, http://www.journalofvision.org/content/7/8/9, doi:10.1167/7.8.9. [PubMed] [Article]
16. Hein, E., & Cavanagh, P. (2012). Motion correspondence in the Ternus display shows feature bias in spatiotopic coordinates. Journal of Vision, 12(7):16, 1-14, http://www.journalofvision.org/content/XX/XX/16, doi:10.1167/12.7.16. [PubMed] [Article]
17. Hein, E., & Moore, C. M. (2012). Spatio-temporal priority revisited: The role of feature identity and similarity for object correspondence in apparent motion. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 975-988.
18. Hillis, J. M., Ernst, M. O., Banks, M. S. & Landy, M. S. (2002). Combining sensory information: mandatory fusion within, but not between, senses. Science, 298, 1627-1630.
19. Johnston, A., & Nishida, S. (2001). Time perception: Brain time or event time?. Current Biology, 11, R427-R430.
20. Kafaligonul, H., & Stoner, G. R. (2010). Auditory modulation of visual apparent motion with short spatial and temporal intervals. Journal of Vision, 10(12):31, 1-13, http://www.journalofvision.org/content/10/12/31, doi:10.1167/10.12.31. [PubMed] [Article]
21. Kafaligonul, H., & Stoner, G. R. (2012). Static sound timing alters sensitivity to low-level visual motion. Journal of Vision, 12(11):2, 1-9, http://www.journalofvision.org/content/12/11/2, doi:10.1167/12.11.2. [PubMed] [Article]
22. Keetels, M., Stekelenburg, J., & Vroomen, J. (2007). Auditory grouping occurs prior to intersensory pairing: Evidence from temporal ventriloquism. Experimental Brain Research, 180, 449-456.
23. Keetels, M., & Vroomen, J. (2005). The role of spatial disparity and hemifields in audio-visual temporal order judgments. Experimental Brain Research, 167, 635-640.
24. Keetels, M., & Vroomen, J. (2011a). No effect of synesthetic congruency on temporal ventriloquism. Attention, Perception, & Psychophysics, 73(1), 209- 18.
25. Keetels, M., & Vroomen, J. (2011b). Sound affects the speed of visual processing. Journal of Experimental Psychology: Human Perception and Performance, 37, 699-708.
26. King, A. J. (2005). Multisensory integration: strategies for synchronization. Current Biology, 15, R339- R341.
27. Klink, P. C., Montijn, J. S. & van Wezel, R. J. A. (2011). Crossmodal duration perception involves perceptual grouping, temporal ventriloquism, and variable internal clock rates. Attention, Perception, & Psychophysics, 73, 219-236.
28. Kolers, P. A. (1972). Aspects of motion perception. Oxford, UK: Pergamon.
29. Linares, D., & Ló pez-Moliner. (2006). Perceptual asynchrony between color and motion with a single direction change. Journal of Vision, 6(9):10, 974- 981, http://www.journalofvision.org/content/6/9/10, doi:10.1167/6.9.10. [PubMed] [Article]
30. Linares, D., & Ló pez-Moliner. (2007). Absence of flash-lag when judging global shape from local positions. Vision Research, 47, 357-362.
31. Miner, N., & Caudell, T. (1998). Computational requirements and synchronization issues of virtual acoustic displays. Presence: Teleoperators and Virtual Environments, 7, 396-409.
32. Morein-Zamir, S., Soto-Faraco, S., & Kingstone, A. (2003). Auditory capture of vision: examining temporal ventriloquism. Cognitive Brain Research, 17, 154-163.
33. Moutoussis, K., & Zeki, S. (1997). A direct demonstration of perceptual asynchrony in vision. Proceedings of the Royal Society of London B, 264, 393-399.
34. Navarra, J., Hartcher-O'Brien, J., Piazza, E., & Spence, C. (2009). Adaptation to audiovisual asynchrony modulates the speeded detection of sound. In Proceedings of the National Academy of Sciences of the United States of America, 106, 9169-9173.
35. Navarra, J., Vatakis, A., Zampini, M., Soto-Faraco, S.,Humphreys, W., & Spence, C. (2005). Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research, 25, 499-507.
36. Nicol, J. R., & Shore, D. I. (2007). Perceptual grouping impairs temporal resolution. Experimental Brain Research, 183(2), 141-148.
37. Nishida, S., & Johnston, A. (2002). Marker correspondence, not processing latency, determines temporal binding of visual attributes. Current Biology, 12(5), 359-368.
38. O'Leary, A., & Rhodes, G. (1984). Cross-modal effects on visual and auditory object perception. Perception Psychophysics, 35(6), 565-569.
39. Parise, C., & Spence, C. (2009). "When birds of a feather flock together": synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS One, 4, e5664.
40. Powers, R., Hillock, A. R., & Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. Journal of Neuroscience, 29, 12 265-12 274.
41. Roach, N. W., Heron, J., & McGraw, P. V. (2006). Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integration. Proceedings of the Royal Society of London B, 273, 2159-2168.
42. Roseboom, W., & Arnold, D. H. (2011). Twice upon a time: Multiple, concurrent, temporal recalibrations of audio-visual speech. Psychological Science, 22, 872-877.
43. Roseboom, W., Nishida, S., & Arnold, D. H. (2009). The sliding window of audio-visual simultaneity.Journal of Vision, 9(12):4, 1-8, http://www.org/content/9/12/4, doi:10.1167/9. 12.4. [PubMed] [Article]
44. Roseboom, W., Nishida, S., Fujisaki, W., & Arnold, D. H. (2011). Audio-visual speech timing sensitivity is enhanced in cluttered conditions. PLoS One, 6, e18309.
45. Sanabria, D., Soto-Faraco, S., Chan, J., & Spence, C. (2005). Intramodal perceptual grouping modulates multisensory integration: Evidence from the crossmodal dynamic capture task. Neuroscience Letters, 377(1), 59-64.
46. Scheier, C. R., Nijhawan, R., & Shimojo, S. (1999). Sound alters visual temporal resolution. Investigative Ophthalmology & Visual Science, 40, S792.
47. Shams, L., Kamitani, Y., & Shimojo, S. (2000). Illusions. What you see is what you hear. Nature, 408, 788.
48. Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. Neuroreport, 12, 7-10.
49. Spence, C., Baddeley, R., Zampini, M., James, R., & Shore, D. I. (2003). Multisensory temporal order judgments: When two locations are better than one. Perception Psychophysics, 65, 318-328.
50. Spence, C., & Chen, Y. C. (2012). Intramodal and cross-modal perceptual grouping. In B. E. Stein (Ed.), The new handbook of multisensory processes. Cambridge, MA: MIT Press (pp. 265-281).
51. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.
52. Ullman, S. (1979). The interpretation of visual motion. Cambridge, MA: MIT Press.
53. Vatakis, A., & Spence, C. (2006). Audiovisual synchrony perception for music, speech, and object actions. Brain Research, 1111, 134-142.
54. Vatakis, A., & Spence, C. (2007). Crossmodal binding: evaluating the "unity assumption" using audiovisual speech stimuli. Perception and Psychophysics, 69(5), 744-756.
55. Vroomen, J., & Keetels, M. (2006). The spatial constraint in intersensory pairing: No role in temporal ventriloquism. Journal of Experimental Psychology: Human Perception and Performance, 32, 1063-1071.
56. Vroomen, J., & Keetels, M. (2009). Sounds change four-dot masking. Acta Psychologica, 130, 58-63.
57. Vroomen, J., Keetels, M., de Gelder, B., & Bertelson, P. (2004). Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cognitive Brain Research, 22, 32-35.
58. Watanabe, K., & Shimojo, S. (2001). When sound affects vision: Effects of auditory grouping on visual motion perception. Psychological Science, 12, 109-116.
59. Whitney, D., & Murakami, I. (1998). Latency difference, not spatial extrapolation. Nature Neuroscience, 1, 656-657.
60. Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2005). Audio-visual simultaneity judgments. Perception & Psychophysics, 67, 531-544.
61. Zampini, M., Shore, D. I., & Spence, C. (2003). Audiovisual temporal order judgments. Experimental Brain Research, 152, 198-210.