Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli

Perception and Psychophysics

Volumn 69, Issue 5, 2007, Pages 744-756

  Vatakis, Argiro ^a   Spence, Charles ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34447097853     PISSN: 00315117     EISSN: None     Source Type: Journal    
DOI: 10.3758/BF03193776     Document Type: Article

References (70)

1. ABRY, C., CATHIARD, M. A., ROBERT-R IBES, J., & SCHWARTZ, J. L. (1994). The coherence of speech in audio-visual integration. Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 13, 52-59.
2. ALAIS, D., & BURR, D. (2004). The ventriloquist effect results from nearoptimal bimodal integration. Current Biology, 14, 257-262.
3. ALSIUS, A., NAVARRA, J., CAMPBELL, R., & SOTO-FARACO, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15, 1-5.
4. ARMEL, K. C., & RAMACHANDRAN, V. S. (2003). Projecting sensations to external objects: Evidence from skin conductance response. Proceedings of the Royal Society of London: Series B, 270, 1499-1506.
5. ARNOLD, D. H., JOHNSTON, A., & NISHIDA, S. (2005). Timing sight and sound. Vision Research, 45, 1275-1284.
6. BATTAGLIA, P. W, JACOBS, R. A., & ASLIN, R. N. (2003). Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society ofAmericaA, 20, 1391-1397.
7. BEDFORD, F. L. (1994). A pair of paradoxes and the perceptual pairing processes. Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 13, 60-68.
8. BEDFORD, F. L. (2001). Towards a general law of numerical/object identity. Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 20, 113-175.
9. BERMANT, R. I., & WELCH, R. B. (1976). Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition. Perceptual & Motor Skills, 42, 487-493.
10. BERTELSON, P., & ASCHERSLEBEN, G. (1998). Automatic visual bias of perceived auditory location. Psychonomic Bulletin & Review, 5, 482-489.
11. BERTELSON, P., & ASCHERSLEBEN, G. (2003). Temporal ventriloquism: Crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. International Journal of Psychophysiology, 50, 147-155.
12. BERTELSON, P., & DE GELDER, fi. (2004). The psychology of multimodal perception. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention (pp. 141-177). Oxford: Oxford University Press.
13. CACLIN, A., SOTO-FARACO, S., KINGSTONE, A., & SPENCE, C. (2002). Tactile "capture" of audition. Perception & Psychophysics, 64, 616-630.
14. CALVERT, G. A., SPENCE, C., & STEIN, B. E. (EDS.). (2004). The handbook of multisensory processing. Cambridge, MA: MIT Press.
15. CORBALLIS, M. C. (1994). Do you need module any more? Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 13, 81-83.
16. COREN, S., WARD, L. M., & ENNS, J. T. (2004). Sensation and perception (6th ed.). Fort Worth, TX: Harcourt Brace.
17. EASTON, R. D., & BASALA, M. (1982). Perceptual dominance during lipreading. Perception & Psychophysics, 32, 562-570.
18. EPSTEIN, W. (1975). Recalibration by pairing: A process of perceptual learning. Perception, 4, 59-72.
19. ERNST, M. O., & BANKS, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
20. FENDRICH, R., & CORBALLIS, P. M. (2001). The temporal cross-capture of audition and vision. Perception & Psychophysics, 63, 719-725.
21. FINNEY, D. J. (1964). Probit analysis: Statistical treatment of the sigmoid response curve. Cambridge: Cambridge University Press.
22. FISHER, B. D., & PYLYSHYN, Z. W. (1994). The cognitive architecture of bimodal event perception: A commentary and addendum to Radeau (1994). Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 13, 92-96.
23. GREEN, K. P., & GERDEMAN, A. (1995). Cross-modal discrepancies in coarticulation and the integration of speech information: The McGurk effect with mismatched vowels. Journal of Experimental Psychology: Human Perception & Performance, 21, 1409-1426.
24. GREEN, K. P., KUHL, P. K., MELTZOFF, A. N., & STEVENS, E. B. (1991). Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect. Perception & Psychophysics, 50, 524-536.
25. GUSKI, R., & TROJE, N. F. (2003). Audiovisual phenomenal causality. Perception & Psychophysics, 65, 789-800.
26. HAIRSTON, W. D., WALLACE, M. T., VAUGHAN, J. W., STEIN, B. E., NORRIS, J. L., & SCHIRILLO, J. A. (2003). Visual localization ability influences cross-modal bias. Journal of Cognitive Neuroscience, 15, 20-29.
27. HERON, J., WHITAKER, D., & MCGRAW, P. V. (2004). Sensory uncertainty governs the extent of audio-visual interaction. Vision Research, 44, 2875-2884.
28. JACK, C. E., & THURLOW, W. R. (1973). Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. Perceptual & Motor Skills, 37, 967-979.
29. JACKSON, C. V. (1953). Visual factors in auditory localization. Quarterly Journal of Experimental Psychology, 5, 52-65.
30. JONES, J. A., & JARICK, M. (2006). Multisensory integration of speech signals: The relationship between space and time. Experimental Brain Research, 17A, 588-594.
31. LAURIENTI, P. J., KRAFT, R. A., MALDJIAN, J. A., BURDETTE, J. H., & WALLACE, M. T. (2004). Semantic congruence is a critical factor in multisensory behavioral performance. Experimental Brain Research, 158, 405-414.
32. LAVIE, N. (2005). Distracted and confused?: Selective attention under load. Trends in Cognitive Sciences, 9, 75-82.
33. LYONS, G., SANABRIA, D., VATAKIS, A., & SPENCE, C. (2006). The modulation of crossmodal integration by unimodal perceptual grouping: A visuo-tactile apparent motion study. Experimental Brain Research, 174, 510-516.
34. MACDONALD, J., & MCGURK, H. (1978). Visual influences on speech perception processes. Perception & Psychophysics, 24, 253-257.
35. MCGRATH, M., & SUMMERFIELD, Q. (1985). Intermodal timing relations and audiovisual speech recognition by normal hearing adults. Journal of the Acoustical Society of America, 77, 678-685.
36. MCGURK, H., & MACDONALD, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
37. MICHOTTE, A. (1946). La Perception de la Causalité [The perception of causality]. Louvain: Institut Supérieur de Philosophie.
38. MILLER, E. A. (1972). Interactions of vision and touch in conflict and nonconflict form perception tasks. Journal of Experimental Psychology, 114-123.
39. MOREIN-ZAMIR, S., SOTO-FARACO, S., & KINGSTONE, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154-163.
40. MUNHALL, K. G., & VATIKIOTIS-BATESON, E. (2004). Spatial and temporal constraints on audiovisual speech perception. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), The handbook of multisensory processing (pp. 177-188). Cambridge, MA: MIT Press.
41. RADEAU, M. (1994). Auditory-visual spatial interaction and modularity. Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 13, 3-51.
42. RADEAU, M., & BERTELSON, P. (1977). Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations. Perception & Psychophysics, 22, 137-146.
43. RADEAU, M., & BERTELSON, P. (1987). Auditory-visual interaction and the timing of inputs. Thomas (1941) revisited. Psychological Research, 49, 17-22.
44. ROACH, N. W., HERON, J., & MCGRAW, P. V. (2006). Resolving multisensory conflict: A strategy for balancing the costs and benefits of audio-visual integration. Proceedings of the Royal Society of London: Series B, 273, 2159-2168.
45. ROSENBLUM, L. D., & SALDAÑA, H. M. (1992). Discrimination tests of visually influenced syllables. Perception & Psychophysics, 52, 461-473.
46. SANABRIA, D., SOTO-FARACO, S., CHAN, J. S., & SPENCE, C. (2004). When does visual perceptual grouping affect multisensory integration? Cognitive, Affective, & Behavioral Neuroscience, 4, 218-229.
47. SCHEIER, C. R., NUHAWAN, R., & SHIMOJO, S. (1999). Sound alters visual temporal resolution. Investigative Ophthalmology & Visual Science, 40, S792.
48. SHAW, M. L. (1980). Identifying attentional and decision-making components in information processing. In R. S. Nickerson (Ed.), Attention and performance VIII (pp. 277-296). Hillsdale, NJ: Erlbaum.
49. SHORE, D. I., SPENCE, C., & KLEIN, R. M. (2001). Visual prior entry. Psychological Science, 12, 205-212.
50. SLUTSKY, D. A., & RECANZONE, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. NeuroReport, 12, 7-10.
51. SOTO-FARACO, S., KINGSTONE, A., & SPENCE, C. (2003). Multisensory contributions to the perception of motion. Neuropsychologia, 41, 1847-1862.
52. SPENCE, C., SANABRIA, D., & SOTO- FARACO, S. (2007). Intersensory Gestalten: Assessing the influence of intramodal perceptual grouping on crossmodal interactions. In K. Noguchi (Ed.), The psychology of beauty and Kansei: New horizons of Gestalt perception (pp. 519-579). Tokyo: Fuzanbo International.
53. SPENCE, C., SHORE, D. L., & KLEIN, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799-832.
54. STEIN, B. E., & MEREDITH, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.
55. STONE, J. V., HUNKIN, N. M., PORRILL, J., WOOD, R., KEELER, V., BEANLAND. M., ET AL. (2001). When is now? Perception of simultaneity, Proceedings of the Royal Society of London: Series B, 268, 31-38.
56. SUGITA, Y., & SUZUKI, Y. (2003). Implicit estimation of sound-arrival time. Nature, 421, 911.
57. THOMAS, G. J. (1941). Experimental study of the influence of vision on sound localization. Journal of Experimental Psychology, 28, 163-177.
58. THURLOW, W. R., & JACK, C. E. (1973). Certain determinants of the 'Ventriloquism effect." Perceptual & Motor Skills, 36, 1171-1184.
59. VAN WASSENHOVE, V., GRANT, K. W., & POEPPEL, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, 102, 1181-1186.
60. VATAKIS, A., & SPENCE, C. (2006). Audiovisual synchrony perception for speech and music using a temporal order judgment task. Neuroscience Letters, 393, 40-44.
61. VROOMEN, J. (1999). Ventriloquism and the nature of the unity assumption. In G. Aschersleben, T. Bachmann, & J. Müsseier (Eds.), Cognitive contributions to the perception of spatial and temporal events (pp. 389-393). Amsterdam: Elsevier.
62. VROOMEN, J., & DE GELDER, B. (2000). Sound enhances visual perception: Cross-modal effects of auditory organization on vision. Journal of Experimental Psychology: Human Perception & Performance, 26, 1583-1590.
63. VROOMEN, J., & KEETELS, M. (2006). The spatial constraint in intersensory pairing: No role in temporal ventriloquism. Journal of Experimental Psychology: Human Perception & Performance, 32, 1063-1071.
64. WALKER, S., BRUCE, V., & O'MALLEY, C .(1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect Perception & Psychophysics, 57, 1124-1133.
65. WARREN, D. H., WELCH, R. B., & MCCARTHY, T. J. (1981). The role of visual-auditory "compellingness" in the ventriloquism effect: Implications for transitivity among the spatial senses. Perception & Psychophysics, 30, 557-564.
66. WEERTS, T. C., & THURLOW, W. R. (1971). The effects of eye position and expectation on sound localization. Perception & Psychophysics, 9, 35-39.
67. WELCH, R. B. (1972). The effect of experienced limb identity upon adaptation to simulated displacement of the visual field. Perception & Psychophysics, 12, 453-456.
68. WELCH, R. B. (1999). Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions. In G. Aschersleben, T. Bachmann, & J. Müsseier (Eds.), Cognitive contributions to the perception of spatial and temporal events (pp. 371-387). Amsterdam: Elsevier.
69. WELCH, R. B.,&WARREN,D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638-667.
70. WITKIN, H. A., WAPNER, S., & LEVENTHAL, T. (1952). Sound localization with conflicting visual and auditory cues. Journal of Experimental Psychology, 43, 58-67.