Assessing the role of the 'unity assumption' on multisensory integration: A review

Frontiers in Psychology

Volumn 8, Issue MAR, 2017, Pages

  Chen, Yi Chuan ^a   Spence, Charles ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Coupling priors; Crossmodal correspondences; Semantic congruency; The unity assumption; The unity effect

Indexed keywords

EID: 85016571429     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2017.00445     Document Type: Review

References (177)

1. Adam, R., and Noppeney, U. (2010). Prior auditory information shapes visual category-selectivity in ventral occipito-temporal cortex. NeuroImage 52, 1592-1602. doi: 10.1016/j.neuroimage.2010.05.002
2. Alais, D., and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257-262. doi: 10.1016/j.cub.2004.01.029
3. Alais, D., and Carlile, S. (2005). Synchronizing to real events: Subjective audiovisual alignment scales with perceived auditory depth and speed of sound. Proc. Natl. Acad. Sci. U.S.A. 102, 2244-2247. doi: 10.1073/pnas.0407034102
4. Arnold, D. H., Johnston, A., and Nishida, S. (2005). Timing sight and sound. Vis. Res. 45, 1275-1284. doi: 10.1016/j.visres.2004.11.014
5. Bar, M. (2007). The proactive brain: using analogies and associations to generate predictions. Trends Cogn. Sci. 11, 280-289. doi: 10.1016/j.tics.2007.05.005
6. Baumgartner, H. A., and Oakes, L. M. (2011). Infants' developing sensitivity to object function: attention to features and feature correlations. J. Cogn. Dev. 12, 275-298. doi: 10.1080/15248372.2010.542217
7. Bedford, F. L. (2001). Towards a general law of numerical/object identity. Curr. Psychol. Cogn. 20, 113-175
8. Beierholm, U. R., Quartz, S. R., and Shams, L. (2009). Bayesian priors are encoded independently from likelihoods in human multisensory perception. J. Vis. 9, 23.1-9. doi: 10.1167/9.5.23
9. Bertelson, P., and Aschersleben, G. (1998). Automatic visual bias of perceived auditory location. Psychon. Bull. Rev. 5, 482-489. doi: 10.3758/BF03208826
10. Bertelson, P., and Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 29, 578-584. doi: 10.3758/BF03207374
11. Bowers, J. S., and Davis, C. J. (2012). Bayesian just-so stories in psychology and neuroscience. Psychol. Bull. 138, 389-414. doi: 10.1037/a0026450
12. Bremner, A., Caparos, S., Davidoff, J., de Fockert, J., Linnell, K., and Spence, C. (2013). Bouba and Kiki in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners. Cognition 126, 165-172. doi: 10.1016/j.cognition.2012.09.007
13. Bremner, A., Lewkowicz, D., and Spence, C. (eds) (2012). Multisensory Development. Oxford: Oxford University Press
14. Bruner, J. S. (1957). On perceptual readiness. Psychol. Rev. 64, 123-152. doi: 10.1037/h0043805
15. Busse, L., Roberts, K. C., Crist, R. E., Weissman, D. H., and Woldorff, M. G. (2005). The spread of attention across modalities and space in a multisensory object. Proc. Natl. Acad. Sci. U.S.A. 102, 18751-18756. doi: 10.1073/pnas.0507704102
16. Calvert, G. A., Spence, C., and Stein, B. E. (eds) (2004). The Handbook of Multisensory Processing. Cambridge, MA: MIT Press
17. Campanella, S., and Belin, P. (2007). Integrating face and voice in person perception. Trends Cogn. Sci. 11, 535-543. doi: 10.1016/j.tics.2007.10.001
18. Charbonneau, G., Véronneau, M., Boudrias-Fournier, C., Lepore, F., and Collignon, O. (2013). The ventriloquist in periphery: impact of eccentricity-related reliability on audio-visual localization. J. Vis. 13:20. doi: 10.1167/13.12.20
19. Chen, L., and Vroomen, J. (2013). Intersensory binding across space and time: a tutorial review. Attent. Percept. Psychophys. 75, 790-811. doi: 10.3758/s13414-013-0475-4
20. Chen, Y.-C., Huang, P.-C., Woods, A., and Spence, C. (2016). When "Bouba" equals "Kiki": cultural commonalities and cultural differences in sound-shape correspondences. Sci. Rep. 27:26681. doi: 10.1038/srep26681
21. Chen, Y.-C., and Spence, C. (2010). When hearing the bark helps to identify the dog: semantically-congruent sounds modulate the identification of masked pictures. Cognition 114, 389-404. doi: 10.1016/j.cognition.2009.10.012
22. Chen, Y.-C., and Spence, C. (2011a). Crossmodal semantic priming by naturalistic sounds and spoken words enhances visual sensitivity. J. Exp. Psychol. 37, 1554-1568. doi: 10.1037/a0024329
23. Chen, Y.-C., and Spence, C. (2011b). The crossmodal facilitation of visual object representations by sound: evidence from the backward masking paradigm. J. Exp. Psychol. 37, 1784-1802. doi: 10.1037/a0025638
24. Chen, Y.-C., and Spence, C. (2013). The time-course of the cross-modal semantic modulation of visual picture processing by naturalistic sounds and spoken words. Multisens. Res. 26, 371-386
25. Choe, C. S., Welch, R. B., Gilford, R. M., and Juola, J. F. (1975). The "ventriloquist effect": visual dominance or response bias? Percep. Psychophys. 18, 55-60. doi: 10.3758/BF03199367
26. Colavita, F. B. (1974). Human sensory dominance. Percept. Psychophys. 16, 409-412. doi: 10.3758/BF03203962
27. Colin, C., Radeau, M., Deltenre, P., and Morais, J. (2001). Rules of intersensory integration in spatial scene analysis and speechreading. Psychol. Belg. 41, 131-144
28. Connolly, K. (2014). Multisensory perception as an associative learning process. Front. Psychol. 5:1095. doi: 10.3389/fpsyg.2014.01095
29. de Gelder, B., and Vroomen, J. (2000). The perception of emotions by ear and by eye. Cogn. Emot. 14, 289-311. doi: 10.1080/026999300378824
30. Deroy, O. (2013). "The unity assumption and the many unities of consciousness," in Sensory Integration and the Unity of Consciousness, eds D. J. Bennett and C. S. Hill (Cambridge, MA: MIT Press), 105-124
31. Doehrmann, O., and Naumer, M. J. (2008). Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res. 1242, 136-150. doi: 10.1016/j.brainres.2008.03.071
32. Easton, R. D., and Basala, M. (1982). Perceptual dominance during lipreading. Percept. Psychophys. 32, 562-570. doi: 10.3758/BF03204211
33. Edmiston, P., and Lupyan, G. (2015). What makes words special? Words as unmotivated cues. Cognition 143, 93-100. doi: 10.1016/j.cognition.2015.06.008
34. Engel, G. R., and Dougherty, W. G. (1971). Visual-auditory distance constancy. Nature 234, 308. doi: 10.1038/234308a0
35. Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. J. Vis. 7, 7. 1-14. doi: 10.1167/7.5.7
36. Fairhall, S. L., and Macaluso, E. (2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. Eur. J. Neurosci. 29, 1247-1257. doi: 10.1111/j.1460-9568.2009.06688.x
37. Faivre, N., Mudrik, L., Schwartz, N., and Koch, C. (2014). Multisensory integration in complete unawareness: evidence from audiovisual congruency priming. Psychol. Sci. 25, 2006-2016. doi: 10.1177/0956797614547916
38. Falchier, A., Clavagnier, S., Barone, P., and Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749-5759
39. Fiebelkorn, I. C., Foxe, J. J., and Molholm, S. (2010). Dual mechanisms for the cross-sensory spread of attention: how much do learned associations matter? Cereb. Cortex 20, 109-120. doi: 10.1093/cercor/bhp083
40. Fiebelkorn, I. C., Foxe, J. J., and Molholm, S. (2012). "Attention and multisensory feature integration," in The New Handbook of Multisensory Processing, ed. B. E. Stein (Cambridge, MA: MIT Press), 383-394
41. Firestone, C., and Scholl, B. J. (2016). Cognition does not affect perception: evaluating the evidence for 'top-down'effects. Behav. Brain Sci. 39, e229. doi: 10.1017/S0140525X15000965
42. Föcker, J., Gondan, M., and Röder, B. (2011). Preattentive processing of audio-visual emotional signals. Acta Psychol. 137, 36-47. doi: 10.1016/j.actpsy.2011.02.004
43. Fodor, J. A. (1983). The Modularity of Mind: An Essay on Faculty Psychology. Cambridge, MA: MIT Press
44. Francis, G. (2012). The same old new look: publication bias in a study of wishful seeing. i-Perception 3, 176-178. doi: 10.1068/i0519ic
45. Friston, K. (2005). A theory of cortical responses. Philos. Trans. R. Soc. B 360, 815-836. doi: 10.1098/rstb.2005.1622
46. Friston, K. J., Bastos, A. M., Pinotsis, D., and Litvak, V. (2015). LFP and oscillations-What do they tell us? Curr. Opin. Neurobiol. 31, 1-6. doi: 10.1016/j.conb.2014.05.004
47. Fujisaki, W., Shimojo, S., Kashino, M., and Nishida, S. Y. (2004). Recalibration of audiovisual simultaneity. Nat. Neurosci. 7, 773-778. doi: 10.1038/nn1268
48. Gallace, A., and Spence, C. (2006). Multisensory synesthetic interactions in the speeded classification of visual size. Percept. Psychophys. 68, 1191-1203. doi: 10.3758/BF03193720
49. García-Pérez, M. A., and Alcalá-Quintana, R. (2012). Response errors explain the failure of independent-channels models of perception of temporal order. Front. Psychol. 3:94. doi: 10.3389/fpsyg.2012.00094
50. Gau, R., and Noppeney, U. (2016). How prior expectations shape multisensory perception. NeuroImage 124, 876-886. doi: 10.1016/j.neuroimage.2015.09.045
51. Gleiss, S., and Kayser, C. (2013). Eccentricity dependent auditory enhancement of visual stimulus detection but not discrimination. Front. Integr. Neurosci. 7:52. doi: 10.3389/fnint.2013.00052
52. Gordon, R. D., and Irwin, D. E. (1996). What's in an object file? Evidence from priming studies. Percept. Psychophys. 58, 1260-1277. doi: 10.3758/BF03207558
53. Green, K., Kuhl, P., Meltzoff, A., and Stevens, E. (1991). Integrating speech information across talkers, gender, and sensory modality: female faces and male voices in the McGurk effect. Percept. Psychophys. 50, 524-536. doi: 10.3758/BF03207536
54. Hartcher-O'Brien, J., Gallace, A., Krings, B., Koppen, C., and Spence, C. (2008). When vision 'extinguishes' touch in neurologically-normal people: extending the Colavita visual dominance effect. Exp. Brain Res. 186, 643-658. doi: 10.1007/s00221-008-1272-5
55. Hein, G., Doehrmann, O., Müller, N. G., Kaiser, J., Muckli, L., and Naumer, M. J. (2007). Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J. Neurosci. 27, 7881-7887. doi: 10.1523/JNEUROSCI.1740-07.2007
56. Helbig, H. B., and Ernst, M. O. (2007). Knowledge about a common source can promote visual-haptic integration. Perception 36, 1523-1533. doi: 10.1068/p5851
57. Helbig, H. B., and Ernst, M. O. (2008). Visual-haptic cue weighting is independent of modality-specific attention. J. Vis. 8, 21. 1-16. doi: 10.1167/8.1.21
58. Jackson, C. V. (1953). Visual factors in auditory localization. Q. J. Exp. Psychol. 5, 52-65. doi: 10.1080/17470215308416626
59. Jones, J. A., and Jarick, M. (2006). Multisensory integration of speech signals: the relationship between space and time. Exp. Brain Res. 174, 588-594. doi: 10.1007/s00221-006-0634-0
60. Jones, J. A., and Munhall, K. G. (1997). Effects of separating auditory and visual sources on audiovisual integration of speech. Canad. Acoust. 25, 13-19
61. Jones, M., and Love, B. C. (2011). Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behav. Brain Sci. 34, 169-231. doi: 10.1017/S0140525X10003134
62. Kanaya, S., and Yokosawa, K. (2011). Perceptual congruency of audio-visual speech affects ventriloquism with bilateral visual stimuli. Psychon. Bull. Rev. 18, 123-128. doi: 10.3758/s13423-010-0027-z
63. Keetels, M., and Vroomen, J. (2011). No effect of synesthetic congruency on temporal ventriloquism. Attent. Percept. Psychophys. 73, 209-218. doi: 10.3758/s13414-010-0019-0
64. Klapetek, A., Ngo, M. K., and Spence, C. (2012). Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search? Attent. Percept. Psychophys. 74, 1154-1167. doi: 10.3758/s13414-012-0317-9
65. Kok, P., and de Lange, F. P. (2015). "Predictive coding in sensory cortex," in An Introduction to Model-Based Cognitive Neuroscience, eds B. U. Forstmann and E.-J. Wagenmakers (New York, NY: Springer), 221-244
66. Koppen, C., Alsius, A., and Spence, C. (2008). Semantic congruency and the Colavita visual dominance effect. Exp. Brain Res. 184, 533-546. doi: 10.1007/s00221-007-1120-z
67. Koppen, C., and Spence, C. (2007a). Audiovisual asynchrony modulates the Colavita visual dominance effect. Brain Res. 1186, 224-232
68. Koppen, C., and Spence, C. (2007b). Spatial coincidence modulates the Colavita visual dominance effect. Neurosci. Lett. 417, 107-111
69. Körding, K. P., Beierholm, U., Ma, W. J., Tenenbaum, J. B., Quartz, S., and Shams, L. (2007). Causal inference in multisensory perception. PLoS ONE 2:e943. doi: 10.1371/journal.pone.0000943
70. Kubovy, M., and van Valkenburg, D. (2001). Auditory and visual objects. Cognition 80, 97-126. doi: 10.1016/S0010-0277(00)00155-4
71. Lacey, S., and Sathian, K. (2014). Visuo-haptic multisensory object recognition, categorization, and representation. Front. Psychol. 5:730. doi: 10.3389/fpsyg.2014.00730
72. Laurienti, P. J., Kraft, R. A., Maldjian, J. A., Burdette, J. H., and Wallace, M. T. (2004). Semantic congruence is a critical factor in multisensory behavioral performance. Exp. Brain Res. 158, 405-414. doi: 10.1007/s00221-004-1913-2
73. Lee, H., and Noppeney, U. (2014). Music expertise shapes audiovisual temporal integration windows for speech, sinewave speech, and music. Front. Psychol. 5:868. doi: 10.3389/fpsyg.2014.00868
74. Lewkowicz, D. J., Leo, I., and Simion, F. (2010). Intersensory perception at birth: newborns match nonhuman primate faces and voices. Infancy 15, 46-60. doi: 10.1111/j.1532-7078.2009.00005.x
75. Lippert, M., Logothetis, N. K., and Kayser, C. (2007). Improvement of visual contrast detection by a simultaneous sound. Brain Res. 1173, 102-109. doi: 10.1016/j.brainres.2007.07.050
76. Liu, B., Wu, G., and Meng, X. (2012). Cross-modal priming effect based on short-term experience of ecologically unrelated audio-visual information: an event-related potential study. Neuroscience 223, 21-27. doi: 10.1016/j.neuroscience.2012.06.009
77. Love, S. A., Petrini, K., Cheng, A., and Pollick, F. E. (2013). A psychophysical investigation of differences between synchrony and temporal order judgments. PLoS ONE 8:e54798. doi: 10.1371/journal.pone.0054798
78. Macpherson, F. (2012). Cognitive penetration of colour experience: rethinking the issue in light of an indirect mechanism. Philos. Phenomenol. Res. 84, 24-62. doi: 10.1111/j.1933-1592.2010.00481.x
79. Macpherson, F. (2017). The relationship between cognitive penetration and predictive coding. Conscious. Cogn. 47, 6-16. doi: 10.1016/j.concog.2016.04.001
80. Maiworm, M., Bellantoni, M., Spence, C., and Röder, B. (2012). When emotional valence affects audiovisual integration. Attent. Percept. Psychophys. 74, 1302-1311. doi: 10.3758/s13414-012-0310-3
81. Massaro, D. W., and Cohen, M. M. (1993). Perceiving asynchronous bimodal speech in consonant-vowel and vowel syllables. Speech Commun. 13, 127-134. doi: 10.1016/0167-6393(93)90064-R
82. Massaro, D. W., and Cohen, M. M. (2000). Fuzzy logical model of bimodal emotion perception: comment on "The perception of emotions by ear and by eye" by de Gelder and Vroomen. Cogn. Emot. 14, 313-320. doi: 10.1080/026999300378833
83. Massaro, D. W., and Egan, P. B. (1996). Perceiving affect from the voice and the face. Psychon. Bull. Rev. 3, 215-221. doi: 10.3758/BF03212421
84. McGurk, H., and MacDonald, J. (1976). Hearing lips and seeing voices. Nature 264, 746-748. doi: 10.1038/264746a0
85. Miller, E. A. (1972). Interaction of vision and touch in conflict and nonconflict form perception tasks. J. Exp. Psychol. 96, 114-123. doi: 10.1037/h0033468
86. Miller, J. O. (1991). Channel interaction and the redundant targets effect in bimodal divided attention. J. Exp. Psychol. 17, 160-169. doi: 10.1037/0096-1523.17.1.160
87. Misceo, G. F., Jackson, S. V., and Perdue, J. R. (2014). Again, knowledge of common source fails to promote visual-haptic integration. Percept. Motor Skills 118, 183-194. doi: 10.2466/24.23.PMS.118k11w0
88. Misceo, G. F., and Taylor, N. J. (2011). A "unity assumption" does not promote intersensory integration. Exp. Psychol. 58, 385-390. doi: 10.1027/1618-3169/a000107
89. Mitterer, H., and Jesse, A. (2010). Correlation versus causation in multisensory perception. Psychon. Bull. Rev. 17, 329-334. doi: 10.3758/PBR.17.3.329
90. Molholm, S., Ritter, W., Javitt, D. C., and Foxe, J. J. (2004). Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb. Cortex 14, 452-465. doi: 10.1093/cercor/bhh007
91. Morein-Zamir, S., Soto-Faraco, S., and Kingstone, A. (2003). Auditory capture of vision: examining temporal ventriloquism. Cogn. Brain Res. 17, 154-163. doi: 10.1016/S0926-6410(03)00089-2
92. Morrongiello, B. A., Fenwick, K. D., and Chance, G. (1998). Crossmodal learning in newborn infants: inferences about properties of auditory-visual events. Infant Behav. Dev. 21, 543-553. doi: 10.1016/S0163-6383(98)90028-5
93. Mozolic, J. L., Hugenschmidt, C. E., Peiffer, A. M., and Laurienti, P. J. (2008). Modality-specific selective attention attenuates multisensory integration. Exp. Brain Res. 184, 39-52. doi: 10.1007/s00221-007-1080-3
94. Munhall, K. G., Gribble, P., Sacco, L., and Ward, M. (1996). Temporal constraints on the McGurk effect. Percept. Psychophys. 58, 351-362. doi: 10.3758/BF03206811
95. Murphy, G. L. (1993). "A rational theory of concepts," in Categorization by Humans and Machines: Advances in Research and Theory, eds G. V. Nakamura, D. L. Medin, and R. Taraban (San Diego, CA: Academic Press), 327-359. doi: 10.1016/S0079-7421(08)60144-9
96. Murray, M. M., Lewkowicz, D. J., Amedi, A., and Wallace, M. T. (2016). Multisensory processes: a balancing act across the lifespan. Trends Neurosci. 39, 567-579. doi: 10.1016/j.tins.2016.05.003
97. Nahorna, O., Berthommier, F., and Schwartz, J. L. (2012). Binding and unbinding the auditory and visual streams in the McGurk effect. J. Acoust. Soc. Am. 132, 1061-1077. doi: 10.1121/1.4728187
98. Nahorna, O., Berthommier, F., and Schwartz, J. L. (2015). Audio-visual speech scene analysis: characterization of the dynamics of unbinding and rebinding the McGurk effect. J. Acoust. Soc. Am. 137, 362-377. doi: 10.1121/1.4904536
99. Naumer, M. J., and Kaiser, J. (eds) (2010). Multisensory Object Perception in the Primate Brain. New York, NY: Springer
100. Noesselt, T., Tyll, S., Boehler, C. N., Budinger, E., Heinze, H. J., and Driver, J. (2010). Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J. Neurosci. 30, 13609-13623. doi: 10.1523/JNEUROSCI.4524-09.2010
101. Odegaard, B., and Shams, L. (2016). The brain's tendency to bind audiovisual signals is stable but not general. Psychol. Sci. 27, 583-591. doi: 10.1177/0956797616628860
102. Odegaard, B., Wozny, D. R., and Shams, L. (2016). The effects of selective and divided attention on sensory precision and integration. Neurosci. Lett. 614, 24-28. doi: 10.1016/j.neulet.2015.12.039
103. Parise, C., and Spence, C. (2008). Synaesthetic congruency modulates the temporal ventriloquism effect. Neurosci. Lett. 442, 257-261. doi: 10.1016/j.neulet.2008.07.010
104. Parise, C., and Spence, C. (2009). 'When birds of a feather flock together': synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS ONE 4:e5664. doi: 10.1371/journal.pone.0005664
105. Parise, C. V., Harrar, V., Ernst, M. O., and Spence, C. (2013). Cross-correlation between auditory and visual signals promotes multisensory integration. Multisens. Res. 26, 307-316. doi: 10.1163/22134808-00002417
106. Parise, C. V., and Spence, C. (2013). "Audiovisual crossmodal correspondences," in The Oxford Handbook of Synesthesia, eds J. Simner and E. M. Hubbard (Oxford: Oxford University Press), 790-815
107. Parise, C. V., Spence, C., and Ernst, M. (2012). When correlation implies causation in multisensory integration. Curr. Biol. 22, 46-49. doi: 10.1016/j.cub.2011.11.039
108. Petrini, K., McAleer, P., and Pollick, F. (2010). Audiovisual integration of emotional signals from music improvisation does not depend on temporal correspondence. Brain Res. 1323, 139-148. doi: 10.1016/j.brainres.2010.02.012
109. Petrini, K., Russell, M., and Pollick, F. (2009). When knowing can replace seeing in audiovisual integration of actions. Cognition 110, 432-439. doi: 10.1016/j.cognition.2008.11.015
110. Powers, A. R., Hillock, A. R., and Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29, 12265-12274. doi: 10.1523/JNEUROSCI.3501-09.2009
111. Pylyshyn, Z. (1999). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. Behav. Brain Sci. 22, 341-365. doi: 10.1017/S0140525X99002022
112. Radeau, M., and Bertelson, P. (1974). The aftereffects of ventriloquism. Q. J. Exp. Psychol. 26, 63-71. doi: 10.1080/14640747408400388
113. Radeau, M., and Bertelson, P. (1977). Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations. Percept. Psychophys. 22, 137-146. doi: 10.3758/BF03198746
114. Radeau, M., and Bertelson, P. (1978). Cognitive factors and adaptation to auditory-visual discordance. Percept. Psychophys. 23, 341-343. doi: 10.3758/BF03199719
115. Ramachandran, V. S., and Hubbard, E. M. (2001). Synaesthesia-A window into perception, thought and language. J. Conscious. Stud. 8, 3-34
116. Rock, I., and Victor, J. (1964). Vision and touch: an experimentally created conflict between the two senses. Science 143, 594-596. doi: 10.1126/science.143.3606.594
117. Rockland, K. S., and Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 50, 19-26. doi: 10.1016/S0167-8760(03)00121-1
118. Rohe, T., and Noppeney, U. (2015a). Cortical hierarchies perform Bayesian causal inference in multisensory perception. PLoS Biol. 13:e1002073. doi: 10.1371/journal.pbio.1002073
119. Rohe, T., and Noppeney, U. (2015b). Sensory reliability shapes perceptual inference via two mechanisms. J. Vis. 15:22. doi: 10.1167/15.5.22
120. Rohe, T., and Noppeney, U. (2016). Distinct computational principles govern multisensory integration in primary sensory and association cortices. Curr. Biol. 26, 509-514. doi: 10.1016/j.cub.2015.12.056
121. Rosenblum, L. D. (2008). Speech perception as a multimodal phenomenon. Curr. Dir. Psychol. Sci. 17, 405-409. doi: 10.1111/j.1467-8721.2008.00615.x
122. Saldaña, H. M., and Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgments. Percep. Psychophys. 54, 406-416. doi: 10.3758/BF03205276
123. Sato, Y., Toyoizumi, T., and Aihara, K. (2007). Bayesian inference explains perception of unity and ventriloquism aftereffect: identification of common sources of audiovisual stimuli. Neural Comput. 19, 3335-3355. doi: 10.1162/neco.2007.19.12.3335
124. Schutz, M., and Kubovy, M. (2009). Causality and cross-modal integration. J. Exp. Psychol. 35, 1791-1810. doi: 10.1037/a0016455
125. Senkowski, D., Schneider, T. R., Foxe, J. J., and Engel, A. K. (2008). Crossmodal binding through neural coherence: implications for multisensory processing. Trends Neurosci. 31, 401-409. doi: 10.1016/j.tins.2008.05.002
126. Shams, L., and Beierholm, U. R. (2010). Causal inference in perception. Trends Cogn. Sci. 14, 425-432. doi: 10.1016/j.tics.2010.07.001
127. Shams, L., and Beierholm, U. R. (2011). "Humans' multisensory perception, from integration to segregation, follows Bayesian inference," in Sensory Cue Integration, eds J. Trommershäuser, K. Körding, and M. S. Landy (Oxford: Oxford University Press), 251-262
128. Shams, L., Kamitani, Y., and Shimojo, S. (2000). What you see is what you hear. Nature 408, 788. doi: 10.1038/35048669
129. Shams, L., Ma, W. J., and Beierholm, U. (2005). Sound-induced flash illusion as an optimal percept. Neuroreport 16, 1923-1927. doi: 10.1097/01.wnr.0000187634.68504.bb
130. Snodgrass, J. G., and Vanderwart, M. (1980). A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J. Exp. Psychol. 6, 174-215. doi: 10.1037/0278-7393.6.2.174
131. Soto-Faraco, S., and Alsius, A. (2009). Deconstructing the McGurk-MacDonald illusion. J. Exp. Psychol. 35, 580-587. doi: 10.1037/a0013483
132. Spence, C. (2007). Audiovisual multisensory integration. Acoust. Sci. Technol. 28, 61-70. doi: 10.1250/ast.28.61
133. Spence, C. (2011). Crossmodal correspondences: a tutorial review. Attent. Percept. Psychophys. 73, 971-995. doi: 10.3758/s13414-010-0073-7
134. Spence, C. (2013). Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule. Ann. N. Y. Acad. Sci. 1296, 31-49. doi: 10.1111/nyas.12121
135. Spence, C., Chen, Y.-C., and Deroy, O. (2014). "Do we perceive multisensory objects?" in Presentation at the Constructing Perceptual Objects: Binding within and Across the Senses. Rethinking the Senses Workshop, London
136. Spence, C., Parise, C., and Chen, Y.-C. (2011). "The Colavita visual dominance effect," in Frontiers in the Neural Bases of Multisensory Processes, eds M. M. Murray and M. Wallace (Boca Raton, FL: CRC Press), 523-550
137. Stein, B. E. (ed.) (2012). The New Handbook of Multisensory Processing. Cambridge, MA: MIT Press
138. Stein, B. E., and Meredith, M. A. (1993). The Merging of the Senses. Cambridge, MA: MIT Press
139. Stekelenburg, J. J., and Keetels, M. (2015). The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect. Exp. Brain Res. 234, 1209-1219. doi: 10.1007/s00221-015-4363-0
140. Stevenson, R. J., and Boakes, R. A. (2004). "Sweet and sour smells: learned synaesthesia between the senses of taste and smell," in The Handbook of Multisensory Processing, eds G. A. Calvert, C. Spence, and B. E. Stein (Cambridge, MA: MIT Press), 69-83
141. Sugita, Y., and Suzuki, Y. (2003). Implicit estimation of sound-arrival time. Nature 421, 911. doi: 10.1038/421911a
142. Talsma, D., Doty, T. J., and Woldorff, M. G. (2007). Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb. Cortex 17, 679-690. doi: 10.1093/cercor/bhk016
143. Talsma, D., Senkowski, D., Soto-Faraco, S., and Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 14, 400-410. doi: 10.1016/j.tics.2010.06.008
144. Taylor, K. I., Moss, H. E., Stamatakis, E. A., and Tyler, L. K. (2006). Binding crossmodal object features in perirhinal cortex. Proc. Natl. Acad. Sci. U.S.A. 103, 8239-8244. doi: 10.1073/pnas.0509704103
145. Thomas, G. J. (1941). Experimental study of the influence of vision on sound localization. J. Exp. Psychol. 28, 163-177. doi: 10.1037/h0055183
146. Tuomainen, J., Andersen, T. S., Tiippana, K., and Sams, M. (2005). Audio-visual speech perception is special. Cognition 96, B13-B22. doi: 10.1016/j.cognition.2004.10.004
147. Turatto, M., Mazza, V., and Umiltà, C. (2005). Crossmodal object-based attention: auditory objects affect visual processing. Cognition 96, B55-B64. doi: 10.1016/j.cognition.2004.12.001
148. van Atteveldt, N., Murray, M. M., Thut, G., and Schroeder, C. E. (2014). Multisensory integration: flexible use of general operations. Neuron 81, 1240-1253. doi: 10.1016/j.neuron.2014.02.044
149. van der Burg, E., Alais, D., and Cass, J. (2013). Rapid recalibration to audiovisual asynchrony. J. Neurosci. 33, 14633-14637. doi: 10.1523/JNEUROSCI.1182-13.2013
150. van der Burg, E., Orchard-Mills, E., and Alais, D. (2015). Rapid temporal recalibration is unique to audiovisual stimuli. Exp. Brain Res. 233, 53-59. doi: 10.1007/s00221-014-4085-8
151. van Eijk, R. L., Kohlrausch, A., Juola, J. F., and van de Par, S. (2008). Audiovisual synchrony and temporal order judgments: effects of experimental method and stimulus type. Percept. Psychophys. 70, 955-968. doi: 10.3758/PP.70.6.955
152. van Wassenhove, V., Grant, K. W., and Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45, 598-607. doi: 10.1016/j.neuropsychologia.2006.01.001
153. Vatakis, A., Ghazanfar, A., and Spence, C. (2008). Facilitation of multisensory integration by the 'unity assumption': is speech special? J. Vision, 8, 14. 1-11. doi: 10.1167/8.9.14
154. Vatakis, A., and Spence, C. (2007). Crossmodal binding: evaluating the "unity assumption" using audiovisual speech stimuli. Percep. Psychophys. 69, 744-756. doi: 10.3758/BF03193776
155. Vatakis, A., and Spence, C. (2008). Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. Acta Psychol. 127, 12-23. doi: 10.1016/j.actpsy.2006.12.002
156. Vercillo, T., and Gori, M. (2015). Attention to sound improves auditory reliability in audio-tactile spatial optimal integration. Front. Integr. Neurosci. 9:34. doi: 10.3389/fnint.2015.00034
157. Vetter, P., and Newen, A. (2014). Varieties of cognitive penetration in visual perception. Conscious. Cogn. 27, 62-75. doi: 10.1016/j.concog.2014.04.007
158. Vroomen, J. (1999). "Ventriloquism and the nature of the unity assumption," in Cognitive Contributions to the Perception of Spatial and Temporal Events, eds G. Ashersleben, T. Bachmann, and J. Müsseler (Amsterdam: Elsevier Science), 389-393. doi: 10.1016/S0166-4115(99)80037-5
159. Vroomen, J., Driver, J., and de Gelder, B. (2001). Is cross-modal integration of emotional expressions independent of attentional resources? Cogn. Affect. Behav. Neurosci. 1, 382-387. doi: 10.3758/CABN.1.4.382
160. Vroomen, J., and Keetels, M. (2010). Perception of intersensory synchrony: a tutorial review. Attent. Percept. Psychophys. 72, 871-884. doi: 10.3758/APP.72.4.871
161. Vroomen, J., Keetels, M., De Gelder, B., and Bertelson, P. (2004). Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cogn. Brain Res. 22, 32-35. doi: 10.1016/j.cogbrainres.2004.07.003
162. Vroomen, J., and Stekelenburg, J. J. (2011). Perception of intersensory synchrony in audiovisual speech: not that special. Cognition 118, 75-83. doi: 10.1016/j.cognition.2010.10.002
163. Wahn, B., and König, P. (2016). Attentional resource allocation in visuotactile processing depends on the task, but optimal visuotactile integration does not depend on attentional resources. Front. Integr. Neurosci. 10:13. doi: 10.3389/fnint.2016.00013
164. Walker, S., Bruce, V., and O'Malley, C. (1995). Facial identity and facial speech processing: familiar faces and voices in the McGurk effect. Percept. Psychophys. 57, 1124-1133. doi: 10.3758/BF03208369
165. Walker-Andrews, A. (1994). "Taxonomy for intermodal relations," in The Development of Intersensory Perception: Comparative Perspectives, eds D. J. Lewkowicz and R. Lickliter (Hillsdale, NJ: Lawrence Erlbaum), 39-56
166. Wallace, M. T., Roberson, G. E., Hairston, W. D., Stein, B. E., Vaughan, J. W., and Schirillo, J. A. (2004). Unifying multisensory signals across time and space. Exp. Brain Res. 158, 252-258. doi: 10.1007/s00221-004-1899-9
167. Warren, D. H., Welch, R. B., and McCarthy, T. J. (1981). The role of visual-auditory 'compellingness' in the ventriloquism effect: implications for transitivity among the spatial senses. Percept. Psychophys. 30, 557-564. doi: 10.3758/BF03202010
168. Welch, R. B. (1972). The effect of experienced limb identity upon adaptation to simulated displacement of the visual field. Percept. Psychophys. 12, 453-456. doi: 10.3758/BF03210933
169. Welch, R. B. (1999). "Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions," in Cognitive Contributions to the Perception of Spatial and Temporal Events, eds G. Ashersleben, T. Bachmann, and J. Müsseler (Amsterdam: Elsevier Science), 371-387. doi: 10.1016/S0166-4115(99)80036-3
170. Welch, R. B., and Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 3, 638-667. doi: 10.1037/0033-2909.88.3.638
171. Welch, R. B., and Warren, D. H. (1986). "Intersensory interactions," in Handbook of Perception and Performance: Sensory Processes and Perception, Vol. 1, eds K. R. Boff, L. Kaufman, and J. P. Thomas (New York, NY: Wiley), 25-36
172. Werner, S., and Noppeney, U. (2010). Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J. Neurosci. 30, 2662-2675. doi: 10.1523/JNEUROSCI.5091-09.2010
173. Wozny, D. R., Beierholm, U. R., and Shams, L. (2010). Probability matching as a computational strategy used in perception. PLoS Comput. Biol. 6:e1000871. doi: 10.1371/journal.pcbi.1000871
174. Wozny, D. R., and Shams, L. (2011). Recalibration of auditory space following milliseconds of cross-modal discrepancy. J. Neurosci. 31, 4607-4612. doi: 10.1523/JNEUROSCI.6079-10.2011
175. Yuval-Greenberg, S., and Deouell, L. Y. (2007). What you see is not (always) what you hear: induced gamma band responses reflect cross-modal interactions in familiar object recognition. J. Neurosci. 27, 1090-1096. doi: 10.1523/JNEUROSCI.4828-06.2007
176. Zampini, M., Guest, S., Shore, D. I., and Spence, C. (2005). Audio-visual simultaneity judgments. Percept. Psychophys. 67, 531-544. doi: 10.3758/BF03193329
177. Zangenehpour, S., and Zatorre, R. J. (2010). Cross-modal recruitment of primary visual cortex following brief exposure to bimodal audiovisual stimuli. Neuropsychologia 48, 591-600. doi: 10.1016/j.neuropsychologia.2009.10.022