The supramodal brain: implications for auditory perception

Journal of Cognitive Psychology

Volumn 29, Issue 1, 2017, Pages 65-87

  Rosenblum, Lawrence D ^a   Dias, James W ^a   Dorsi, Josh ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

auditory; Multisensory; speech

Indexed keywords

ARTICLE; BEHAVIOR; BLINDNESS; BRAIN; BRAIN CORTEX; ECHOLOCATION; HEARING; HEARING IMPAIRMENT; HUMAN; NERVE CELL PLASTICITY; NONHUMAN; PARITY; PERSONAL EXPERIENCE; PRIORITY JOURNAL; SENSORIMOTOR INTEGRATION; SPEECH; SPEECH PERCEPTION; SUPRAMODAL BRAIN; VISION;

EID: 84966783643     PISSN: 20445911     EISSN: 2044592X     Source Type: Journal    
DOI: 10.1080/20445911.2016.1181691     Document Type: Article

References (219)

1. Abboud, S., Maidenbaum, S., Dehaene,S., & Amedi, A., (2015). A number-form area in the blind. Nature Communications, 6. doi:10.1038/ncomms7026
2. Allman, B. L., Keniston, L. P., & Meredith, M. A., (2009). Adult deafness induces somatosensory conversion of ferret auditory cortex. Proceedings of the National Academy of Sciences, 106(14), 5925–5930. doi:10.1073/pnas.0809483106
3. Altieri, N., Pisoni, D. B., & Townsend, J. T., (2011). Some behavioral and neurobiological constraints on theories of audiovisual speech integration:A review and suggestions for new directions. Seeing and Perceiving, 24(6), 513–539. doi:10.1163/187847611X595864
4. Andersson, U., & Lidestam, B., (2005). Bottom-Up driven speechreading in a speechreading expert:The case of AA (JK023). Ear and Hearing, 26(2), 214–224. doi:10.1097/00003446-200504000-00008
5. Arnold, P., & Hill, F., (2001). Bisensory augmentation:A speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92(2), 339–355. doi:10.1348/000712601162220
6. Arnott, S. R., & Goodale, M. A., (2006). Distorting visual space with sound. Vision Research, 46(10), 1553–1558. doi:10.1016/j.visres.2005.11.020
7. Arnott, S. R., Thaler, L., Milne, J. L., Kish, D., & Goodale, M. A., (2013). Shape-specific activation of occipital cortex in an early blind echolocation expert. Neuropsychologia, 51(5), 938–949. doi:10.1016/j.neuropsychologia.2013.01.024
8. Ashmead, D. H., Wall, R. S., Eaton, S. B., Ebinger, K. A., Snook-Hill, M. M., Guth, D. A., & Yang, X., (1998). Echolocation reconsidered:Using spatial variations in the ambient sound field to guide locomotion. Journal of Visual Impairment & Blindness, 92(9), 615–632. http://www.afb.org/jvib/jvibabstractNew.asp?articleid=jvib920905
9. Auer, E. T., & Bernstein, L. E., (2007). Enhanced visual speech perception in individuals with early-onset hearing impairment. Journal of Speech, Language, and Hearing Research, 50(5), 1157–1165. doi:10.1044/1092-4388(2007/080)
10. Auer, E. T., Bernstein, L. E., Sungkarat, W., & Singh, M., (2007). Vibrotactile activation of the auditory cortices in deaf versus hearing adults. NeuroReport, 18(7), 645–648. doi:10.1097/WNR.0b013e3280d943b9
11. Baart, M., Stekelenburg, J. J., & Vroomen, J., (2014). Electrophysiological evidence for speech-specific audiovisual integration. Neuropsychologia, 53, 115–121. doi:10.1016/j.neuropsychologia.2013.11.011
12. Battaglia, P. W., Jacobs, R. A., & Aslin, R. N., (2003). Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A, 20(7), 1391–1397. doi:10.1364/JOSAA.20.001391
13. Bavelier, D., Dye, M. W. G., & Hauser, P. C., (2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10(11), 512–518. doi:10.1016/j.tics.2006.09.006
14. Bernstein, L. E., Auer, E. T. J., & Moore, J. K., (2004). Audiovisual speech binding:Convergence or association. In G. A., Calvert (Ed.), The handbook of multisensory processes (pp. 203–223). Cambridge, MA:MIT Press.
15. Bernstein, L. E., Auer Jr, E. T., & Takayanagi, S., (2004). Auditory speech detection in noise enhanced by lipreading. Speech Communication, 44(1–4), 5–18. doi:10.1016/j.specom.2004.10.011
16. Bertelson, P., & De Gelder, B., (2004). The psychology of multimodal perception. In C., Spence & J., Driver (Eds.), Crossmodal space and crossmodal attention (pp. 141–179). Oxford:Oxford University Press.
17. Bertelson, P., Vroomen, J., Wiegeraad, G., & de Gelder, B., (1994). Exploring the relation between McGurk interference and ventriloquism. In International conference on spoken language processing (Vol. 2, pp. 559–562). Yokohama:Acoustical Society of Japan.
18. Besle, J., Bertrand, O., & Giard, M.-H., (2009). Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex. Hearing Research, 258(1–2), 143–151. doi:10.1016/j.heares.2009.06.016
19. Besle, J., Fort, A., Delpuech, C., & Giard, M.-H., (2004). Bimodal speech:Early suppressive visual effects in human auditory cortex. European Journal of Neuroscience, 20(8), 2225–2234. doi:10.1111/j.1460-9568.2004.03670.x
20. Bizley, J. K., Nodal, F. R., Bajo, V. M., Nelken, I., & King, A. J., (2007). Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cerebral Cortex, 17(9), 2172–2189. doi:10.1093/cercor/bhl128
21. Bonath, B., Noesselt, T., Martinez, A., Mishra, J., Schwiecker, K., Heinze, H.-J., & Hillyard, S. A., (2007). Neural basis of the ventriloquist illusion. Current Biology, 17(19), 1697–1703. doi:10.1016/j.cub.2007.08.050
22. Boonman, A., Bar-On, Y., Cvikel, N., & Yovel, Y., (2013). It’s not black or white–on the range of vision and echolocation in echolocating bats. Frontiers in Physiology, 4, 248. doi:10.3389/fphys.2013.00248
23. Bottari, D., Heimler, B., Caclin, A., Dalmolin, A., Giard, M.-H., & Pavani, F., (2014). Visual change detection recruits auditory cortices in early deafness. NeuroImage, 94, 172–184. doi:10.1016/j.neuroimage.2014.02.031
24. Buckingham, G., (2014). Getting a grip on heaviness perception:A review of weight illusions and their probable causes. Experimental Brain Research, 232(6), 1623–1629. doi:10.1007/s00221-014-3926-9
25. Buckingham, G., Milne, J. L., Byrne, C. M., & Goodale, M. A., (2015). The size-weight illusion induced through human echolocation. Psychological Science, 26(2), 237–242. doi:10.1177/0956797614561267
26. Burnham, D. K., Ciocca, V., Lauw, C., Lau, S., & Stokes, S., (2000). Perception of visual information for Cantonese tones. Proceedings of the Eighth Australian International Conference on Speech Science and Technology, pp. 86–91.
27. Burnham, D. K., & Dodd, B., (2004). Auditory-visual speech integration by prelinguistic infants:Perception of an emergent consonant in the McGurk effect. Developmental Psychobiology, 45(4), 204–220. doi:10.1002/dev.20032
28. Butler, B. E., & Lomber, S. G., (2013). Functional and structural changes throughout the auditory system following congenital and early-onset deafness:Implications for hearing restoration. Frontiers in Systems Neuroscience, 7. Article 92. doi:10.3389/fnsys.2013.00092
29. Callan, D. E., Callan, A. M., Kroos, C., & Vatikiotis-Bateson, E., (2001). Multimodal contribution to speech perception revealed by independent component analysis:A single-sweep EEG case study. Cognitive Brain Research, 10, 349–353. doi:10.1016/S0926-6410(00)00054-9
30. Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., … David, A. S., (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593–596. doi:10.1126/science.276.5312.593
31. Campbell, C., (2008). The processing of audio-visual speech:Empirical and neural bases. Philosophical Transactions of the Royal Society B:Biological Sciences, 363, 1001–1010. doi:10.1098/rstb.2007.2155
32. Campbell, R., (2011). Speechreading:What’s MISS-ing? In A., Calder, G., Rhodes, M., Johnson, & J., Haxby (Eds.), Oxford handbook of face perception (pp. 605–630). Oxford:Oxford University Press.
33. Cappe, C., Thelen, A., Romei, V., Thut, G., & Murray, M. M., (2012). Looming signals reveal synergistic principles of multisensory integration. Journal of Neuroscience, 32(4), 1171–1182. doi:10.1523/JNEUROSCI.5517-11.2012
34. Cappe, C., Thut, G., Romei, V., & Murray, M. M., (2009). Selective integration of auditory-visual looming cues by humans. Neuropsychologia, 47(4), 1045–1052. doi:10.1016/j.neuropsychologia.2008.11.003
35. Carello, C., Grosofsky, A., Reichel, F. D., Solomon, H. Y., & Turvey, M. T., (1989). Visually perceiving what is reachable. Ecological Psychology, 1, 27–54. doi:10.1207/s15326969eco0101_3
36. Carello, C., Wagman, J. B., & Turvey, M. T., (2005). Acoustic specification of object properties. In J. D., Anderson & B. F., Anderson (Eds.), Moving image theory:Ecological considerations (pp. 79–104). Carbondale, IL:Souther Illinois University Press.
37. Carlson-Smith, C., & Wiener, W. R., (1996). The auditory skills necessary for echolocation:A new explanation. Journal of Visual Impairment & Blindness, 90, 21–35. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=eue&AN=508580609&site=ehost-live
38. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A. A., (2009). The natural statistics of audiovisual speech. PLoS Computational Biology, 5(7), 1–18. doi:10.1371/journal.pcbi.1000436
39. Chen, L., & Vroomen, J., (2013). Intersensory binding across space and time:A tutorial review. Attention, Perception, & Psychophysics, 75(5), 790–811. doi:10.3758/s13414-013-0475-4
40. Chen, T. H., & Massaro, D. W., (2008). Seeing pitch:Visual information for lexical tones of Mandarin-Chinese. The Journal of the Acoustical Society of America, 123(4), 2356–2366. doi:10.1121/1.2839004
41. Cheung, S. H., Fang, F., & He, S., (2009). Retinotopically specific reorganization of visual cortex for tactile pattern recognition. Current Biology, 19, 596–601. doi:10.1016/j.cub.2009.02.063
42. Choe, C. S., Welch, R. B., Gilford, R. M., & Juola, J. F., (1975). The “ventriloquism effect”:Visual dominance or response bias. Perception & Psychophysics, 18(1), 55–60. doi:10.3758/BF03199367
43. Colin, C., Radeau, M., Soquet, A., Demolin, D., Colin, F., & Deltenre, P., (2002). Mismatch negativity evoked by the McGurk-MacDonald effect:A phonetic representation within short-term memory. Clinical Neurophysiology, 113, 495–506. doi:10.1016/S1388-2457(02)00024-X
44. Collignon, O., Lassonde, M., Lepore, F., Bastien, D., & Veraart, C., (2007). Functional cerebral reorganization for auditory spatial processing and auditory substitution of vision in early blind subjects. Cerebral Cortex, 17(2), 457–465. doi:10.1093/cercor/bhj162
45. Collignon, O., Voss, P., Lassonde, M., & Lepore, F., (2009). Cross-modal plasticity for the spatial processing of sounds in visually deprived subjects. Experimental Brain Research, 192(3), 343–358. doi:10.1007/s00221-008-1553-z
46. Cooper, W. E., Blumstein, S. E., & Nigro, G., (1975). Articulatory effects on speech perception:A preliminary report. Journal of Phonetics, 3, 87–98.
47. Cooper, W. E., Billings, D., & Cole, R. A., (1976). Articulatory effects on speech perception:A second report. Journal of Phonetics, 4(3), 219–232.
48. Cooper, W. E., & Lauritsen, M. R., (1974). Feature processing in the perception and production of speech. Nature, 252, 121–123. doi:10.1038/252121a0
49. Deneve, S., & Pouget, A., (2004). Bayesian multisensory integration and cross-modal spatial links. Journal of Physiology-Paris, 98(1–3), 249–258. doi:10.1016/j.jphysparis.2004.03.011
50. Derrick, D., & Gick, B., (2013). Aerotactile integration from distal skin stimuli. Multisensory Research, 26(5), 405–416. doi:10.1163/22134808-00002427
51. Desjardins, R. N., & Werker, J. F., (2004). Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology, 45(4), 187–203. doi:10.1002/dev.20033
52. Després, O., Candas, V., & Dufour, A., (2005). Spatial auditory compensation in early-blind humans:Involvement of eye movements and/or attention orienting? Neuropsychologia, 43(13), 1955–1962. doi:10.1016/j.neuropsychologia.2005.03.002
53. Dias, J. W., Cook, T. C., & Rosenblum, L. D., (2016). Influences of selective adaptation on perception of audiovisual speech. Journal of Phonetics, 56, 75–84. doi:10.1016/j.wocn.2016.02.004
54. Dias, J. W., Cook, T. C., & Rosenblum, L. D., (in press). The McGurk effect and the primacy of multisensory perception. In A. G., Shapiro & D., Todorovic (Eds.), Oxford compendium of visual illusions. Oxford University Press.
55. Dias, J. W., & Rosenblum, L. D., (2011). Visual influences on interactive speech alignment. Perception, 40, 1457–1466. doi:10.1068/p7071
56. Dias, J. W., & Rosenblum, L. D., (2015a). Visibility of speech articulation enhances auditory phonetic convergence. Attention, Perception, & Psychophysics, 77(6). doi:10.3758/s13414-015-0982-6
57. Dias, J. W., & Rosenblum, L. D., (2015b). Selective adaptation of crossmodal speech information:A case of small but significant effects. Abstracts of the 56th Meeting of the Psychonomics Society. 202–203.
58. Dufour, A., & Gérard, Y., (2000). Improved auditory spatial sensitivity in near-sighted subjects. Cognitive Brain Research, 10(1–2), 159–165. doi:10.1016/S0926-6410(00)00014-8
59. Dupont, S., Aubin, J., & Menard, L., (2005). The study of the McGurk effect in 4 and 5-year-old French Canadian children. ZAS Papers in Linguistics, 40, 1–17.
60. Eberhardt, S. P., Auer Jr, E. T., & Bernstein, L. E., (2014). Multisensory training can promote or impede visual perceptual learning of speech stimuli:Visual-tactile vs. visual-auditory training. Frontiers in Human Neuroscience, 8, 185. doi:10.3389/fnhum.2014.00829
61. Ehrenstein, W. H., & Reinhardt-Rutland, A. H., (1996). A cross-modal aftereffect:Auditory displacement following adaptation to visual motion. Perceptual and Motor Skills, 82, 23–26. doi:10.2466/pms.1996.82.1.23
62. Fowler, C. A., (1986). An event approach to the study of speech perception from a direct-realist perspective. Journal of Phonetics, 14, 3–28.
63. Fowler, C. A., (2004). Speech as a supramodal or amodal phenomenon. In G. A., Calvert, C., Spence, & B. E., Stein (Eds.), The handbook of multisensory processing (pp. 189–202). Cambridge, MA:MIT Press.
64. Fowler, C. A., & Dekle, D. J., (1991). Listening with eye and hand:Cross-modal contributions to speech perception. Journal of Experimental Psychology:Human Perception and Performance, 17(3), 816–828. doi:10.1037/0096-1523.17.3.816
65. Freeman, E., & Driver, J., (2008). Direction of visual apparent motion driven solely by timing of a static sound. Current Biology, 18(16), 1262–1266. doi:10.1016/j.cub.2008.07.066
66. Frissen, I., Vroomen, J., de Gelder, B., & Bertelson, P., (2003). The aftereffects of ventriloquism:Are they sound-frequency specific? Acta Psychologica, 113, 315–327. doi:10.1016/S0001-6918(03)00043-X
67. Fuster-Duran, A., (1996). Perception of conflicting audio-visual speech:An examination across Spanish and German. Speechreading by Humans and Machines, 150, 135–143. doi:10.1007/978-3-662-13015-5_9
68. Galantucci, B., Fowler, C. A., & Turvey, M. T., (2006). The motor theory of speech perception reviewed. Psychonomic Bulletin & Review, 13, 361–377. doi:10.3758/BF03193857
69. Gentilucci, M., & Cattaneo, L., (2005). Automatic audiovisual integration in speech perception. Experimental Brain Research, 167, 66–75. doi:10.1007/s00221-005-0008-z
70. Gibson, J. J., (1979). The ecological approach to visual perception. Boston, MA:Houghton Mifflin.
71. Gick, B., & Derrick, D., (2009). Aero-tactile integration in speech perception. Nature, 462, 502–504. doi:10.1038/nature08572
72. Goldinger, S. D., (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105(2), 251–279. doi:10.1037/0033-295X.105.2.251
73. Gordon, M. S., & Rosenblum, L. D., (2004). Perception of sound-obstructing surfaces using body-scaled judgments. Ecological Psychology, 16(2), 87–113. doi:10.1207/s15326969eco1602_1
74. Gordon, M. S., & Rosenblum, L. D., (2005). Effects of intrastimulus modality change on audiovisual time-to-arrival judgments. Perception & Psychophysics, 67(4), 580–594. doi:10.3758/BF03193516
75. Green, K. P., (1998). The use of auditory and visual information during phonetic processing:Implications for theories of speech perception. In R., Campbell & B., Dodd (Eds.), Hearing by eye II:Advances in the psychology of speechreading and audiovisual speech (pp. 3–25). Hove:Psychology Press.
76. Hackett, T. A., De La Mothe, L. A., Ulbert, I., Karmos, G., Smiley, J., & Schroeder, C. E., (2007). Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. The Journal of Comparative Neurology, 502(6), 924–952. doi:10.1002/cne.21326
77. Horowitz, S. S., Cheney, C. A., & Simmons, J. A., (2004). Interaction of vestibular, echolocation, and visual modalities guiding flight by the big brown bat, Eptesicus fuscus. Journal of Vestibular Research:Equilibrium & Orientation, 14(1), 17–32. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=a2h&AN=12968600&site=ehost-live
78. Hulusic, V., Harvey, C., Debattista, K., Tsingos, N., Walker, S., Howard, D., & Chalmers, A., (2012). Acoustic rendering and auditory–visual cross-modal perception and interaction. Computer Graphics Forum, 31(1), 102–131. doi:10.1111/j.1467-8659.2011.02086.x
79. Ito, T., Tiede, M., & Ostry, D. J., (2009). Somatosensory function in speech perception. Proceedings of the National Academy of Sciences of the United States of America, 106(4), 1245–1248. doi:10.1073/pnas.0810063106
80. Jaekl, P., Pesquita, A., Alsius, A., Munhall, K., & Soto-Faraco, S., (2015). The contribution of dynamic visual cues to audiovisual speech perception. Neuropsychologia, 75, 402–410. doi:10.1016/j.neuropsychologia.2015.06.025
81. Jerger, S., Damian, M. F., Spence, M. J., Tye-Murray, N., & Abdi, H., (2009). Developmental shifts in children's sensitivity to visual speech:A new multimodal picture-word task. Journal of Experimental Child Psychology, 102, 40–59. doi:10.1016/j.jecp.2008.08.002
82. Karns, C. M., Dow, M. W., & Neville, H. J., (2012). Altered cross-modal processing in the primary auditory cortex of congenitally deaf adults:A visual-somatosensory fMRI study with a double-flash illusion. Journal of Neuroscience, 32(28), 9626–9638. doi:10.1523/JNEUROSCI.6488-11.2012
83. Kitagawa, N., & Ichihara, S., (2002). Hearing visual motion in depth. Nature, 416, 172–174. doi:10.1038/416172a
84. Knill, D. C., & Pouget, A., (2004). The Bayesian brain:The role of uncertainty in neural coding and computation. Trends in Neurosciences, 27(12), 712–719. doi:10.1016/j.tins.2004.10.007
85. Kolarik, A. J., Cirstea, S., Pardhan, S., & Moore, B. C. J., (2014). A summary of research investigating echolocation abilities of blind and sighted humans. Hearing Research, 310, 60–68. doi:10.1016/j.heares.2014.01.010
86. Kolarik, A. J., Timmis, M. A., Cirstea, S., & Pardhan, S., (2014). Sensory substitution information informs locomotor adjustments when walking through apertures. Experimental Brain Research, 232(3), 975–984. doi:10.1007/s00221-013-3809-5
87. Konkle, T., Wang, Q., Hayward, V., & Moore, C. I., (2009). Motion aftereffects transfer between touch and vision. Current Biology, 19(9), 745–750. doi:10.1016/j.cub.2009.03.035
88. Kral, A., Schröder, J.-H., Klinke, R., & Engel, A. K., (2003). Absence of cross-modal reorganization in the primary auditory cortex of congenitally deaf cats. Experimental Brain Research, 153(4), 605–613. doi:10.1007/s00221-003-1609-z
89. Kuczaj, S., Solangi, M., Hoffland, T., & Romagnoli, M., (2008). Recognition and discrimination of human actions across the senses of echolocation and vision in the Bottlenose Dolphin:Evidence for Dolphin cross-modal integration of dynamic information. International Journal of Comparative Psychology, 21(2). Retrieved from http://www.escholarship.org/uc/item/8805892k
90. Kupers, R., Chebat, D. R., Madsen, K. H., Paulson, O. B., & Ptito, M., (2010). Neural correlates of virtual route recognition in congenital blindness. Proceedings of the National Academy of Sciences, 107(28), 12716–12721. doi:10.1073/pnas.1006199107
91. Lachs, L., & Pisoni, D. B., (2004a). Cross-modal source information and spoken word recognition. Journal of Experimental Psychology:Human Perception and Performance, 30(2), 378–396. doi:10.1037/0096-1523.30.2.378
92. Lachs, L., & Pisoni, D. B., (2004b). Crossmodal source identification in speech perception. Ecological Psychology, 16(3), 159–187. doi:10.1207/s15326969eco1603_1
93. Lachs, L., & Pisoni, D. B., (2004c). Specification of cross-modal source information in isolated kinematic displays of speech. The Journal of the Acoustical Society of America, 116(1), 507–518. doi:10.1121/1.1757454
94. Lander, K., & Davies, R., (2008). Does face familiarity influence speechreadability? The Quarterly Journal of Experimental Psychology, 61(7), 961–967. doi:10.1080/17470210801908476
95. Larsson, P., Västfjäll, D., & Kleiner, M., (2001). Better presence and performance in virtual environments by improved binaural sound rendering. Paper presented at the 22nd International Conference:Virtual, Synthetic, and Entertainment Audio.
96. Lee, D., van der Weel, F. R., Hitchcock, T., Matejowsky, E., & Pettigrew, J., (1992). Common principle of guidance by echolocation and vision. Journal of Comparative Physiology A, 171(5), 563–571. doi:10.1007/BF00194105
97. Levitan, C. A., Ban, Y.-H. A., Stiles, N. R. B., & Shimojo, S., (2015). Rate perception adapts across the senses:Evidence for a unified timing mechanism. Scientific Reports, 5. doi:10.1038/srep08857
98. Liberman, A. M., & Mattingly, I. G., (1985). The motor theory of speech perception revised. Cognition, 21, 1–36. doi:10.1016/0010-0277(85)90021-6
99. Lomber, S. G., Meredith, M. A., & Kral, A., (2010). Cross-modal plasticity in specific auditory cortices underlies visual compensations in the deaf. Nature Neuroscience, 13(11), 1421–1427. doi:10.1038/nn.2653
100. Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A., (2006). Bayesian inference with probabilistic population codes. Nature Neuroscience, 9(11), 1432–1438. doi:10.1038/nn1790
101. MacDonald, J., & McGurk, H., (1978). Visual influences on speech perception processes. Perception & Psychophysics, 24(3), 253–257. doi:10.3758/BF03206096
102. Maempel, H., & Jentsch, M., (2013). Audio-visual interaction of size and distance perception in concert halls–a preliminary study. Paper presented at the International Symposium on Room Acoustics, Toronto, Canada.
103. Mahon, B. Z., Anzellotti, S., Schwarzbach, J., Zampini, M., & Caramazza, A., (2009). Category-Specific organization in the human brain does not require visual experience. Neuron, 63(3), 397–405. doi:10.1016/j.neuron.2009.07.012
104. Massaro, D. W., (1998). Perceiving talking faces:From speech perception to a behavioral principle. Cambridge, MA:MIT Press.
105. Massaro, D. W., & Cohen, M. M., (1993). Perceiving asynchronous bimodal speech in consonant-vowel and vowel syllables. Speech Communication, 13, 127–134. doi:10.1016/0167-6393(93)90064-R
106. Massaro, D. W., Cohen, M. M., Gesi, A., & Heredia, R., (1993). Bimodal speech perception:An examination across languages. Journal of Phonetics, 21, 445–478.
107. Matsumiya, K., (2013). Seeing a haptically explored face:Visual facial-expression aftereffect from haptic adaptation to a face. Psychological Science. doi:10.1177/0956797613486981
108. Matteau, I., Kupers, R., Ricciardi, E., Pietrini, P., & Ptito, M., (2010). Beyond visual, aural and haptic movement perception:hMT+ is activated by electrotactile motion stimulation of the tongue in sighted and in congenitally blind individuals. Brain Research Bulletin, 82(5–6), 264–270. doi:10.1016/j.brainresbull.2010.05.001
109. McGurk, H., & MacDonald, J., (1976). Hearing lips and seeing voices. Nature, 264, 746–748. doi:10.1038/264746a0
110. Merabet, L. B., & Pascual-Leone, A., (2010). Neural reorganization following sensory loss:The opportunity of change. Nature Reviews Neuroscience, 11, 44–52. doi:10.1038/nrn2758
111. Meredith, M. A., & Allman, B. L., (2012). Early hearing-impairment results in crossmodal reorganization of ferret core auditory cortex. Neural Plasticity, 2012, 6, 1–13. doi:10.1155/2012/601591
112. Miller, R. M., Sanchez, K., & Rosenblum, L. D., (2010). Alignment to visual speech information. Attention, Perception, & Psychophysics, 72(6), 1614–1625. doi:10.3758/APP.72.6.1614
113. Mills, A. E., (1987). The development of phonology in the blind child. In B., Dodd & R., Campbell (Eds.), Hearing by eye:The psychology of lip reading (pp. 145–162). Hillsdale, NJ:Lawrence Erlbaum.
114. Milne, J. L., Arnott, S. R., Kish, D., Goodale, M. A., & Thaler, L., (2015). Parahippocampal cortex is involved in material processing via echoes in blind echolocation experts. Vision Research, 109(Part B), 139–148. doi:10.1016/j.visres.2014.07.004
115. Milne, J. L., Goodale, M. A., & Thaler, L., (2014). The role of head movements in the discrimination of 2-D shape by blind echolocation experts. Attention, Perception, & Psychophysics, 76(6), 1828–1837. doi:10.3758/s13414-014-0695-2
116. Milton, C., & Dallenbach, K. M., (1950). “Facial vision:” The rôle of pitch and loudness in the perception of obstacles by the blind. The American Journal of Psychology, 63(4), 485–515. doi:10.2307/1418868
117. Munhall, K. G., Gribble, P., Sacco, L., & Ward, M., (1996). Temporal constraints on the McGurk effect. Perception & Psychophysics, 58(3), 351–362.
118. Munhall, K. G., & Vatikiotis-Bateson, E., (1998). The moving face during speech communication. In R., Campbell, B., Dodd, & D., Burnham (Eds.), Hearing by eye II:Advances in the psychology of speechreading and audio-visual speech (pp. 123–139). East Sussex:Psychology Press.
119. Munhall, K. G., & Vatikiotis-Bateson, E., (2004). Spatial and temporal constraints on audiovisual speech perception. In G., Calvert, C., Spence, & B. E., Stein (Eds.), The handbook of multisensory processes (pp. 177–187). Cambridge, MA:MIT Press.
120. Musacchia, G., Sams, M., Nicol, T., & Kraus, N., (2006). Seeing speech affects acoustic information processing in the human brainstem. Experimental Brain Research, 168(1–2), 1–10. doi:10.1007/s00221-005-0071-5
121. Navarra, J., & Soto-Faraco, S., (2007). Hearing lips in a second language:Visual articulatory information enables the perception of second language sounds. Psychological Research, 71, 4–12. doi:10.1007/s00426-005-0031-5
122. Nygaard, L. C., (2005). Perceptual integration of linguistic and non-linguistic properties of speech. In D., Pisoni & R., Remez (Eds.), Handbook of speech perception (pp. 390–419). Malden, MA:Blackwell.
123. Orbach, D. N., & Fenton, B., (2010). Vision impairs the abilities of bats to avoid colliding with stationary obstacles. PLoS One, 5(11), e13912. doi:10.1371/journal.pone.0013912
124. Ostrand, R., Blumstein, S. E., & Morgan, J. L., (2011). When hearing lips and seeing voices becomes perceiving speech:Auditory-visual integration in lexical access. Proceedings of the Annual Meeting of the Cognitive Science Society, 33, 1376–1381.
125. Pack, A. A., & Herman, L. M., (1995). Sensory integration in the bottlenosed dolphin:Immediate recognition of complex shapes across the senses of echolocation and vision. The Journal of the Acoustical Society of America, 98(2), 722–733. doi:10.1121/1.413566
126. Pack, A. A., Herman, L. M., Hoffmann-Kuhnt, M., & Branstetter, B. K., (2002). The object behind the echo:dolphins (Tursiops truncatus) perceive object shape globally through echolocation. Behavioural Processes, 58(1–2), 1–26. doi:10.1016/S0376-6357(01)00200-5
127. Pardo, J. S., (2006). On phonetic convergence during conversational interaction. The Journal of the Acoustical Society of America, 119(4), 2382–2393. doi:10.1121/1.2178720
128. Pardo, J. S., & Remez, R. E., (2006). The perception of speech. In M., Traxler & M. A., Gernsbacher (Eds.), The handbook of psycholinguistics (2nd ed., pp. 201–248). New York, NY:Academic Press.
129. Pascual-Leone, A., & Hamilton, R., (2001). The metamodal organization of the brain. Progress in Brain Research, 134(Chapter 27), 1–19.
130. Pekkola, J., Ojanen, V., Autti, T., Jaaskelainen, I. P., Mottonen, R., Tarkiainen, A., & Sams, M., (2005). Primary auditory cortex activation by visual speech:An fMRI study at 3T. Auditory and Vestibular Systems, 16(2), 125–128.
131. Rebillard, G., Carlier, E., Rebillard, M., & Pujol, R., (1977). Enhancement of visual responses on the primary auditory cortex of the cat after an early destruction of cochlear receptors. Brain Research, 129(1), 162–164. doi:10.1016/0006-8993(77)90980-5
132. Reed, C. M., Rabinowitz, W. M., Durlach, N. I., Braida, L. D., Conway-Fithian, S., & Schultz, M. C., (1985). Research on the Tadoma method of speech communication. The Journal of the Acoustical Society of America, 77(1), 247–257. doi:10.1121/1.392266
133. Reich, L., Maidenbaum, S., & Amedi, A., (2012). The brain as a flexible task machine:Implications for visual rehabilitation using noninvasive vs. invasive approaches. Current Opinion in Neurology, 25(1), 86–95. doi:10.1097/WCO.0b013e32834ed723
134. Reisberg, D., McLean, J., & Goldfield, A., (1987). Easy to hear but hard to understand:A lip-reading advantage with intact auditory stimuli. In B., Dodd & R., Campbell (Eds.), Hearing by eye:The psychology of lip-reading (pp. 97–113). Hillsdale, NJ:Lawrence Erlbaum Associates.
135. Remez, R. E., Fellowes, J. M., Pisoni, D. B., Goh, W. D., & Rubin, P. E., (1998). Multimodal perceptual organization of speech:Evidence from tone analogs of spoken utterances. Speech Communication, 26, 65–73. doi:10.1016/S0167-6393(98)00050-8
136. Remez, R. E., Rubin, P. E., Pisoni, D. B., & Carrell, T. D., (1981). Speech perception without traditional speech cues. Science, 212, 947–949. doi:10.1126/science.7233191
137. Ricciardi, E., Bonino, D., Pellegrini, S., & Pietrini, P., (2014). Mind the blind brain to understand the sighted one! Is there a supramodal cortical functional architecture? Neuroscience & Biobehavioral Reviews, 41, 64–77. doi:10.1016/j.neubiorev.2013.10.006
138. Robart, R. L., & Rosenblum, L. D., (2005). Hearing space:Identifying rooms by reflected sound. In H., Heft & K. L., Marsh (Eds.), Studies in perception and action XIII (pp. 153–161). Hillsdale, NJ:Lawrence Erlbaum Associates.
139. Roberts, M., & Summerfield, Q., (1981). Audiovisual presentation demonstrates that selective adaptation in speech perception is purely auditory. Perception & Psychophysics, 30(4), 309–314. doi:10.3758/BF03206144
140. Rohe, T., & Noppeney, U., (2015). Cortical hierarchies perform Bayesian causal inference in multisensory perception. PLOS Biology, 13(2), e1002073. doi:10.1371/journal.pbio.1002073
141. Rosen, S. M., Fourcin, A. J., & Moore, B. C. J., (1981). Voice pitch as an aid to lipreading. Nature, 291(5811), 150–152. doi:10.1038/291150a0
142. Rosenblum, L. D., (2004). Perceiving articulatory events:Lessons for an ecological psychoacoustics. In J. G., Neuhoff (Ed.), Ecological psychoacoustics (pp. 219–248). San Diego, CA:Elsevier Academic Press.
143. Rosenblum, L. D., (2005). Primacy of multimodal speech perception. In D., Pisoni & R., Remez (Eds.), Handbook of speech perception (pp. 51–78). Malden, MA:Blackwell.
144. Rosenblum, L. D., (2008). Speech perception as a multimodal phenomenon. Current Directions in Psychological Science, 17(6), 405–409. doi:10.1111/j.1467-8721.2008.00615.x
145. Rosenblum, L. D., (2010). See what i’m saying:The extraordinary powers of our five senses. New York, NY:W. W. Norton & Company.
146. Rosenblum, L. D., (2013). A confederacy of senses. Scientific American, 308, 72–75. doi:10.1038/scientificamerican0113-72
147. Rosenblum, L. D., Gordon, M. S., & Jarquin, L., (2000). Echolocating distance by moving and stationary listeners. Ecological Psychology, 12(3), 181–206. doi:10.1207/S15326969ECO1203_1
148. Rosenblum, L. D., Johnson, J. A., & Saldaña, H. M., (1996). Point-light facial displays enhance comprehension of speech in noise. Journal of Speech Language and Hearing Research, 39(6), 1159–1170. doi:10.1044/jshr.3906.1159
149. Rosenblum, L. D., Miller, R. M., & Sanchez, K., (2007). Lip-read me now, hear me better later:Cross-modal transfer of talker-familiarity effects. Psychological Science, 18(5), 392–396. doi:10.1111/j.1467-9280.2007.01911.x
150. Rosenblum, L. D., & Saldaña, H. M., (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology:Human Perception and Performance, 22(2), 318–331. doi:10.1037/0096-1523.22.2.318
151. Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A., (1997). The McGurk effect in infants. Perception & Psychophysics, 59(3), 347–357. doi:10.3758/BF03211902
152. Rosenblum, L. D., Smith, N. M., Nichols, S. M., Hale, S., & Lee, J., (2006). Hearing a face:Cross-modal speaker matching using isolated visible speech. Perception & Psychophysics, 68(1), 84–93. doi:10.3758/BF03193658
153. Rosenblum, L. D., Wuestefeld, A. P., & Anderson, K. L., (1996). Auditory reachability:An affordance approach to the perception of sound source distance. Ecological Psychology, 8(1), 1–24. doi:10.1207/s15326969eco0801_1
154. Ross, H. E., (1969). When is a weight not illusory? Quarterly Journal of Experimental Psychology, 21(4), 346–355. doi:10.1080/14640746908400230
155. Saldaña, H. M., & Rosenblum, L. D., (1993). Visual influences on auditory pluck and bow judgments. Perception & Psychophysics, 54(3), 406–416. doi:10.3758/BF03205276
156. Saldaña, H. M., & Rosenblum, L. D., (1994). Selective adaptation in speech perception using a compelling audiovisual adaptor. The Journal of the Acoustical Society of America, 95(6), 3658–3661. doi:10.1121/1.409935
157. Sams, M., Aulanko, R., Hamalainen, M., Hari, R., Lounasmaa, O. V., Lu, S., & Simola, J., (1991). Seeing speech:Visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters, 127, 141–145. doi:10.1016/0304-3940(91)90914-F
158. Sams, M., Manninen, P., Surakka, V., Helin, P., & Kättö, R., (1998). McGurk effect in Finnish syllables, isolated words, and words in sentences:Effects of word meaning and sentence context. Speech Communication, 26(1–2), 75–87. doi:10.1016/S0167-6393(98)00051-X
159. Sams, M., Mottonen, R., & Sihvonen, T., (2005). Seeing and hearing others and oneself talk. Cognitive Brain Research, 23, 429–435. doi:10.1016/j.cogbrainres.2004.11.006
160. Samuel, A. G., & Lieblich, J., (2014). Visual speech acts differently than lexical context in supporting speech perception. Journal of Experimental Psychology:Human Perception and Performance, 40(4), 1479–1490. doi:10.1037/a0036656
161. Sanchez, K., Dias, J. W., & Rosenblum, L. D., (2013). Experience with a talker can transfer across modalities to facilitate lipreading. Attention, Perception, & Psychophysics, 75(7), 1359–1365. doi:10.3758/s13414-013-0534-x
162. Sanchez, K., Miller, R. M., & Rosenblum, L. D., (2010). Visual influences on alignment to voice onset time. Journal of Speech Language and Hearing Research, 53, 262–272. doi:10.1044/1092-4388(2009/08-0247)
163. Sandmann, P., Dillier, N., Eichele, T., Meyer, M., Kegel, A., Pascual-Marqui, R. D., … Debener, S., (2012). Visual activation of auditory cortex reflects maladaptive plasticity in cochlear implant users. Brain, 135, 555–568. doi:10.1093/brain/awr329
164. Scharine, A. A., Letowski, T. R., Mermagen, T., & Henry, P. P., (2010). Learning to detect and identify acoustic environments from reflected sound. Military Psychology, 22(1), 24–40. doi:10.1080/08995600903206461
165. Schroeder, C. E., & Foxe, J. J., (2002). The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cognitive Brain Research, 14(1), 187–198. doi:10.1016/S0926-6410(02)00073-3
166. Schroeder, C. E., Lindsley, R. W., Specht, C., Marcovici, A., Smiley, J. F., & Javitt, D. C., (2001). Somatosensory input to auditory association cortex in the Macaque monkey. Journal of Neurophysiology, 85(3), 1322–1327.
167. Schutz, M., & Lipscomb, S., (2007). Hearing gestures, seeing music:Vision influences perceived tone duration. Perception, 36, 888–897. doi:10.1068/p5635
168. Scott, G. D., Karns, C. M., Dow, M. W., Stevens, C., & Neville, H. J., (2014). Enhanced peripheral visual processing in congenitally deaf humans is supported by multiple brain regions, including primary auditory cortex. Frontiers in Human Neuroscience, 8, 5925. doi:10.3389/fnhum.2014.00177
169. Sekiyama, K., & Tohkura, Y., (1991). McGurk effect in non-English listeners:Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. The Journal of the Acoustical Society of America, 90(4), 1797–1805. doi:10.1121/1.401660
170. Sekiyama, K., & Tohkura, Y., (1993). Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics, 21(4), 427–444.
171. Shams, L., (2011). Early integration and Bayesian causal inference in multisensory perception. In M. M., Murray & M. T., Wallace, The neural bases of multisensory processes (pp. 217–232). Boca Raton, FL:CRC Press/Tay & Francis.
172. Shams, L., Iwaki, S., Chawla, A., & Bhattacharya, J., (2005). Early modulation of visual cortex by sound:An MEG study. Neuroscience Letters, 378, 76–81. doi:10.1016/j.neulet.2004.12.035
173. Shams, L., Kamitani, Y., & Shimojo, S., (2000). What you see is what you hear. Nature, 408, 788. doi:10.1038/35048669
174. Shams, L., Kamitani, Y., & Shimojo, S., (2002). Visual illusion induced by sound. Cognitive Brain Research, 14, 147–152. doi:10.1016/S0926-6410(02)00069-1
175. Shams, L., Kamitani, Y., Thompson, S., & Shimojo, S., (2001). Sound alters visual evoked potentials in humans. NeuroReport, 12, 3849–3852. doi:10.1097/00001756-200112040-00049
176. Shams, L., & Kim, R., (2010). Crossmodal influences on visual perception. Physics of Life Reviews, 7, 269–284. doi:10.1016/j.plrev.2010.04.006
177. Shams, L., Wozny, D. R., Kim, R., & Seitz, A., (2011). Influences of multisensory experience on subsequent unisensory processing. Frontiers in Psychology, 2, 264. doi:10.3389/fpsyg.2011.00264
178. Smiley, J. F., & Falchier, A., (2009). Multisensory connections of monkey auditory cerebral cortex. Hearing Research, 258(1–2), 37–46. doi:10.1016/j.heares.2009.06.019
179. Smiley, J. F., Hackett, T. A., Ulbert, I., Karmas, G., Lakatos, P., Javitt, D. C., & Schroeder, C. E., (2007). Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. The Journal of Comparative Neurology, 502(6), 894–923. doi:10.1002/cne.21325
180. Smith, D., & Burnham, D. K., (2012). Faciliation of Mandarin tone perception by visual speech in clear and degraded audio:Implications for cochlear implants. The Journal of the Acoustical Society of America, 131(2), 1480–1489. doi:10.1121/1.3672703
181. Soto-Faraco, S., & Alsius, A., (2009). Deconstructing the McGurk-MacDonald illusion. Journal of Experimental Psychology:Human Perception and Performance, 35(2), 580–587. doi:10.1037/a0013483
182. Stekelenburg, J. J., & Vroomen, J., (2007). Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience, 19(12), 1964–1973. doi:10.1162/jocn.2007.19.12.1964
183. Stekelenburg, J., & Vroomen, J., (2012). Electrophysiological correlates of predictive coding of auditory location in the perception of natural audiovisual events. Frontiers in Integrative Neuroscience, 6. doi:10.3389/fnint.2012.00026
184. Stewart, D. L., & Starr, A., (1970). Absence of visually influenced cells in auditory cortex of normal and congenitally deaf cats. Experimental Neurology, 28(3), 525–528. doi:10.1016/0014-4886(70)90190-1
185. Stoffregen, T. A., & Pittenger, J. B., (1995). Human echolocation as a basic form of perception and action. Ecological Psychology, 7(3), 181–216. doi:10.1207/s15326969eco0703_2
186. Strange, W., Jenkins, J. J., & Johnson, T. L., (1983). Dynamic specification of coarticulated vowels. The Journal of the Acoustical Society of America, 74(3), 695–705. doi:10.1121/1.389855
187. Strelow, E. R., & Brabyn, J. A., (1982). Locomotion of the blind controlled by natural sound cues. Perception, 11(6), 635–640. doi:10.1068/p110635
188. Striem-Amit, E., & Amedi, A., (2014). Visual cortex extrastriate body-selective area activation in congenitally blind people “seeing” by using sounds. Current Biology, 24(6), 687–692. doi:10.1016/j.cub.2014.02.010
189. Striem-Amit, E., Dakwar, O., Hertz, U., Meijer, P., Stern, W., Pascual-Leone, A., & Amedi, A., (2011). The neural network of sensory-substitution object shape recognition. Functional Neurology, Rehabilitation, and Ergonomics, 1(2), 271–278.
190. Striem-Amit, E., Ovadia-Caro, S., Caramazza, A., Margulies, D. S., Villringer, A., & Amedi, A., (2015). Functional connectivity of visual cortex in the blind follows retinotopic organization principles. Brain, 138, 1679–1695. doi:10.1093/brain/awv083
191. Stropahl, M., Plotz, K., Schönfeld, R., Lenarz, T., Sandmann, P., Yovel, G., … Debener, S., (2015). Cross-modal reorganization in cochlear implant users:Auditory cortex contributes to visual face processing. NeuroImage, 121, 159–170. doi:10.1016/j.neuroimage.2015.07.062
192. Sumby, W. H., & Pollack, I., (1954). Visual contribution of speech intelligibility in noise. The Journal of the Acoustical Society of America, 26, 212–215. doi:10.1121/1.1907309
193. Summerfield, Q., (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In I., Barbara & C., Ruth (Eds.), Hearing by eye:The psychology of lip-reading (pp. 3–51). Hillsdale, NJ:Erlbaum.
194. Summerfield, Q., & McGrath, M., (1984). Detection and resolution of audio-visual incompatibility in the perception of vowels. The Quarterly Journal of Experimental Psychology Section A, 36, 51–74. doi:10.1080/14640748408401503
195. Sutherland, C. A. M., Thut, G., & Romei, V., (2014). Hearing brighter:Changing in-depth visual perception through looming sounds. Cognition, 132(3), 312–323. doi:10.1016/j.cognition.2014.04.011
196. Teinonen, T., Aslin, R. N., Alku, P., & Csibra, G., (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 108(3), 850–855. doi:10.1016/j.cognition.2008.05.009
197. Teng, S., Puri, A., & Whitney, D., (2012). Ultrafine spatial acuity of blind expert human echolocators. Experimental Brain Research, 216, 483–488. doi:10.1007/s00221-011-2951-1
198. Teramoto, W., Manaka, Y., Hidaka, S., Sugita, Y., Miyauchi, R., Sakamoto, S., … Suzuki, Y., (2010). Visual motion perception induced by sounds in vertical plane. Neuroscience Letters, 479, 221–225. doi:10.1016/j.neulet.2010.05.065
199. Thaler, L., (2013). Echolocation may have real-life advantages for blind people:An analysis of survey data. Frontiers in Physiology, 4, 1–8. doi:10.3389/fphys.2013.00098
200. Thaler, L., Arnott, S. R., & Goodale, M. A., (2011). Neural correlates of natural human echolocation in early and late blind echolocation experts. PLoS One, 6(5), e20162. doi:10.1371/journal.pone.0020162
201. Thompson, W. F., Russo, F. A., & Livingstone, S. R., (2010). Facial expressions of singers influence perceived pitch relations. Psychonomic Bulletin & Review, 17(3), 317–322. doi:10.3758/PBR.17.3.317
202. Thompson, W. F., Russo, F. A., & Quinto, L., (2008). Audio-visual integration of emotional cues in song. Cognition and Emotion, 22(8), 1457–1470. doi:10.1080/02699930701813974
203. Tiippana, K., (2014). What is the McGurk effect? Frontiers in Psychology, 5, 839. doi:10.3389/fpsyg.2014.00725
204. Treille, A., Cordeboeuf, C., Vilain, C., & Sato, M., (2014). Haptic and visual information speed up the neural processing of auditory speech in live dyadic interactions. Neuropsychologia, 57, 71–77. doi:10.1016/j.neuropsychologia.2014.02.004
205. Treille, A., Vilain, C., & Sato, M., (2014). The sound of your lips:Electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception. Frontiers in Psychology, 5, 195. doi:10.3389/fpsyg.2014.00420
206. Valente, D. L., & Braasch, J., (2010). Subjective scaling of spatial room acoustic parameters influenced by visual environmental cues. The Journal of the Acoustical Society of America, 128(4), 1952–1964. doi:10.1121/1.3478797
207. Vatakis, A., Ghazanfar, A. A., & Spence, C., (2008). Facilitation of multisensory integration by the “unity effect” reveals that speech is special. Journal of Vision, 8(9), 14–14. doi:10.1167/8.9.14
208. Vines, B. W., Krumhansl, C. L., Wanderley, M. M., Dalca, I. M., & Levitin, D. J., (2011). Music to my eyes:Cross-modal interactions in the perception of emotions in musical performance. Cognition, 118(2), 157–170. doi:10.1016/j.cognition.2010.11.010
209. Vroomen, J., & Baart, M., (2012). Phonetic recalibration in audiovisual speech. In M. M., Murray & M. T., Wallace (Eds.), The neural bases of multisensory processes (pp. 363–379). Boca Raton, FL:CRC Press.
210. Vroomen, J., & de Gelder, B., (2004). Perceptual effects of cross-modal stimulation:Ventriloquism and the freezing phenomenon. In G., Calvert, C., Spence, & B. E., Stein (Eds.), Handbook of multisensory processess (pp. 581–596). Cambridge, MA:MIT Press.
211. Vuoskoski, J. K., Thompson, M. R., Clarke, E. F., & Spence, C., (2014). Crossmodal interactions in the perception of expressivity in musical performance. Attention, Perception, & Psychophysics, 76(2), 591–604. doi:10.3758/s13414-013-0582-2
212. van Wassenhove, V., (2013). Speech through ears and eyes:Interfacing the senses with the supramodal brain. Frontiers in Psychology, 4, 1–17. doi:10.3389/fpsyg.2013.00388
213. van Wassenhove, V., Grant, K. W., & Poeppel, D., (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America, 102(4), 1181–1186. doi:10.1073/pnas.0408949102
214. Watkins, S., Shams, L., Tanaka, S., Haynes, J.-D., & Rees, G., (2006). Sound alters activity in human V1 in association with illusory visual perception. NeuroImage, 31(3), 1247–1256. doi:10.1016/j.neuroimage.2006.01.016
215. Weisser, V., Stilla, R., Peltier, S., Hu, X., & Sathian, K., (2005). Short-term visual deprivation alters neural processing of tactile form. Experimental Brain Research, 166(3–4), 572–582. doi:10.1007/s00221-005-2397-4
216. Williams, D. R., Verbrugge, R., & Studdert-Kennedy, M., (1983). Judging sine wave stimuli as speech and as nonspeech. The Journal of the Acoustical Society of America, 74(S1), S66. doi:10.1121/1.2021086
217. Yakel, D. A., (2000). Effects of time-varying information on vowel identification accuracy in visual speech perception (Ph.D.), University of California, Riverside.
218. Yehia, H. C., Kuratate, T., & Vatikiotis-Bateson, E., (2002). Linking facial animation, head motion, and speech acoustics. Journal of Phonetics, 30(3), 555–568. doi:10.1006/jpho.2002.0165
219. Yehia, H. C., Rubin, P., & Vatikiotis-Bateson, E., (1998). Quantitative association of vocal-tract and facial behavior. Speech Communication, 26(1–2), 23–43. doi:10.1016/S0167-6393(98)00048-X