Multisensory integration, learning, and the predictive coding hypothesis

Frontiers in Psychology

Volumn 5, Issue MAR, 2014, Pages

  Altieri, Nicholas ^a  

^a IDAHO STATE UNIVERSITY   (United States)

Author keywords

Audiovisual speech integration; Bayesian inference; EEG; Parallel models; Predictive coding

Indexed keywords

EID: 84897944068     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2014.00257     Document Type: Note

References (23)

1. Altieri, N., Pisoni, D. B., and Townsend, J. T. (2011). Behavioral, clinical, and neurobiological constraints on theories of audiovisual speech integration: a review and suggestions for new directions. Seeing Perceiving 24, 513-539. doi: 10.1163/187847611X595864.
2. Altieri, N., Stevenson, R. A., Wallace, M. T., and Wenger, M. J. (2013). Learning to associate auditory and visual stimuli: capacity and neural measures of efficiency. Brain Topogr. doi: 10.1007/s10548-013-0333-7. [Epub ahead of print].
3. Altieri, N., and Townsend, J. T. (2011). An assessment of behavioral dynamic information processing measures in audiovisual speech perception. Front. Psychol. 2:238. doi: 10.3389/fpsyg.2011.00238.
4. Conrey, B., and Pisoni, D. B. (2006). Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. J. Acoust. Soc. Am. 119, 4065. doi: 10.1121/1.2195091.
5. Evans, K. K., and Treisman, A. (2010). Natural cross-modal mappings between visual and auditory features. J. Vis. 10:6. doi: 10.1167/10.1.6.
6. Fujisaki, W., Shimojo, S., Kashino, M., and Nishida, S. (2004). Recalibration of audiovisual simultaneity. Nat. Neurosci. 7, 773-778. doi: 10.1038/nn1268.
7. Hillock, A. R., Powers, A. R., and Wallace, M. T. (2011). Binding of sights and sounds: age-related changes in audiovisual temporal processing. Neuropsychologia 49, 461-467. doi: 10.1016/j.neuropsychologia.2010.11.041.
8. Lewkowicz, D. J. (2000). The development of inter-sensory temporal perception: an epignetic systems/limitations view. Psychol. Bull. 162, 281-308. doi: 10.1037/0033-2909.126.2.281.
9. Massaro, D. W. (1987). "Speech perception by ear and eye," in Hearing by Eye: The Psychology of Lip-Reading, eds B. Dodd and R. Campbell (Hillsdale, NJ: Lawrence Erlbaum), 53-83.
10. McGurk, H., and MacDonald, J. W. (1976). Hearing lips and seeing voices. Nature 264, 746-748. doi: 10.1038/264746a0.
11. Nahorna, O., Berthommier, F., and Schwartz, J. L. (2012). Binding and unbinding the auditory and visual streams in the McGurk effect. J. Acoust. Soc. Am. 132, 1061-1077. doi: 10.1121/1.4728187.
12. Navarra, J., Alsius, A., Velasco, I., Soto-Faraco, S., and Spence, C. (2010). Perception of audiovisual speech synchrony for native and non-native language. Brain Res. 1323, 84-93. doi: 10.1016/j.brainres.2010.01.059.
13. Poeppel, D., Idsardi, W. J., and van Wassenhove, V. (2008). Speech perception at the interface of neurobiology and linguistics. Philos. Trans. R. Soc. Lond. B Biol. Sci. 363, 1071-1086. doi: 10.1098/rstb.2007.2160.
14. Powers, A. R. 3rd., Hillock, A. R., and Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29, 12265-12274. doi: 10.1523/JNEUROSCI.3501-09.2009.
15. Rosenblum, L., Schmuckler, M. A., and Johnson, J. A. (1997). The McGurk effect in infants. Percept. Psychophys. 59, 347-357. doi: 10.3758/BF03211902.
16. Schroeder, C., Lakatos, P., Kajikawa, Y., Partan, S., and Puce, A. (2008). Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci. 12, 106-113. doi: 10.1016/j.tics.2008.01.002.
17. Stekelenburg, J. J., and Vroomen, J. (2007). Neural correlates of multisensory integration of ecologically valid audiovisual events. J. Cogn. Neurosci. 19, 1964-1973. doi: 10.1162/jocn.2007.19.12.1964.
18. Stevenson, R. A., Zemtsov, R. K., and Wallace, M. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. J. Exp. Psychol. Hum. Percept. Perform. 38, 1517-1529. doi: 10.1037/a0027339.
19. Sumby, W. H., and Pollack, I. (1954). Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26, 212-215. doi: 10.1121/1.1907309.
20. Summerfield, Q. (1987). "Some preliminaries to a comprehensive account of audio-visual speech perception," in The Psychology of Lip-Reading, eds B. Dodd and R. Campbell (Hillsdale, NJ: LEA), 3-50.
21. van Wassenhove, V. (2013). Speech through ears and eyes: interfacing the senses with the supramodal brain. Front. Psychol. 4:388. doi: 10.3389/fpsyg.2013.00388.
22. van Wassenhove, V., Grant, K. W., and Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. U.S.A. 102, 1181-1186. doi: 10.1073/pnas.0408949102.
23. Vatakis, A., Navarra, J., Soto-Faraco, S., and Spence, C. (2007). Temporal recalibration during asynchronous audiovisual speech perception. Exp. Brain Res. 181, 173-181. doi: 10.1007/s00221-007-0918-z.