Causal inference of asynchronous audiovisual speech

Frontiers in Psychology

Volumn 4, Issue NOV, 2013, Pages

  Magnotti, John F ^a   Ma, Wei Ji ^b,c   Beauchamp, Michael S ^a  

^a UNIVERSITY OF TEXAS MEDICAL SCHOOL   (United States)

^b BAYLOR COLLEGE OF MEDICINE   (United States)

^c NEW YORK UNIVERSITY   (United States)

Author keywords

Bayesian observer; Causal inference; Multisensory integration; Speech perception; Synchrony judgments

Indexed keywords

EID: 84889650789     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2013.00798     Document Type: Article

References (37)

1. Baskent, D., and Bazo, D. (2011). Audiovisual asynchrony detection and speech intelligibility in noise with moderate to severe sensorineural hearing impairment. Ear Hear. 32, 582-592. doi: 10.1097/AUD.0b013e31820fca23
2. Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., and Martin, A. (2004). Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 7, 1190-1192. doi: 10.1038/nn1333
3. Bejjanki, V. R., Clayards, M., Knill, D. C., and Aslin, R. N. (2011). Cue integration in categorical tasks: insights from audio-visual speech perception. PLoS ONE 6:e19812. doi: 10.1371/journal.pone.0019812
4. Brainard, D. H. (1997). The psychophysics toolbox. Spat. Vis. 10, 433-436. doi: 10.1163/156856897X00357
5. Buehner, M. J. (2012). Understanding the past, predicting the future: causation, not intentional action, is the root of temporal binding. Psychol. Sci. 23, 1490-1497. doi: 10.1177/0956797612444612
6. Conrey, B., and Pisoni, D. B. (2006). Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. J. Acoust. Soc. Am. 119, 4065-4073. doi: 10.1121/1.2195091
7. Conrey, B. L., and Pisoni, D. B. (2004). "Detection of auditory-visual asynchrony in speech and nonspeech signals," in Research on Spoken Language Processing Progress Report No 26, (Bloomington, IN: Indiana University), 71-94.
8. Foss-Feig, J. H., Kwakye, L. D., Cascio, C. J., Burnette, C. P., Kadivar, H., Stone, W. L., et al. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Exp. Brain Res. 203, 381-389. doi: 10.1007/s00221-010-2240-4
9. Hillock, A. R., Powers, A. R., and Wallace, M. T. (2011). Binding of sights and sounds: age-related changes in multisensory temporal processing. Neuropsychologia 49, 461-467. doi: 10.1016/j.neuropsychologia.2010.11.041
10. Hillock-Dunn, A., and Wallace, M. T. (2012). Developmental changes in the multisensory temporal binding window persist into adolescence. Dev. Sci. 15, 688-696. doi: 10.1111/j.1467-7687.2012.01171.x
11. Kording, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., and Shams, L. (2007). Causal inference in multisensory perception. PLoS ONE 2:e943. doi: 10.1371/journal.pone.0000943
12. Lachs, L., and Hernandez, L. R. (1998). "Update: the Hoosier audiovisual multitalker database," in Research on Spoken Language Processing Progress Report No 22, (Bloomington, IN), 377-388.
13. Loftus, G. R., and Masson, M. E. (1994). Using confidence intervals in within-subject designs. Psychon. Bull. Rev. 1, 476-490. doi: 10.3758/BF03210951
14. Love, S. A., Petrini, K., Cheng, A., and Pollick, F. E. (2013). A psychophysical investigation of differences between synchrony and temporal order judgments. PLoS ONE 8:e54798. doi: 10.1371/journal.pone.0054798
15. Ma, W. J. (2012). Organizing probabilistic models of perception. Trends Cogn. Sci. 16, 511-518. doi: 10.1016/j.tics.2012.08.010
16. Ma, W. J., Zhou, X., Ross, L. A., Foxe, J. J., and Parra, L. C. (2009). Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space. PLoS ONE 4:e4638. doi: 10.1371/journal.pone.0004638
17. Massaro, D. W. (1989). Testing between the TRACE model and the fuzzy logical model of speech perception. Cogn. Psychol. 21, 398-421. doi: 10.1016/0010-0285(89)90014-5
18. Massaro, D. W., Cohen, M. M., Campbell, C. S., and Rodriguez, T. (2001). Bayes factor of model selection validates FLMP. Psychon. Bull. Rev. 8, 1-17. doi: 10.3758/BF03196136
19. Navarra, J., Alsius, A., Velasco, I., Soto-Faraco, S., and Spence, C. (2010). Perception of audiovisual speech synchrony for native and non-native language. Brain Res. 1323, 84-93. doi: 10.1016/j.brainres.2010.01.059
20. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10, 437-442. doi: 10.1163/156856897X00366
21. Powers, A. R. 3rd., Hillock, A. R., and Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29, 12265-12274. doi: 10.1523/JNEUROSCI.3501-09.2009
22. R Core Team. (2012). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.
23. Rosenblum, L. D., Johnson, J. A., and Saldana, H. M. (1996). Point-light facial displays enhance comprehension of speech in noise. J. Speech Hear. Res. 39, 1159-1170.
24. Rouger, J., Fraysse, B., Deguine, O., and Barone, P. (2008). McGurk effects in cochlear-implanted deaf subjects. Brain Res. 1188, 87-99. doi: 10.1016/j.brainres.2007.10.049
25. Sato, Y., Toyoizumi, T., and Aihara, K. (2007). Bayesian inference explains perception of unity and ventriloquism aftereffect: identification of common sources of audiovisual stimuli. Neural Comput. 19, 3335-3355. doi: 10.1162/neco.2007.19.12.3335
26. Schutz, M., and Kubovy, M. (2009). Causality and cross-modal integration. J. Exp. Psychol. Hum. Percept. Perform. 35, 1791-1810. doi: 10.1037/a0016455
27. Schwartz, J. L., Berthommier, F., and Savariaux, C. (2004). Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 93, B69-B78. doi: 10.1016/j.cognition.2004.01.006
28. Shams, L., and Beierholm, U. R. (2010). Causal inference in perception. Trends Cogn. Sci. 14, 425-432. doi: 10.1016/j.tics.2010.07.001
29. Smith, E. G., and Bennetto, L. (2007). Audiovisual speech integration and lipreading in autism. J. Child Psychol. Psychiatry 48, 813-821. doi: 10.1111/j.1469-7610.2007.01766.x
30. Stevenson, R. A., Altieri, N. A., Kim, S., Pisoni, D. B., and James, T. W. (2010). Neural processing of asynchronous audiovisual speech perception. Neuroimage 49, 3308-3318. doi: 10.1016/j.neuroimage.2009.12.001
31. Stevenson, R. A., and Wallace, M. T. (2013). Multisensory temporal integration: task and stimulus dependencies. Exp. Brain Res. 227, 249-261. doi: 10.1007/s00221-013-3507-3
32. Stevenson, R. A., Wilson, M. M., Powers, A. R., and Wallace, M. T. (2013). The effects of visual training on multisensory temporal processing. Exp. Brain Res. 225, 479-489. doi: 10.1007/s00221-012-3387-y
33. Stevenson, R. A., Zemtsov, R. K., and Wallace, M. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. J. Exp. Psychol. Hum. Percept. Perform. 38, 1517-1529. doi: 10.1037/a0027339
34. Sumby, W. H., and Pollack, I. (1954). Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26, 212-215. doi: 10.1121/1.1907309
35. Vroomen, J., and Keetels, M. (2010). Perception of intersensory synchrony: a tutorial review. Atten. Percept. Psychophys. 72, 871-884. doi: 10.3758/APP.72.4.871
36. Wallace, M. T., Roberson, G. E., Hairston, W. D., Stein, B. E., Vaughan, J. W., and Schirillo, J. A. (2004). Unifying multisensory signals across time and space. Exp. Brain Res. 158, 252-258. doi: 10.1007/s00221-004-1899-9
37. Zatorre, R. J., Bouffard, M., Ahad, P., and Belin, P. (2002). Where is 'where' in the human auditory cortex? Nat. Neurosci. 5, 905-909. doi: 10.1038/nn904