Cue integration in categorical tasks: Insights from audio-visual speech perception

PLoS ONE

Volumn 6, Issue 5, 2011, Pages

  Bejjanki, Vikranth Rao ^a   Clayards, Meghan ^a,b   Knill, David C ^a   Aslin, Richard N ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUDIO VISUAL SPEECH PERCEPTION; AUDITORY STIMULATION; BAYES THEOREM; CONTROLLED STUDY; HUMAN; HUMAN EXPERIMENT; MATHEMATICAL ANALYSIS; MENTAL PERFORMANCE; NORMAL HUMAN; PHONEME; QUANTITATIVE ANALYSIS; SENSORIMOTOR INTEGRATION; SPEECH ARTICULATION; SPEECH PERCEPTION; VISION; VISUAL STIMULATION; ASSOCIATION; BIOLOGICAL MODEL; HEARING; NORMAL DISTRIBUTION; PHOTOSTIMULATION; PHYSIOLOGY; STATISTICAL MODEL; TASK PERFORMANCE;

AUDITORY PERCEPTION; CUES; HUMANS; LIKELIHOOD FUNCTIONS; MODELS, BIOLOGICAL; NORMAL DISTRIBUTION; PHOTIC STIMULATION; SPEECH PERCEPTION; TASK PERFORMANCE AND ANALYSIS; VISUAL PERCEPTION;

EID: 79957551359     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0019812     Document Type: Article

References (48)

1. Ernst MO, Bülthoff HH, (2004) Merging the senses into a robust percept. Trends in Cognitive Sciences 8: 162-169.
2. Jacobs RA, (2002) What determines visual cue reliability? Trends in Cognitive Sciences 6: 345-350.
3. Campbell R, (2008) The processing of audio-visual speech: empirical and neural bases. Philosophical Transactions of the Royal Society B: Biological Sciences 363: 1001-1010.
4. Ma WJ, Zhou X, Ross LA, Foxe JJ, Parra LC, (2009) Lip-Reading Aids Word Recognition Most in Moderate Noise: A Bayesian Explanation Using High-Dimensional Feature Space. PLoS ONE 4: e4638.
5. Massaro DW, (1987) Speech perception by ear and eye: A paradigm for psychological inquiry: Erlbaum Hillsdale, NJ.
6. Massaro DW, Cohen MM, (1983) Evaluation and integration of visual and auditory information in speech perception. Journal of Experimental Psychology Human Perception and Performance 9: 753-771.
7. McGurk H, MacDonald J, (1976) Hearing lips and seeing voices. Nature 264: 746-748.
8. Ross LA, Saint-Amour D, Leavitt VM, Javitt DC, Foxe JJ, (2007) Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex 17: 1147-1147.
9. Sumby WH, Pollack I, (1954) Visual Contribution to Speech Intelligibility in Noise. The Journal of the Acoustical Society of America 26: 212-215.
10. Summerfield Q, (1992) Lipreading and Audio-Visual Speech Perception. Philosophical Transactions: Biological Sciences 335: 71-78.
11. Desjardins RN, Rogers J, Werker JF, (1997) An Exploration of Why Preschoolers Perform Differently Than Do Adults in Audiovisual Speech Perception Tasks. Journal of Experimental Child Psychology 66: 85-110.
12. Erber NP, (1971) Auditory and audiovisual reception of words in low-frequency noise by children with normal hearing and by children with impaired hearing. Journal of Speech and Hearing Research 14: 496-512.
13. Massaro DW, (1984) Children's Perception of Visual and Auditory Speech. Child Development 55: 1777-1788.
14. Massaro DW, Cohen MM, Gesi A, Heredia R, Tsuzaki M, (1993) Bimodal speech perception: An examination across languages. Journal of Phonetics 21: 445-478.
15. Sekiyama K, (1997) Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects. Perception and Psychophysics 59: 73-80.
16. Sekiyama K, Burnham D, (2008) Impact of Language on Development of Auditory-Visual Speech Perception. Developmental Science 11: 306-320.
17. Desjardins RN, Werker JF, (2004) Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology 45: 187-203.
18. Rosenblum LD, Schmuckler MA, Johnson JA, (1997) The McGurk effect in infants. Perception & Psychophysics 59: 347-357.
19. Teinonen T, Aslin RN, Alku P, Csibra G, (2008) Visual speech contributes to phonetic learning in 6-month-old infants. Cognition 108: 850-855.
20. Ernst MO, Banks MS, (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415: 429-433.
21. Knill DC, Pouget A, (2004) The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci 27: 712-719.
22. Yuille AL, Bulthoff HH, (1996) Bayesian decision theory and psychophysics. Perception as Bayesian Inference pp. 123-161.
23. Landy MS, Maloney LT, Johnston EB, Young M, (1995) Measurement and modeling of depth cue combination: In defense of weak fusion. Vision Research 35: 389-412.
24. Alais D, Burr D, (2004) The Ventriloquist Effect Results from Near-Optimal Bimodal Integration. Current Biology 14: 257-262.
25. Battaglia PW, Jacobs RA, Aslin RN, (2003) Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A 20: 1391-1397.
26. Hillis JM, Ernst MO, Banks MS, Landy MS, (2002) Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses. Science 298: 1627-1630.
27. Jacobs RA, (1999) Optimal integration of texture and motion cues to depth. Vision Research 39: 3621-3629.
28. Knill DC, Saunders JA, (2003) Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Research 43: 2539-2558.
29. Kording KP, Wolpert DM, (2004) Bayesian integration in sensorimotor learning. Nature 427: 244-247.
30. van Beers RJ, Sittig AC, Gon JJDvd, (1999) Integration of Proprioceptive and Visual Position-Information: An Experimentally Supported Model. J Neurophysiol 81: 1355-1364.
31. Wolpert DM, Ghahramani Z, Jordan MI, (1995) An internal model for sensorimotor integration. Science 269: 1880-1882.
32. Feldman NH, Griffiths TL, Morgan JL, (2009) The influence of categories on perception: Explaining the perceptual magnet effect as optimal statistical inference. Psychological Review 116: 752-782.
33. Papoulis A, Pillai SU, (2002) Probability, random variables and stochastic processes: McGraw-Hill Education (India) Pvt Ltd.
34. Klatt DH, (1980) Software for a cascade/parallel formant synthesizer. Journal of the Acoustical Society of America 67: 971-995.
35. Cohen MM, Massaro DW, (1990) Synthesis of visible speech. Synthesis 22: 260-263.
36. Remez RE, Pardo JS, Piorkowski RL, Rubin PE, (2001) On the bistability of sine wave analogues of speech. Psychological Science:.
37. Massaro DW, (1998) Perceiving talking faces: From speech perception to a behavioral principle: The MIT Press.
38. Swanson WH, Birch EE, (1992) Extracting thresholds from noisy psychophysical data. Perception & Psychophysics 51: 409-422.
39. Körding KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB, et al. (2007) Causal Inference in Multisensory Perception. PLoS ONE 2: e943.
40. Shams L, Ma WJ, Beierholm U, (2005) Sound-induced flash illusion as an optimal percept. Neuroreport 16: 1923-1927.
41. Wallace MT, Roberson GE, Hairston WD, Stein BE, Vaughan JW, et al. (2004) Unifying multisensory signals across time and space. Experimental Brain Research 158: 252-258.
42. Banks MS, Backus BT, (1998) Extra-retinal and perspective cues cause the small range of the induced effect. Vision Research 38: 187-194.
43. Knill DC, (2007) Robust cue integration: A Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. Journal of Vision 7: 1-24.
44. Peterson GE, Barney HL, (1952) Control methods used in a study of the vowels. Journal of the Acoustical Society of America 24: 175-184.
45. Schwartz J-L, (2006) The 0/0 problem in the fuzzy-logical model of perception. The Journal of the Acoustical Society of America 120: 1795-1798.
46. Schwartz J-L, (2010) A reanalysis of McGurk data suggests that audiovisual fusion in speech perception is subject-dependent. The Journal of the Acoustical Society of America 127: 1584-1594.
47. Holt LL, Lotto AJ, (2006) Cue weighting in auditory categorization: Implications for first and second language acquisition. The Journal of the Acoustical Society of America 119: 3059-3071.
48. Clayards M, Tanenhaus MK, Aslin RN, Jacobs RA, (2008) Speech perception reflects optimal use of probabilistic speech cues. Cognition 108: 804-809.