An assessment of behavioral dynamic information processing measures in audiovisual speech perception

Frontiers in Psychology

Volumn 2, Issue SEP, 2011, Pages

  Altieri, Nicholas ^a   Townsend, James T ^b  

^a UNIVERSITY OF OKLAHOMA   (United States)

^b INDIANA UNIVERSITY   (United States)

Author keywords

Capacity; Coactive; Multisensory integration; Parallel; Speech

Indexed keywords

EID: 84867137042     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2011.00238     Document Type: Article

References (57)

1. Altieri, N. (2010). Toward a Unified Theory of Audiovisual Integration in Speech Perception. Doctoral Dissertation, Indiana University, Bloomington, IN.
2. Altieri, N., and Wenger, M. J. (2011). "Neural and information processing measures of audiovisual integration, "in Conference of the Vision Sciences Society, Poster Presentation, Naples, FL.
3. Arnold, D. H., Tear, M., Schindel, R., and Roseboom, W. (2010). Audiovisual speech cue combination. PLoS ONE 5, e10217. doi: 10.1371/journal.pone.0010217
4. Barutchu, A., Crewther, D. P., and Crewther, S. G. (2009). The race that precedes coactivation: development of multisensory facilitation in children. Dev. Sci. 12, 464-473.
5. Barutchu, A., Danaher, J., Crewther, S. G., Innes-Brown, H., Shivdasani, M. N., and Paolini, A. G. (2010). Audiovisual integration in noise by children and adults. J. Exp. Child. Psychol. 105, 38-50.
6. Bergeson, T. R., and Pisoni, D. B. (2004). "Audiovisual speech perception in deaf adults and children following cochlear implantation, "in The Handbook of Multisensory Processes, eds G. A. Calvert, C. Spence, and B. E. Stein (Cambridge, MA: The MIT Press), 153-176.
7. Bernstein, L. E. (2005). "Phonetic perception by the speech perceiving brain, "in The Handbook of Speech Perception, eds D. B. Pisoni and R. E. Remez (Malden, MA: Blackwell Publishing), 79-98
8. Bernstein, L. E., Auer, E. T., and Moore, J. K. (2004). "Audiovisual speech binding: convergence or association?" in Handbook of Multisensory Processing, eds G. A. Calvert, C. Spence, and B. E. Stein (Cambridge, MA: MIT Press), 203-223
9. Berryhill, M., Kveraga, K., Webb, L., and Hughes, H. C. (2007). Multimodal access to verbal name codes. Percept. Psychophys. 69, 628-640
10. Braida, L. D. (1991). Crossmodal integration in the identification of consonant segments. Q. J. Exp. Psychol. 43A, 647-677.
11. Bundesen, C., and Habekost, T. (2009). Principles of Visual Attention: Linking Mind and Brain. Oxford: Oxford University Press.
12. Colonius, H. (1990). Possibly dependent probability summation of reaction time. J. Math. Psychol. 34, 253-275.
13. Eidels, A., Houpt, J., Altieri, N., Pei, L., and Townsend, J. T. (2011). Nice guys finish fast and bad guys finish last: A theory of interactive parallel processing. J. Math. Psychol. 55, 176-190.
14. Fournier, L. R., and Eriksen, C. W. (1990). Coactivation in the perception of redundant targets. J. Exp. Psychol. Hum. Percept. Perform. 16, 538-550.
15. Fowler, C. A., and Dekle, D. J. (1991). Listening with eye and hand: cross-modal contributions to speech perception. J. Exp. Psychol. Hum. Percept. Perform 17, 816-828.
16. Fowler, C. A., and Rosenblum, L. D. (1991). "Perception of the phonetic gesture, "in Modularity and the Motor Theory of Speech Perception, edsI.G. Mattingly and M. Studdert-Kennedy (Hillsdale, NJ: Lawrence Erlbaum), 33-59.
17. Grant, K. W. (2002). Measures of auditory-visual integration for speech understanding: a theoretical perspective. J. Acoust. Soc. Am. 112, 30-33
18. Grant, K. W., Walden, B. E., and Seitz, P. F. (1998). Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditoryvisual integration. J. Acoust. Soc. Am. 103, 2677-2690.
19. Green, K. P., and Miller, J. L. (1985). On the role of visual rate information in phonetic perception. Percept. Psychophys. 38, 269-276.
20. Grice, G. R., Canham, L., and Gwynne, J. W. (1984). Absence of a redundantsignals effect in a reaction time task with divided attention. Percept. Psychophys. 36, 565-570.
21. Jesse, A., and Massaro, D. W. (2010). The temporal distribution of information in audiovisual spoken-word identification. Atten. Percept. Psychophys. 72, 209-225.
22. Liberman, A. M., and Mattingly, I. G. (1985). The motor theory of speech perception. Cognition 21, 1-36.
23. Ma, W. J., Zhou, X., Ross, L. A., Foxe, J. J., and Parra, L. C. (2009). Lip reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space. PLoS ONE 4, e4638. doi: 10.1371/journal.pone. 0004638
24. Massaro, D. W. (1987). "Speech perception by ear and eye, "in Hearing by Eye: The Psychology of Lip-Reading, eds B. Dodd and R. Campbell (Hillsdale, NJ: Lawrence Erlbaum), 53-83.
25. Massaro, D. W. (2004). "From multisensory integration to talking heads and language learning, "in The Handbook of Multisensory Processes, eds G. A. Calvert, C. Spence, and B. E. Stein (Cambridge, MA: The MIT Press), 153-176
26. Massaro, D. W., and Cohen, M. M. (2000). Tests of auditory-visual integration efficiency within the framework of the fuzzy logical model of perception. J. Acoust. Soc. Am. 108, 784-789
27. Massaro, D. W., Cohen, M. M., and Smeele, P. M. T. (1996). Perception of asynchronous and conflicting visible and auditory speech. J. Acoust. Soc. Am. 100, 1777-1786.
28. McGurk, H., and MacDonald, J. W. (1976). Hearing lips and seeing voices. Nature 264, 746-748.
29. Meredith, M. A., and Stein, B. E. (1983). Interactions among converging sensory inputs in the superior colliculus. Science 221, 389-391.
30. Miller, J. (1982). Divided attention: evidence for coactivation with redundant signals. Cogn. Psychol. 14, 247-279.
31. Miller, J. (1986). Time course of coactivation in bimodal divided attention. Percept. Psychophys. 40, 331-343.
32. Molholm, S., Ritter, W., Javitt, D. C., and Foxe, J. J. (2004). Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb. Cortex 14, 452-465.
33. Pilling, M. (2009). Auditory eventrelated potentials (ERPs) in audiovisual speech perception. J. Speech Lang. Hear. Res. 52, 1073-1081.
34. Ponton, C. W., Bernstein, L. E., and Auer, E. T. (2009). Mismatch negativity with visual-only and audiovisual speech. Brain Topogr. 21, 207-215.
35. Raab, D. H. (1962). Statistical facilitation of simple reaction times. Trans. N. Y. Acad. Sci. 24, 574-590.
36. Rosenblum, L. D. (2005). "Primacy of multimodal speech perception, "in The Handbook of Speech Perception, eds D. B. Pisoni and R. E. Remez (Malden, MA: Blackwell Publishing), 51-78.
37. Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., and Foxe, J. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 17, 1147-1153
38. Sekiyama, K., and Tohkura, Y. (1993). Inter-language differences in the influence of visual cues in speech perception. J. Phon. 21, 427-444.
39. Sherffert, S., Lachs, L., and Hernandez, L. R. (1997). "The Hoosier audiovisual multi-talker database, "In Research on Spoken Language Processing Progress Report No. 21, Bloomington, IN: Speech Research Laboratory, Psychology (Department), Indiana University.
40. Sommers, M., Tye-Murray, N., and Spehar, B. (2005). Auditoryvisual speech perception and auditory-visual enhancement in normal-hearing younger and older adults. Ear Hear. 26, 263-275.
41. Sternberg, S. (1969). The discovery of processing stages: extensions of Donder's method. Acta Psychol. (Amst.) 30, 276-315.
42. Stevenson, R. A., and James, T. W. (2009). Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44, 12101223.
43. Sumby, W. H., and Pollack, I. (1954). Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26, 12-15.
44. Summerfield, Q. (1987). "Some preliminaries to a comprehensive account of audio-visual speech perception, "in The Psychology of Lip-Reading, eds B. Dodd and R. Campbell (Hillsdale, NJ: LEA), 3-50.
45. Townsend, J. T. (1971). A note on the identifiability of parallel and serial processes. Percept. Psychophys. 10, 161-163
46. Townsend, J. T. (1974). "Issues and models concerning the processing of a finite number of inputs, "in Human Information Processing: Tutorials in Performance and Cognition, ed. B. H. Kantowitz (Hillsdale, NJ: Erlbaum Press), 133-168.
47. Townsend, J. T. (1990). Truth and consequences of ordinal differences in statistical distributions: toward a theory of hierarchical inference. Psychol. Bull. 108, 551-567
48. Townsend, J. T., and Ashby, F. G. (1978). "Methods of modeling capacity in simple processing systems, "in Cognitive Theory, Vol. 3, eds J. Castellan and F. Restle (Hillsdale, NJ: Erlbaum Associates), 200-239.
49. Townsend, J. T., and Ashby, F. G. (1983). The Stochastic Modeling of Elementary Psychological Processes. Cambridge: Cambridge University Press. Townsend, J. T., and Nozawa, G. (1995). Spatio-temporal properties of elementary perception: an investigation of parallel, serial, and coactive theories. J. Math. Psychol. 39, 321-359
50. Townsend, J. T., and Schweickert, R. (1989). Toward the trichotomy method: laying the foundation of stochastic mental networks. J. Math. Psychol. 33, 309-327.
51. Townsend, J. T., and Wenger, M. J. (2004a). The serial-parallel dilemma: a case study in a linkage of theory and method. Psychon. Bull. Rev. 11, 391-418.
52. Townsend, J. T., and Wenger, M. J. (2004b). A theory of interactive parallel processing: new capacity measures and predictions for a response time inequality series. Psychol. Rev. 111, 1003-1035.
53. van Wassenhove, V., Grant, K. W., and Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. U.S.A. 102, 1181-1186.
54. Walker, S., Bruce, V., and O'Malley, C. (1995). Facial identity and facial speech processing: familiar faces and voices in the McGurk effect. Percept. Psychophys. 59, 1124-1133.
55. Wenger, M. J., and Townsend, J. T. (2000). Basic response time tools for studying general processing capacity in attention, perception, and cognition. J. Gen. Psychol. 127, 67-99.
56. Wenger, M. J., and Townsend, J. T. (2001). "Faces as gestalt stimuli: process characteristics, "in Computational, Geometric, and Process Perspectives on Facial Cognition, eds M. J. Wenger and J. T. Townsend (Mahwah, NJ: Erlbaum Press), 229-284.
57. Winneke, A. H., and Phillips, N. A. (2011). Does audiovisual speech Altieri and Townsend Audiovisual speech processing offer a fountain of youth for old ears? An event-related brain potential study of age differences in audiovisual speech perception. Psychol. Aging 26, 427-438.