Multimodal access to verbal name codes

Perception and Psychophysics

Volumn 69, Issue 4, 2007, Pages 628-640

  Berryhill, Marian ^a,b   Kveraga, Kestutis ^a,c,d   Webb, Lisa ^a   Hughes, Howard C ^a  

^a DARTMOUTH COLLEGE   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c MASSACHUSETTS GENERAL HOSPITAL   (United States)

^d HARVARD MEDICAL SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34547838121     PISSN: 00315117     EISSN: None     Source Type: Journal    
DOI: 10.3758/BF03193920     Document Type: Article

References (112)

1. AMEDI, A., VON KRIEGSTEIN, K., VAN ATTEVELDT, N. M., BEAUCHAMP, M. S., & NAUMER, M. J. (2005). Functional imaging of human cross-modal identification and object recognition. Experimental Brain Research, 166, 559-571.
2. ASHBY, F. G., & TOWNSEND, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93, 154-179.
3. BEAUCHAMP, M. S., ARGALL, B. D., BODURKA, J., DUYN, J. H., & MARTIN, A. (2004). Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nature Neuroscience, 7, 1190-1192.
4. BELL, A. H., MEREDITH, M. A., VAN OPSTAL, A. J., & MUNOZ, D. P. (2005). Crossmodal integration in the primate superior colliculus underlying the preparation and initiation of saccadic eye movements. Journal of Neurophysiology, 93, 3659-3673.
5. BERNSTEIN, I. H., CHU, P. K., & BRIGGS, P. (1973). Stimulus intensity and foreperiod effects in intersensory facilitation. Quarterly Journal of Experimental Psychology, 25, 171-181.
6. BERRYHILL, M. E., KVERAGA, K., WEBB, L., & HUGHES, H. C. (2005). Effect of uncertainty on the time course for selection of verbal name codes. Perception A Psychophysics, 67, 1437-1445.
7. BILLINGSLEY, P. (1979). Probability and measure. New York: Wiley.
8. CABEZA, R., & NYBERG, L. (2000). Imaging cognition Il: An empirical review of 275 PET and fMRI studies. Journal of Cognitive Neuroscience, 12, 1-47.
9. CALVERT, G. A. (2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex, 11, 1110-1123.
10. CALVERT, G. A., BRAMMER, M. J., BULLMORE, E. T., CAMPBELL, R., IVERSEN, S. D., & DAVID, A. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding. NeuroReport, 10, 2619-2623.
11. CALVERT, G. A., BULLMORE, E. T., BRAMMER, M. J., CAMPBELL, R., WILLIAMS, S. C. R., MCG UIRE, P. K., ET AL. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593-596.
12. CALVERT, G. A., CAMPBELL, R., & BRAMMER, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10, 649-657.
13. CALVERT, G. A., & THESEN, T. (2004). Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology-Paris, 98, 191-205.
14. COLONIUS, H. (1990). Possibly dependent probability summation of reaction time. Journal of Mathematical Psychology, 34, 253-275.
15. COLONIUS, H., & ARNDT, P. (2001). A two-stage model for visual-auditory interaction in saccadic latencies. Perception A Psychophysics, 63, 126-147.
16. COLONIUS, H., & DIEDERICH, A. (2004). Why aren't all deep superior colliculus neurons multisensory? A Bayes' ratio analysis. Cognitive, Affective, A Behavioral Neuroscience, 4, 344-353.
17. COLONIUS, H., & DIEDERICH, A. (2006). The race model inequality: Interpreting a geometric measure of the amount of violation. Psychological Review, 113, 148-154.
18. DAVIS, R., MORAY, N., & TREISMAN, A. (1961). Imitative responses and the rate of gain of information. Quarterly Journal of Experimental Psychology, 13, 78-89.
19. DIEDERICH, A. (1995). Intersensory facilitation of reaction time: Evaluation of counter and diffusion coactivation models. Journal of Mathematical Psychology, 39, 197-215.
20. DIEDERICH, A., & COLONIUS, H. (1987). Intersensory facilitation in the motor component? A reaction time analysis. Psychological Research, 49,23-29.
21. DIEDERICH, A., & COLONIUS, H. (1991). A further test of the superposition model for the redundant-signals effect in bimodal detection. Perception A Psychophysics, 50, 83-86.
22. DIEDERICH, A., & COLONIUS, H. (2004). Bimodal and trimodal multi-sensory enhancement: Effects of stimulus onset and intensity on reaction time. Perception A Psychophysics, 66, 1388-1404.
23. DIEDERICH, A., COLONIUS, H., BOCKHORST, D., & TABELING, S. (2003). Visual-tactile spatial interaction in saccade generation. Experimental Brain Research, 148, 328-337.
24. DIXON, N. F., & SPITZ, L. (1980). The detection of auditory visual desynchrony. Perception, 9, 719-721.
25. DODD, B. (1977). The role of vision in the perception of speech. Perception, 6, 31-40.
26. DOUBELL, T. P., SKALIORA, I., BARON, J., & KING, A. J. (2003). Functional connectivity between the superficial and deeper layers of the superior colliculus: An anatomical substrate for sensorimotor integration. Journal of Neuroscience, 23, 6596-6607.
27. DRÄGER, U. C., & HUBEL, D. H. (1975a). Physiology of visual cells in mouse superior colliculus and correlation with somatosensory and auditory input Nature, 253, 203-204.
28. DRÄGER, U. C., & HUBEL, D. H. (1975b). Responses to visual stimulation and relationship between visual, auditory, and somatosensory inputs in mouse superior colliculus. Journal of Neurophysiology, 38, 690-713.
29. DRÄGER, U. C., & HUBEL, D. H. (1976). Topography of visual and somatosensory projections to mouse superior colliculus. Journal of Neurophysiology, 39, 91-101.
30. EIMER, M. (2001). Crossmodal links in spatial attention between vision, audition, and touch: Evidence from event-related brain potentials. Neuropsychologia, 39, 1292-1303.
31. EIMER, M., & DRIVER, J. (2000). An event-related brain potential study of cross-modal links in spatial attention between vision and touch. Psychophysiology, 37, 697-705.
32. FALCHIER, A., CLAVAGNIER, S., BARONE, P., & KENNEDY, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22, 5749-5759.
33. FITTS, P. M. (1964). Perceptual-motor skills learning. In A. W. Melton (Ed.), Categories of human learning (pp. 243-285). New York: Academic Press.
34. FORT, A., DELPUECH, C., PERNIER, J., & GIARD, M.-H. (2002). Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cerebral Cortex, 12, 1031-1039.
35. FOXE, J. J., & SCHROEDER, C. E. (2005). The case for feedforward multisensory convergence during early cortical processing. NeuroReport, 16, 419-423.
36. FRENS, M. A., VAN OPSTAL, A. J., & VAN DER WILLIGEN, R. F. (1995). Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Perception A Psychophysics, 57, 802-816.
37. GESCHWIND, N. (1965). Disconnexion syndromes in animals and man: I. Bruin, 88, 237-294.
38. GIARD, M.-H., & PERONNET, F. (1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473-490.
39. GIELEN, S. C. A. M., SCHMIDT, R. A., & VAN DEN HEUVEL, P. J. M. (1983). On the nature of intersensory facilitation of reaction time. Perception A Psychophysics, 34, 161-168.
40. GIRAY, M., & ULRICH, R. (1993). Motor coactivation revealed by response force in divided and focused attention. Journal of Experimental Psychology: Human Perception A Performance, 6, 1278-1291.
41. GREEN, D. M., & SWETS, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.
42. GREEN, K. P., STEVENS, E. B., KUHL, P. K., & MELTZOFF, A. M. (1990). Exploring the basis of the "McGurk effect": Can perceivers combine information from a female face and a male voice? Journal of the Acoustical Society of America, 87, S125.
43. GRICE, G. R., CANHAM, L., & BOROUGHS, J. M. (1984). Combination rule for redundant information in reaction time tasks with divided attention. Perception A Psychophysics, 35, 451-463.
44. HEIM, S., OPITZ, B., MÜLLER, K., & FRIEDERICI, A. D. (2003). Phonological processing during language production: fMRI evidence for a shared production-comprehension network. Cognitive Brain Research, 16, 285-296.
45. HOLMES, N. P., & SPENCE, C. (2005). Multisensory integration: Space, time and superadditivity. Current Biology, 15, R762-R764.
46. HUGHES, H. C., NELSON, M. D., & ARONCHICK, D. M. (1998). Spatial characteristics of visual-auditory summation in human saccades. Vision Research, 38, 3955-3963.
47. HUGHES, H. C., NOZAWA, G., & KITTERLE, F. (1996). Global precedence, spatial frequency, channels, and the statistics of natural images. Journal of Cognitive Neuroscience, 8, 197-230.
48. HUGHES, H. C., REUTER-LORENZ, P. A., NOZAWA, G., & FENDRICH, R. (1994). Visual-auditory interactions in sensorimotor processing: Saccades versus manual responses. Journal of Experimental Psychology: Human Perception & Performance,20, 131-153.
49. HUGHES, H. C., & TOWNSEND, J. T. (1998). Varieties of binocular interaction in human vision. Psychological Science, 9, 53-60.
50. IACOBONI, M., WOODS, R. P., & MAZZIOTTA, J. C. (1998). Bimodal (auditory and visual) left frontoparietal circuitry for sensorimotor integration and sensorimotor learning. Brain, 121, 2135-2143.
51. ISA, T., & SASAKI, S. (2002). Brainstem control of head movements during orienting: Organization of the premotor circuits. Progress in Neurobiology, 66, 205-241.
52. JAY, M. F., & SPARKS, D. L. (1984). Auditory receptive fields in primate superior colliculus shift with changes in eye position. Nature, 309, 345-347.
53. JOASSIN, F., MAURAGE, P., BRUYER, R., CROMMELINCK, M., & CAMPANELLA, S. (2004). When audition alters vision: An event-related potential study of the cross-modal interactions between faces and voices. Neuroscience Letters, 369, 132-137.
54. KAYSER, C., PETKOV, C. I., AUGATH, M., & LOGOTHETIS, N. K. (2005). Integration of touch and sound in auditory cortex. Neuron, 48, 373-384.
55. KIM, Y. Y., LEE, B., SHIN, Y. W., KWON, J. S., & KIM, M. S. (2006). Activity of left inferior frontal gyrus related to word repetition effects: LORETA imaging with 128-channel EEG and individual MRI. Neuro-lmage, 29, 712-720.
56. KIMURA, A., & TAMAI, Y. (1992). Sensory response of cortical neurons in the anterior ectosylvian sulcus, including the area evoking eye movement Brain Research, 575, 181-186.
57. KRUMMENACHER, J., MÜLLER, H. J., & HELLER, D. (2001). Visual search for dimensionally redundant pop-out targets: Evidence for parallel-coactive processing of dimensions. Perception A Psychophysics, 63, 901-917.
58. LAURIENTI, P. J., BURDETTE, J. H., WALLACE, M. T., YEN, Y.-F., FIELD, A. S., & STEIN, B. E. (2002). Deactivation of sensory-specific cortex by cross-modal stimuli. Journal of Cognitive Neuroscience, 14, 420-429.
59. LAURIENTI, P. J., PERRAULT, T. J., STANFORD, T. R., WALLACE, M. T., & STEIN, B. E. (2005). On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Experimental Brain Research, 166, 289-297.
60. MACALUSO, E., GEORGE, N., DOLAN, R., SPENCE, C., & DRIVER, J. (2004). Spatial and temporal factors during processing of audiovisual speech: A PET study. Neurolmage, 21, 725-732.
61. MARINKOVIC, K., DHOND, R. P., DALE, A. M., GLESSNER, M., CARR, V., & HALGREN, E. (2003). Spatiotemporal dynamics of modality-specific and supramodal word processing. Neuron, 38, 487-497.
62. MAY, P. J. (2005). The mammalian superior colliculus: Laminar structure and connections. Progress in Brain Research, 151, 321-378.
63. MCGURK, H., & MACDONALD, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
64. MEREDITH, M. A., & STEIN, B. E. (1983). interactions among converging sensory inputs in the superior colliculus. Science, 221, 389-391.
65. MEREDITH, M. A., & STEIN, B. E. (1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Research, 365, 350-354.
66. MEREDITH, M. A., & STEIN, B. E. (1986b). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56, 640-662.
67. MEREDITH, M. A., & STEIN, B. E. (1996). Spatial determinants of multi-sensory integration in cat superior colliculus. Journal of Neurophysiology, 75, 1843-1857.
68. MESULAM, M.-M. (1998). From sensation to cognition. Brain, 121, 1013-1052.
69. MEYER, G. F., WUERGER, S. M., RÖHRBEIN, F., & ZETZSCHE, C. (2005). Low-level integration of auditory and visual motion signals requires spatial co-localisation. Experimental Brain Research, 166, 538-547.
70. MILLER, J. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14, 247-279.
71. MILLER, J. (1986). Timecourse of coactivation in bimodal divided attention. Perception A Psychophysics, 40, 331-343.
72. MILLER, L. M., & D'ESPOSITO, M. (2005). Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. Journal of Neuroscience, 25, 5884-5893.
73. MORDKOFF, J. T., & YANTIS, S. (1991). An interactive race model of divided attention. Journal of Experimental Psychology: Human Perception A Performance, 17, 520-538.
74. MOWBRAY, G. H. (1960). Choice reaction times for skilled responses. Quarterly Journal of Experimental Psychology, 7, 193-202.
75. MOWBRAY, G. H., & RHOADES, M. V. (1959). On the reduction of choice reaction times with practice. Quarterly Journal of Experimental Psychology, 6, 16-23.
76. MUNHALL, K. G., GRIBBLE, P., SACCO, L., & WARD, M. (1996). Temporal constraints on the McGurk effect. Perception A Psychophysics, 58, 351-362.
77. NAGY, M. E., & RUGG, M. D. (1989). Modulation of event-related potentials by word repetition: The effects of inter-item lag. Psychophysiology, 26, 431-436.
78. NEIL, P. A., CHEE-RUITER, C., SCHEIER, C., LEWKOWICZ, D. J., & SHIMOJO, S. (2006). Development of multisensory spatial integration and perception in humans. Developmental Science, 9, 454-464.
79. NICKERSON, R. S. (1973). Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychological Review, 80, 489-509.
80. NOZAWA, G., REUTER-LORENZ, P., & HUGHES, H. C. (1994). Parallel and serial processes in the human oculomotor system: Bimodal integration and express saccades. Biological Cybernetics, 72, 19-34.
81. OLSON, I. R., GATENBY. J. C., & GORE, J. G (2002). A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Cognitive Brain Research, 14, 129-138.
82. PATCHING, G. R., & QUINLAN, P. T. (2004). Cross-modal integration of simple auditory and visual events. Perception A Psychophysics, 66, 131-140.
83. PATTON, P., BELKACEM-BOUSSAID, K., & ANASTASIO, T. J. (2002). Multimodality in the superior colliculus: An information theoretic analysis. Cognitive Brain Research, 14, 10-19.
84. POPUUN, L. C., & YIN, T. C. (2002). Bimodal interactions in the superior colliculus of the behaving cat. Journal of Neuroscience, 22, 2826-2834.
85. RAAB, D. H. (1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24, 574-590.
86. RAH, T., UUTELA, K., & HARI, R. (2000). Audiovisual integration of letters in the human brain. Neuron, 28, 617-625.
87. REGAN, J. E. (1981). Automaticity and learning: Effects of familiarity on naming letters. Journal of Experimental Psychology: Human Perception A Performance, 7, 180-195.
88. SAITO, D. N., YOSHIMURA, K., KOCHIYAMA, T., OKADA, T., HONDA, M., & SADATO, N. (2005). Cross-modal binding and activated attentional networks during audio-visual speech integration: A functional MRI study. Cerebral Cortex, 15, 1750-1760.
89. SANTI, A., SERVOS, P., VATIKIOTIS-BATESON, E., KURATATE, T., & MUNHALL, K. (2003). Perceiving biological motion: Dissociating visible speech from walking. Journal of Cognitive Neuroscience, 15, 800-809.
90. SCHACTER, D. L., & BUCKNER, R. L. (1998). Priming and the brain. Neuron, 20, 185-195.
91. SCHNEIDER, K. A., & KASTNER, S. (2005). Visual responses of the human superior colliculus: A high-resolution functional magnetic resonance imaging study. Journal of Neurophysiology, 94, 2491-2503.
92. SPARKS, D. L. (1986). Translation of sensory signals into commands for control of saccadic eye movements: Role of primate superior colliculus. Physiology Review, 66, 118-171.
93. STEIN, B. E. (1978). Development and organization of multimodal representation in cat superior colliculus. Federation Proceedings, 37, 2240-2245.
94. STEIN, B. E. (1998). Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Experimental Brain Research, 123, 124-135.
95. STEIN, B. E., & ARIGBEDE, M. O. (1972). Unimodal and multimodal response properties of neurons in the cat's superior colliculus. Experimental Neurology, 36, 179-196.
96. STEIN, B. E., JIANG, W., WALLACE, M. T., & STANFORD, T. R. (2001). Nonvisual influences on visual-information processing in the superior colliculus. Progress in Brain Research, 134, 143-156.
97. STEIN, B. E., & MEREDITH, A. M. (1993). The merging of the senses. Cambridge, MA: MIT Press.
98. STEIN, B. E., WALLACE, M. T., STANFORD, T. R., & JIANG, W. (2002). Cortex governs multisensory integration in the midbrain. Neuroscientist, 8, 306-314.
99. SUMBY, W. H., & POLLACK, I. (1954). Visual contributions to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
100. TALSMA, D., & WOLDORFF, M. G. (2005). Selective attention and multisensory integration: Multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17, 1098-1114.
101. THEIOS, J. (1975). The components of response latency in simple human information processing tasks. In P. M. A. Rabbitt & S. Dornic (Eds.), Attention and performance V (pp. 418-440). New York: Academic Press.
102. TODD, J. W. (1912). Reaction to multiple stimuli (Archives of Psychology, Vol. 25). New York: Science Press.
103. TOLDI, J., FEHÉR, O., & WOLFF, J. R. (1986). Sensory interactive zones in the rat cerebral cortex. Neuroscience, 18, 461-465.
104. TOWNSEND, J. T., & ASHBY, F. G. (1983). Stochastic modeling of elementary psychological processes. Cambridge: Cambridge University Press.
105. TOWNSEND, J. T., & NOZAWA, G. (1995). On the spatio-temporal properties of elementary perception: An investigation of parallel, serial and coactive theories. Journal of Mathematical Psychology, 39, 321-360.
106. TOWNSEND, J. T., & WENGER, M. J. (2004). A theory of interactive paral lel processing: New capacity measures and predictions for a response time inequality series. Psychological Review, 111, 1003-1035.
107. TURKELTAUB, P. E., EDEN, G. F., JONES, K. M., & ZEFFIRO, T. (2002). Meta-analysis of the functional neuroanatomy of single-word reading: Method and validation. NeuroImage, 16, 765-780.
108. VAN ATTEVELDT, N. M., FORMISANO, E., BLOMERT, L., & GOEBEL, R. (2007). The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cerebral Cortex, 17, 962-974.
109. WALLACE, M. T., MEREDITH, M. A., & STEIN, B. E. (1992). Integration of multiple sensory modalities in cat cortex. Experimental Brain Research, 91, 484-488.
110. WALLACE, M. T., & STEM, B. E (2001). Sensory and multisensory responses in the newborn monkey superior colliculus. Journal of Neuroscience, 21, 8886-8894.
111. WOODWORTH, R. S., & SCHLOSBERG, H. (1954). Experimental psychology (Rev. ed.). New York: Holt, Rinehart & Winston.
112. WUERGER, S. M., HOFBAUER, M., & MEYER, G. F. (2003). The integration of auditory and visual motion signals at threshold. Perception A Psychophysics, 65, 1188-1196.