Crossmodal Semantic Priming by Naturalistic Sounds and Spoken Words Enhances Visual Sensitivity

Journal of Experimental Psychology: Human Perception and Performance

Volumn 37, Issue 5, 2011, Pages 1554-1568

  Chen, Yi Chuan ^a   Spence, Charles ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Multisensory; Semantic congruency; Signal detection theory; Speech; Time course of audiovisual interaction

Indexed keywords

ADULT; ARTICLE; ASSOCIATION; AWARENESS; FEMALE; HEARING; HUMAN; MALE; PATTERN RECOGNITION; PERCEPTIVE DISCRIMINATION; REACTION TIME; RECOGNITION; SEMANTICS; SPEECH PERCEPTION; TIME PERCEPTION;

ASSOCIATION; AUDITORY PERCEPTION; AWARENESS; CUES; FEMALE; HUMANS; MALE; PATTERN RECOGNITION, VISUAL; REACTION TIME; RECOGNITION (PSYCHOLOGY); SEMANTICS; SIGNAL DETECTION, PSYCHOLOGICAL; SPEECH PERCEPTION; TIME PERCEPTION; YOUNG ADULT;

EID: 80053984328     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0024329     Document Type: Article

References (69)

1. Badgaiyan R.D., Schacter D.L., Alpert N.M. Priming within and across modalities: Exploring the nature of rCBF increases and decreases. Neuroimage 2001, 13:272-282.
2. Bates E., D'Amico S., Jacobsen T., Székely A., Andonova E., Devescovi A., Tzeng O. Timed picture naming in seven languages. Psychonomic Bulletin & Review 2003, 10:344-380.
3. Beauchamp M.S., Lee K.E., Argall B.D., Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41:809-823.
4. Buckner R.L., Koutstaal W., Schacter D.L., Rosen B.R. Functional MRI evidence for a role of frontal and inferior temporal cortex in amodal components of priming. Brain 2000, 123:620-640.
5. Chen Y-C., Spence C. When hearing the bark helps to identify the dog: Semantically-congruent sounds modulate the identification of masked pictures. Cognition 2010, 114:389-404.
6. Coltheart M. Dual routes from print to speech and dual routes from print to meaning: Some theoretical issues. Reading as a perceptual process 2000, 475-490. Elsevier, Oxford. A. Kennedy, R. Radach, J. Pynte, D. Heller (Eds.).
7. Cummings A., Ceponiene R., Koyama A., Saygin A.P., Townsend J., Dick F. Auditory semantic networks for words and natural sounds. Brain Research 2006, 1115:92-107.
8. De Lucia M., Clarke S., Murray M.M. A temporal hierarchy for conspecific vocalization discrimination in humans. Journal of Neuroscience 2010, 30:11210-11221.
9. Dick F., Bussiere J., Saygin A.P. The effects of linguistic mediation on the identification of environmental sounds. Center for Research in Language Newsletter 2002, 14:3-9.
10. Dick F., Saygin A.P., Galati G., Pitzalis S., Bentrovato S., D'Amico S., Pizzamiglio L. What is involved and what is necessary fro complex linguistic and nonlinguistic auditory processing: Evidence from functional magnetic resonance imaging and lesion data. Journal of Cognitive Neuroscience 2007, 19:799-816.
11. Doehrmann O., Naumer M.J. Semantics and the multisensory brain: How meaning modulates processes of audio-visual integration. Brain Research 2008, 1242:136-150.
12. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on " sensory specific" brain regions, neural responses, and judgments. Neuron 2008, 57:11-23.
13. Friedman D., Cycowicz Y.M., Dziobek I. Cross-form conceptual relations between sounds and words: Effects on the novelty P3. Cognitive Brain Research 2003, 18:58-64.
14. Fujisaki W., Nishida S. A common perceptual temporal limit of binding synchronous inputs across different sensory attributes and modalities. Proceedings of the Royal Society B 2010, 277:2281-2290.
15. Glaser W.R., Glaser M.O. Context effects in Stroop-like word and picture processing. Journal of Experimental Psychology: General 1989, 118:13-42.
16. Goll J.C., Crutch S.J., Loo J.H.Y., Rohrer J.D., Frost C., Bamiou D-E., Warren J.D. Non-verbal sound processing in the primary progressive aphasias. Brain 2010, 133:272-285.
17. Green D.M., Swets J.A. Signal detection theory and psychophysics 1966, Wiley, New York.
18. Iordanescu L., Grabowecky M., Franconeri S., Theeuwes J., Suzuki S. Characteristic sounds make you look at target objects more quickly. Attention, Perception, & Psychophysics 2010, 72:1736-1741.
19. Iordanescu L., Guzman-Martinez E., Grabowecky M., Suzuki S. Characteristic sound facilitates visual search. Psychonomic Bulletin & Review 2008, 15:548-554.
20. Jackson C.V. Visual factors in auditory localization. Quarterly Journal of Experimental Psychology 1953, 5:52-65.
21. Jesse A., Massaro D.W. The temporal distribution of information in audiovisual spoken-word identification. Attention, Perception, & Psychophysics 2010, 72:209-225.
22. Kirk R.E. Experimental design: Procedures for the behavioral sciences 1995, Brooks/Cole Publishing Company, Pacific Grove, CA.
23. La Heij W., van der Heijden A.H.C., Schreuder R. Semantic priming and Stroop-like interference in word-naming tasks. Journal of Experimental Psychology: Human Perception and Performance 1985, 11:62-80.
24. Laurienti P.J., Kraft R.A., Maldjian J.A., Burdette J.H., Wallace M.T. Semantic congruence is a critical factor in multisensory behavioral performance. Experimental Brain Research 2004, 158:405-414.
25. Macmillan N.A., Creelman C.D. Detection theory: A user's guide 2005, Erlbaum, Mahwah, NJ.
26. McLeod P., Posner M.I. Privileged loops from percept to act. Attention and performance X 1984, 55-66. Erlbaum, Hillsdale, NJ. H. Bouma, D.G. Bouwhuis (Eds.).
27. Miller J. Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology 1982, 14:247-279.
28. Mitterer H., Jesse A. Correlation versus causation in multisensory perception. Psychonomic Bulletin & Review 2010, 17:329-334.
29. Molholm S., Ritter W., Javitt D.C., Foxe J.J. Multisensory visual-auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex 2004, 14:452-465.
30. Murray M.M., Camen C., Gonzalez Andino S.L., Bovet P., Clarke S. Rapid brain discrimination of sounds of objects. Journal of Neuroscience 2006, 25:1293-1302.
31. Murray M.M., Michel C.M., Grave de Peralta R., Ortigue S., Brunet D., Gonzalez Andino S., Schnider A. Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. NeuroImage 2004, 21:125-135.
32. Murray M.M., Molholm S., Michel C.M., Heslenfeld D.J., Ritter W., Javitt D.C., Foxe J.J. Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex 2005, 15:963-974.
33. Murray M.M., Sperdin H.F. Single-trial multisensory learning and memory retrieval. Multisensory object perception in the primate brain 2010, 191-208. Springer, New York. M.J. Naumer, J. Kaiser (Eds.).
34. Murray M.M., Spierer L. Auditory spatio-temporal brain dynamics and their consequences for multisensory interactions in humans. Hearing Research 2009, 258:121-133.
35. Neely J.H. Semantic priming and retrieval from lexical memory: Roles of inhibitionless spreading activation and limited-capacity attention. Journal of Experimental Psychology: General 1977, 106:226-254.
36. Noppeney U., Josephs O., Hocking J., Price C.J., Friston K.J. The effect of prior visual information on recognition of speech and sounds. Cerebral Cortex 2008, 18:598-609.
37. Noppeney U., Ostwald D., Werner S. Perceptual decisions formed by accumulation of audiovisual evidence in prefrontal cortex. Journal of Neuroscience 2010, 30:7434-7446.
38. Orgs G., Lange K., Dombrowski J-H., Heil M. Is conceptual priming for environmental sounds obligatory?. International Journal of Psychophysiology 2007, 65:162-166.
39. Orgs G., Lange K., Dombrowski J-H., Heil M. N400-effects to task-irrelevant environmental sounds: Further evidence for obligatory conceptual processing. Neuroscience Letters 2008, 436:133-137.
40. Potter M.C. Meaning in visual search. Science 1975, 187:965-966.
41. Potter M.C. Very short-term conceptual memory. Memory & Cognition 1993, 21:156-161.
42. Potter M.C. Understanding sentences and scenes: The role of conceptual short-term memory. Fleeting memories: Cognition of brief visual stimuli 1999, 13-46. MIT Press, Cambridge, MA. V. Coltheart (Ed.).
43. Potter M.C., Faulconer B.A. Time to understand pictures and words. Nature 1975, 253:437-438.
44. Roelofs A. The visual-auditory color-word Stroop asymmetry and its time course. Memory & Cognition 2005, 33:1325-1336.
45. Saygin A.P., Dick F., Bates E. An on-line task for contrasting auditory processing in the verbal and nonverbal domains and norms for younger and older adults. Behavior Research Methods 2005, 37:99-110.
46. Saygin A.P., Dick F., Wilson S.W., Dronkers N.F., Bates E. Neural resources for processing language and environmental sounds. Brain 2003, 126:928-945.
47. Schneider T.R., Engel A.K., Debener S. Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Experimental Psychology 2008, 55:121-132.
48. Shimada H. Effect of auditory presentation of words on color naming: The intermodal Stroop effect. Perceptual & Motor Skills 1990, 70:1155-1161.
49. Smith M.C., Magee L.E. Tracing the time course of picture-word processing. Journal of Experimental Psychology: General 1980, 109:373-392.
50. Snodgrass J.G., Vanderwart M. A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory 1980, 6:174-215.
51. Soto-Faraco S., Alsius A. Deconstructing the McGurk-MacDonald illusion. Journal of Experimental Psychology: Human Perception and Performance 2009, 35:580-587.
52. Spence C. Audiovisual multisensory integration. Acoustical Science & Technology 2007, 28:61-70.
53. Stevenson R.A., James T.W. Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. NeuroImage 2009, 44:1210-1223.
54. Stuart D.M., Carrasco M. Semantic component of a cross-modal Stroop-like task. American Journal of Psychology 1993, 106:383-405.
55. Stuart G.P., Jones D.M. Priming the identification of environmental sounds. Quarterly Journal of Experimental Psychology 1995, 48A:741-761.
56. Stuart G.P., Jones D.M. From auditory image to auditory percept: Facilitation through common processes?. Memory & Cognition 1996, 24:296-304.
57. Suied C., Bonneel N., Viaud-Delmon I. Integration of auditory and visual information in the recognition of realistic objects. Experimental Brain Research 2009, 194:91-102.
58. Thierry G., Giraud A.L., Price C.J. Hemispheric dissociation in access to the human semantic system. Neuron 2003, 38:499-506.
59. Thierry G., Price C.J. Dissociating verbal and nonverbal conceptual processing in the human brain. Journal of Cognitive Neuroscience 2006, 18:1018-1028.
60. Thorpe S., Fize D., Marlot C. Speed of processing in the human visual system. Nature 1996, 381:520-522.
61. van Petten C., Rheinfelder H. Conceptual relationships between spoken words and environmental sounds: Event-related brain potential measures. Neuropsychologia 1995, 33:485-508.
62. van Wassenhove V., Grant K.W., Poeppel D. Temporal window of integration in audio-visual speech perception. Neuropsychologia 2007, 45:598-607.
63. Vatakis A., Ghazanfar A., Spence C. Facilitation of multisensory integration by the " unity assumption": Is speech special?. Journal of Vision 2008, 8(9):1-11. 14.
64. Vatakis A., Spence C. Crossmodal binding: Evaluating the " unity assumption" using audiovisual speech stimuli. Perception & Psychophysics 2007, 69:744-756.
65. Vatakis A., Spence C. Evaluating the influence of the " unity assumption" on the temporal perception of realistic audiovisual stimuli. Acta Psychologica 2008, 127:12-23.
66. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 1980, 3:638-667.
67. Werner S., Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cerebral Cortex 2010, 20:1829-1842.
68. Yuval-Greenberg S., Deouell L.Y. What you see is not (always) what you hear: Induced Gamma band responses reflect cross-modal interactions in familiar object recognition. Journal of Neuroscience 2007, 27:1090-1096.
69. Yuval-Greenberg S., Deouell L.Y. The dog's meow: Asymmetrical interaction in cross-modal object recognition. Experimental Brain Research 2009, 193:603-614.