Single-trial multisensory memories affect later auditory and visual object discrimination

Cognition

Volumn 138, Issue , 2015, Pages 148-160

  Thelen, Antonia ^a,e   Talsma, Durk ^b   Murray, Micah M ^a,c,d,e  

^a UNIVERSITY OF LAUSANNE   (Switzerland)

^b GHENT UNIVERSITY   (Belgium)

^c University of Lausanne   (Switzerland)

^d LAUSANNE UNIVERSITY HOSPITAL   (Switzerland)

^e VANDERBILT UNIVERSITY MEDICAL CENTER   (United States)

Author keywords

Auditory; Implicit; Memory; Multisensory; Object recognition; Visual

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; AUDITORY DISCRIMINATION; AUDITORY OBJECT DISCRIMINATION; CLINICAL ARTICLE; CONTROLLED STUDY; FEMALE; HUMAN; MALE; MEMORY; PRIORITY JOURNAL; PROBABILITY; SEMANTICS; SHORT TERM MEMORY; SOUND; TASK PERFORMANCE; VISUAL DISCRIMINATION; VISUAL OBJECT DISCRIMINATION; HEARING; PERCEPTIVE DISCRIMINATION; VISION; YOUNG ADULT;

ADOLESCENT; ADULT; AUDITORY PERCEPTION; DISCRIMINATION (PSYCHOLOGY); FEMALE; HUMANS; MALE; MEMORY; MEMORY, SHORT-TERM; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84923918569     PISSN: 00100277     EISSN: 18737838     Source Type: Journal    
DOI: 10.1016/j.cognition.2015.02.003     Document Type: Article

References (75)

1. Altieri N., Stevenson R.A., Wallace M.T., Wenger M.J. Learning to associate auditory and visual stimuli: Behavioral and neural mechanisms. Brain Topography 2013.
2. Botvinick M.M., Cohen J.D., Carter C.S. Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences 2004, 8(12):539-546.
3. Breitmeyer B.G., Ogmen H. Recent models and findings in visual backward masking: A comparison, review, and update. Perception & Psychophysics 2000, 62(8):1572-1595.
4. Brunel L., Goldstone R.L., Vallet G., Riou B., Versace R. When seeing a dog activates the bark: Multisensory generalization and distinctiveness effects. Experimental Psychology 2013, 60(2):100-112.
5. Brunel L., Labeye E., Lesourd M., Versace R. The sensory nature of episodic memory: Sensory priming effects due to memory trace activation. Journal of Experimental Psychology: Learning, Memory, and Cognition 2009, 35(4):1081-1088.
6. Cappe C., Thelen A., Romei V., Thut G., Murray M.M. Looming signals reveal synergistic principles of multisensory interactions. Journal of Neuroscience 2012, 32:1171-1182.
7. Chen Y.C., Spence C. Crossmodal semantic priming by naturalistic sounds and spoken words enhances visual sensitivity. Journal of Experimental Psychology: Human Perception and Performance 2011, 37(5):1554-1568.
8. Cohen M.A., Horowitz T.S., Wolfe J.M. Auditory recognition memory is inferior to visual recognition memory. Proceedings of the National Academy of Sciences of the United States of America 2009, 106(14):6008-6010.
9. Crouzet S.M., Overgaard M., Busch N.A. The fastest saccadic responses escape visual masking. PLoS One 2014, 9(2):e87418.
10. De Lucia M., Clarke S., Murray M.M. A temporal hierarchy for conspecific vocalization discrimination in humans. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 2010, 30(33):11210-11221.
11. De Lucia M., Cocchi L., Martuzzi R., Meuli R.a., Clarke S., Murray M.M. Perceptual and semantic contributions to repetition priming of environmental sounds. Cerebral Cortex (New York, N.Y.: 1991) 2010, 20(7):1676-1684.
12. De Lucia M., Tzovara A., Bernasconi F., Spierer L., Murray M.M. Auditory perceptual decision-making based on semantic categorization of environmental sounds. NeuroImage 2012, 60(3):1704-1715.
13. Diamantopoulou S., Poom L., Klaver P., Talsma D. Visual working memory capacity and stimulus categories: A behavioral and electrophysiological investigation. Experimental Brain Research 2011, 209(4):501-513.
14. Donk M., van Zoest W. Effects of salience are short-lived. Psychological Science 2008, 19(7):733-739.
15. Donohue S.E., Todisco A.E., Woldorff M.G. The rapid distraction of attentional resources toward the source of incongruent stimulus input during multisensory conflict. Journal of Cognitive Neuroscience 2013, 25(4):623-635.
16. Egner T. Congruency sequence effects and cognitive control. Cognitive, Affective, & Behavioral Neuroscience 2007, 7(4):380-390.
17. Gibson J.R., Maunsell J.H. Sensory modality specificity of neural activity related to memory in visual cortex. Journal of Neurophysiology 1997, 78(3):1263-1275.
18. Goll J.C., Crutch S.J., Warren J.D. Central auditory disorders: Toward a neuropsychology of auditory objects. Current Opinion in Neurology 2010, 23(6):617-627.
19. Gottfried J.A., Smith A.P., Rugg M.D., Dolan R.J. Remembrance of odors past: Human olfactory cortex in cross-modal recognition memory. Neuron 2004, 42(4):687-695.
20. Griffiths T.D. Central auditory pathologies. British Medical Bulletin 2002, 63:107-120.
21. Hecht D., Reiner M., Karni A. Repetition priming for multisensory stimuli: Task-irrelevant and task-relevant stimuli are associated if semantically related but with no advantage over uni-sensory stimuli. Brain Research 2009, 1251:236-244.
22. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 1979, 6(2):65-70.
23. Intraub H. Presentation rate and the representation of briefly glimpsed pictures in memory. Journal of Experimental Psychology: Human Learning and Memory 1980, 6(1):1-12.
24. Intraub H. Conceptual masking: The effects of subsequent visual events on memory for pictures. Journal of Experimental Psychology: Learning, Memory, and Cognition 1984, 10(1):115-125.
25. Johansson B.B. Multisensory stimulation in stroke rehabilitation. Frontiers in Human Neuroscience 2012, 6(April):60. 10.3389/fnhum.2012.00060.
26. Joubert O.R., Fize D., Rousselet G.a., Fabre-Thorpe M. Early interference of context congruence on object processing in rapid visual categorization of natural scenes. Journal of Vision 2008, 8(13):11.1-11.18.
27. King J.A., Korb F.M., Egner T. Priming of control: Implicit contextual cuing of top-down attentional set. Journal of Neuroscience 2012, 32(24):8192-8200.
28. Kiss M., Eimer M. Faster target selection in preview visual search depends on luminance onsets: Behavioral and electrophysiological evidence. Attention, Perception & Psychophysics 2011, 73(6):1637-1642.
29. Knebel J.-F., Toepel U., Hudry J., Le Coutre J., Murray M.M. Generating controlled image sets in cognitive neuroscience research. Brain Topography 2008, 20(4):284-289.
30. Lehmann S., Murray M.M. The role of multisensory memories in unisensory object discrimination. Brain Research. Cognitive Brain Research 2005, 24(2):326-334.
31. Lindsay D.S., Jacoby L.L. Stroop process dissociations: The relationship between facilitation and interference. Journal of Experimental Psychology: Human Perception and Performance 1994, 20(2):219-234.
32. McDonald J.J., Stormer V.S., Martinez A., Feng W., Hillyard S.A. Salient sounds activate human visual cortex automatically. Journal of Neuroscience 2013, 33(21):9194-9201.
33. Meylan R.V., Murray M.M. Auditory-visual multisensory interactions attenuate subsequent visual responses in humans. NeuroImage 2007, 35(1):244-254.
34. Murray, M. M., & Sperdin, H. F. (2010). Single-trial multisensory learning and memory retrieval. In J. Kaiser, M. J. Naumer (Eds.), Multisensory object perception in the primate brain.
35. Murray M.M., Camen C., Gonzalez Andino S.L., Bovet P., Clarke S. Rapid brain discrimination of sounds of objects. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 2006, 26(4):1293-1302.
36. Murray M.M., Cappe C., Romei V., Martuzzi R., Thut G. Auditory-visual multisensory interactions in humans: A synthesis of findings from behavior, ERPs, fMRI, and TMS. The new handbook of multisensory processes 2012, 223-238. MIT Press, Cambridge, MA, USA. B.E. Stein (Ed.).
37. Murray M.M., De Santis L., Thut G., Wylie G.R. The costs of crossing paths and switching tasks between audition and vision. Brain and Cognition 2009, 69(1):47-55.
38. Murray M.M., Foxe J.J., Wylie G.R. The brain uses single-trial multisensory memories to discriminate without awareness. NeuroImage 2005, 27(2):473-478.
39. Murray M.M., Michel C.M., Grave de Peralta R., Ortigue S., Brunet D., Gonzalez Andino S., et al. Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. NeuroImage 2004, 21(1):125-135.
40. Nyberg L., Habib R., McIntosh A.R., Tulving E. Reactivation of encoding-related brain activity during memory retrieval. Proceedings of the National Academy of Sciences of the United States of America 2000, 97(20):11120-11124.
41. Oldfield R.C. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 1971, 9(1):97-113.
42. O'Mahony C., Newell F.N. Integration of faces and voices, but not faces and names, in person recognition. British Journal of Psychology 2012, 103(1):73-82.
43. Potter M.C. Short-term conceptual memory for pictures. Journal of Experimental Psychology: Human Learning and Memory 1976, 2(5):509-522.
44. Romei V., Murray M.M., Merabet L.B., Thut G. Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: Implications for multisensory interactions. Journal of Neuroscience 2007, 27:11465-11472.
45. Sarmiento B.R., Shore D.I., Milliken B., Sanabria D. Audiovisual interactions depend on context of congruency. Attention, Perception & Psychophysics 2012, 74(3):563-574.
46. Schall S., Kiebel S.J., Maess B., von Kriegstein K. Early auditory sensory processing of voices is facilitated by visual mechanisms. NeuroImage 2013, 77:237-245.
47. Schneider T.R., Debener S., Oostenveld R., Engel A.K. Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming. NeuroImage 2008, 42(3):1244-1254.
48. Schneider T.R., Engel A.K., Debener S. Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Experimental Psychology (formerly "Zeitschrift Für Experimentelle Psychologie") 2008, 55(2):121-132.
49. Schneider T.R., Lorenz S., Senkowski D., Engel A.K. Gamma-band activity as a signature for cross-modal priming of auditory object recognition by active haptic exploration. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 2011, 31(7):2502-2510.
50. Senkowski D., Schneider T.R., Tandler F., Engel A.K. Gamma-band activity reflects multisensory matching in working memory. Experimental Brain Research. Experimentelle Hirnforschung. Expérimentation Cérébrale 2009, 198(2-3):363-372.
51. Shams L., Seitz A.R. Benefits of multisensory learning [Review]. Trends in Cognitive Sciences 2008, 12(11):411-417.
52. Shams L., Wozny D.R., Kim R., Seitz A. Influences of multisensory experience on subsequent unisensory processing. Frontiers in Psychology 2011, 2:264.
53. Sheffert S.M., Olson E. Audiovisual speech facilitates voice learning. Perception & Psychophysics 2004, 66(2):352-362.
54. Snodgrass J.G., Vanderwart M. A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory 1980, 6(2):174-215.
55. Spierer L., De Lucia M., Bernasconi F., Grivel J., Bourquin N.M.-P., Clarke S., et al. Learning-induced plasticity in human audition: Objects, time, and space. Hearing Research 2011, 271(1):88-102.
56. Stein B.E., Meredith M.A. The merging of senses 1993, MIT Press, Cambridge, MA.
57. Stevenson R.A., Ghose D., Fister J.K., Sarko D.K., Altieri N.A., Nidiffer A.R., et al. Identifying and quantifying multisensory integration: A tutorial review. Brain Topography 2014.
58. Suied C., Viaud-Delmon I. Auditory-visual object recognition time suggests specific processing for animal sounds. PLoS One 2009, 4(4):e5256.
59. Talsma D., Doty T.J., Strowd R., Woldorff M.G. Attentional capacity for processing concurrent stimuli is larger across sensory modalities than within a modality. Psychophysiology 2006, 43(6):541-549.
60. Thelen A., Cappe C., Murray M.M. Electrical neuroimaging of memory discrimination based on single-trial multisensory learning. NeuroImage 2012, 62(3):1478-1488.
61. Thelen A., Matusz P.J., Murray M.M. Multisensory context portends object memory. Current Biology 2014, 24(16):R734-R735.
62. Thelen A., Murray M.M. The efficacy of single-trial multisensory memories. Multisensory Research 2013.
63. Van Atteveldt N., Murray M.M., Thut G., Schroeder C.E. Multisensory integration: Flexible use of general operations. Neuron 2014, 81(6):1240-1253.
64. Van den Brink R.L., Cohen M.X., Van der Burg E., Talsma D., Vissers M.E., Slagter H.a. Subcortical, modality-specific pathways contribute to multisensory processing in humans. Cerebral Cortex 2013.
65. Van der Burg E., Olivers C.N.L., Bronkhorst A.W., Theeuwes J. Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance 2008, 34(5):1053-1065.
66. Van der Burg E., Talsma D., Olivers C.N.L., Hickey C., Theeuwes J. Early multisensory interactions affect the competition among multiple visual objects. NeuroImage 2011, 55(3):1208-1218.
67. Vetter P., Smith F.W., Muckli L. Decoding sound and imagery content in early visual cortex. Current Biology: CB 2014, 24(11):1256-1262.
68. von Kriegstein K., Giraud A.L. Implicit multisensory associations influence voice recognition. PLoS Biology 2006, 4(10):e326.
69. von Kriegstein K., Kleinschmidt A., Sterzer P., Giraud A.L. Interaction of face and voice areas during speaker recognition. Journal of Cognitive Neuroscience 2005, 17(3):367-376.
70. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 1980, 88(3):638-667.
71. Wheeler M.E., Petersen S.E., Buckner R.L. Memory's echo: Vivid remembering reactivates sensory-specific cortex. Proceedings of the National Academy of Sciences of the United States of America 2000, 97(20):11125-11129.
72. White-Traut R., Norr K.F., Fabiyi C., Rankin K.M., Li Z., Liu L. Mother-infant interaction improves with a developmental intervention for mother-preterm infant dyads. Infant Behavior & Development 2013, 36(4):694-706.
73. Yuval-Greenberg S., Deouell L.Y. What you see is not (always) what you hear: Induced gamma band responses reflect cross-modal interactions in familiar object recognition. Journal of Neuroscience 2007, 27(5):1090-1096.
74. Yuval-Greenberg S., Deouell L.Y. The dog's meow: Asymmetrical interaction in cross-modal object recognition. Experimental Brain Research 2009, 193(4):603-614.
75. Zimmer U., Roberts K.C., Harshbarger T.B., Woldorff M.G. Multisensory conflict modulates the spread of visual attention across a multisensory object. NeuroImage 2010, 52(2):606-616.