Multisensory interactions between auditory and haptic object recognition

Cerebral Cortex

Volumn 23, Issue 5, 2013, Pages 1097-1107

  Kassuba, Tanja ^a,b,c   Menz, Mareike M ^b   Röder, Brigitte ^d   Siebner, Hartwig R ^a,b,c  

^a HVIDOVRE HOSPITAL   (Denmark)

^b UNIVERSITY MEDICAL CENTER HAMBURG EPPENDORF   (Germany)

^c UNIVERSITY OF KIEL   (Germany)

^d UNIVERSITY OF HAMBURG   (Germany)

Author keywords

auditory perception; functional magnetic resonance imaging; multisensory interactions; object recognition; touch perception

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BRAIN FUNCTION; CONTROLLED STUDY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUSIFORM GYRUS; HEARING; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; MALE; NORMAL HUMAN; PRIORITY JOURNAL; RECOGNITION; SOMATOSENSORY CORTEX; SUPERIOR TEMPORAL SULCUS; TACTILE STIMULATION; TASK PERFORMANCE; TOUCH;

ADULT; AUDITORY PERCEPTION; BRAIN MAPPING; CEREBRAL CORTEX; DECISION MAKING; FEMALE; HUMANS; MALE; NERVE NET; PATTERN RECOGNITION, PHYSIOLOGICAL; PERCEPTUAL MASKING; RECOGNITION (PSYCHOLOGY); TOUCH; YOUNG ADULT;

EID: 84876198383     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bhs076     Document Type: Article

References (65)

1. Adams RB, Janata P. 2002. A comparison of neural circuits underlying auditory and visual object categorization. Neuroimage. 16:361-377.
2. Ahveninen J, Jaaskelainen IP, Raij T, Bonmassar G, Devore S, Hamalainen M, Levanen S, Lin FH, Sams M, Shinn-Cunningham BG, et al. 2006. Task-modulated "what" and "where" pathways in human auditory cortex. Proc Natl Acad Sci USA. 103:14608-14613.
3. Amedi A, Jacobson G, Hendler T, Malach R, Zohary E. 2002. Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb Cortex. 12:1202-1212.
4. Amedi A, Malach R, Hendler T, Peled S, Zohary E. 2001. Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci. 4:324-330.
5. Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P, Pascual-Leone A. 2007. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat Neurosci. 10:687-689.
6. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ. 2005. Functional imaging of human crossmodal identification and object recognition. Exp Brain Res. 166:559-571.
7. Annett M. 1970. A classification of hand preference by association analysis. Br J Psychol. 61:303-321.
8. Beauchamp MS. 2005. See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neuro-biol. 15:145-153.
9. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A. 2004. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci. 7:1190-1192.
10. Beauchamp MS, Lee KE, Argall BD, Martin A. 2004. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron. 41:809-823.
11. Beauchamp MS, Yasar NE, Frye RE, Ro T. 2008. Touch, sound and vision in human superior temporal sulcus. Neuroimage. 41:1011-1020.
12. Binkofski F, Buccino G, Zilles K, Fink GR. 2004. Supramodal representation of objects and actions in the human inferior temporal and ventral premotor cortex. Cortex. 40:159-161.
13. Cruikshank SJ, Weinberger NM. 1996. Receptive-field plasticity in the adult auditory cortex induced by Hebbian covariance. J Neurosci. 16:861-875.
14. Damasio AR. 1989. Time-locked multiregional retroactivation: a systems-level proposal for the neural substrates of recall and recognition. Cognition. 33:25-62.
15. Doehrmann O, Naumer MJ. 2008. Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res. 1242:136-150.
16. Doehrmann O, Weigelt S, Altmann CF, Kaiser J, Naumer MJ. 2010. Audio-visual functional magnetic resonance imaging adaptation reveals multisensory integration effects in object-related sensory cortices. J Neurosci. 30:3370-3379.
17. Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K. 2005. A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage. 25:1325-1335.
18. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM. 2002. Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol. 88:540-543.
19. Glascher J. 2009. Visualization of group inference data in functional neuroimaging. Neuroinformatics. 7:73-82.
20. Gold BT, Balota DA, Jones SJ, Powell DK, Smith CD, Andersen AH. 2006. Dissociation of automatic and strategic lexical-semantics: functional magnetic resonance imaging evidence for differing roles of multiple frontotemporal regions. J Neurosci. 26:6523-6532.
21. Goldberg RF, Perfetti CA, Schneider W. 2006. Perceptual knowledge retrieval activates sensory brain regions. J Neurosci. 26:4917-4921.
22. Grefkes C, Weiss PH, Zilles K, Fink GR. 2002. Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys. Neuron. 35:173-184.
23. Griffiths TD, Warren JD. 2004. What is an auditory object? Nat Rev Neurosci. 5:887-892.
24. Grill-Spector K, Henson R, Martin A. 2006. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn Sci. 10:14-23.
25. Hein G, Doehrmann O, Muller NG, Kaiser J, Muckli L, Naumer MJ. 2007. Object familiarity and semantic congruency modulate responses in cortical audio-visual integration areas. J Neurosci. 27:7881-7887.
26. Henson R, Penny W. 2005. ANOVAs and SPM. Technical report. London: Wellcome Department of Imaging Neuroscience. 1-24. Available from: http://www.fil.ion.ucl.ac.uk/~wpenny/publications/rik-anova.pdf (last accessed 8 March, 2012).
27. Ishai A, Ungerleider LG, Haxby JV. 2000. Distributed neural systems for the generation of visual images. Neuron. 28:979-990.
28. James TW, Gauthier I. 2003. Auditory and action semantic features activate sensory-specific perceptual brain regions. Curr Biol. 13:1792-1796.
29. James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA. 2002. Haptic study of three-dimensional objects activates extras-triate visual areas. Neuropsychologia. 40:1706-1714.
30. Kaas JH, Hackett TA. 2000. Subdivisions of auditory cortex and processing streams in primates. Proc Natl Acad Sci USA. 97:11793-11799.
31. Kassuba T, Klinge C, Holig C, Menz MM, Ptito M, Roder B, Siebner HR. 2011. The left fusiform gyrus hosts trisensory representations of manipulable objects. Neuroimage. 56:1566-1577.
32. Kayser C, Petkov CI, Augath M, Logothetis NK. 2005. Integration of touch and sound in auditory cortex. Neuron. 48:373-384.
33. Kellenbach ML, Brett M, Patterson K. 2001. Large, colorful, or noisy? Attribute-and modality-specific activations during retrieval of perceptual attribute knowledge. Cogn Affect Behav Neurosci. 1:207-221.
34. Kiefer M, Sim EJ, Herrnberger B, Grothe J, Hoenig K. 2008. The sound of concepts: four markers for a link between auditory and conceptual brain systems. J Neurosci. 28:12224-12230.
35. Kim J-K, Zatorre RJ. 2011. Tactile-auditory shape learning engages the lateral occipital complex. J Neurosci. 31:7848-7856.
36. Lacey S, Flueckiger P, Stilla R, Lava M, Sathian K. 2010. Object familiarity modulates the relationship between visual object imagery and haptic shape perception. Neuroimage. 49:1977-1990.
37. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT. 2004. Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res. 158:405-414.
38. Martin A. 2007. The representation of object concepts in the brain. Annu Rev Psychol. 58:25-45.
39. Martin A, Chao LL. 2001. Semantic memory and the brain: structure and processes. Curr Opin Neurobiol. 11:194-201.
40. Meredith MA. 2002. On the neuronal basis for multisensory convergence: a brief overview. Brain Res Cogn Brain Res. 14:31-40.
41. Mesulam MM. 1998. From sensation to cognition. Brain. 121:1013-1052.
42. Meyer K, Damasio A. 2009. Convergence and divergence in a neural architecture for recognition and memory. Trends Neurosci. 32:376-382.
43. Naumer MJ, Doehrmann O, Muller NG, Muckli L, Kaiser J, Hein G. 2009. Cortical plasticity of audio-visual object representations. Cereb Cortex. 19:1641-1653.
44. Naumer MJ, Ratz L, Yalachkov Y, Polony A, Doehrmann O, van de Ven V, Muller NG, Kaiser J, Hein G. 2010. Visuohaptic convergence in a corticocerebellar network. Eur J Neurosci. 31:1730-1736.
45. Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ. 2008. The effect of prior visual information on recognition of speech and sounds. Cereb Cortex. 18:598-609.
46. Noppeney U, Price CJ. 2003. Functional imaging of the semantic system: retrieval of sensory-experienced and verbally learned knowledge. Brain Lang. 84:120-133.
47. Noppeney U, Price CJ. 2004. Retrieval of abstract semantics. Neuroimage. 22:164-170.
48. O'Craven KM, Kanwisher N. 2000. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J Cogn Neurosci. 12:1013-1023.
49. Oldfield RC. 1971. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 9:97-113.
50. Poldrack RA, Wagner AD, Prull MW, Desmond JE, Glover GH, Gabrieli JD. 1999. Functional specialization for semantic and phonological processing in the left inferior prefrontal cortex. Neuroimage. 10:15-35.
51. Price CJ. 2000. The anatomy of language: contributions from functional neuroimaging. J Anat. 197:335-359.
52. Reed CL, Klatzky RL, Halgren E. 2005. What vs. where in touch: an fMRI study. Neuroimage. 25:718-726.
53. Renier LA, Anurova I, De Volder AG, Carlson S, VanMeter J, Rauschecker JP. 2009. Multisensory integration of sounds and vibrotactile stimuli in processing streams for "what" and "where". J Neurosci. 29:10950-10960.
54. Schneider TR, Lorenz S, Senkowski D, Engel AK. 2011. Gamma-band activity as a signature for cross-modal priming of auditory object recognition by active haptic exploration. J Neurosci. 31: 2502-2510.
55. Schurmann M, Caetano G, Hlushchuk Y, Jousmaki V, Hari R. 2006. Touch activates human auditory cortex. Neuroimage. 30:1325-1331.
56. Sestieri C, Di Matteo R, Ferretti A, Del Gratta C, Caulo M, Tartaro A, Olivetti Belardinelli M, Romani GL. 2006. "What" versus "where" in the audio-visual domain: an fMRI study. Neuroimage. 33:672-680.
57. Stein BE, Stanford TR. 2008. Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci. 9:255-266.
58. Stevenson RA, James TW. 2009. Audio-visual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage. 44:1210-1223.
59. Stevenson RA, Kim S, James TW. 2009. An additive-factors design to disambiguate neuronal and areal convergence: measuring multisen-sory interactions between audio, visual, and haptic sensory streams using fMRI. Exp Brain Res. 198:183-194.
60. Tal N, Amedi A. 2009. Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach. Exp Brain Res. 198:165-182.
61. Tanabe HC, Honda M, Sadato N. 2005. Functionally segregated neural substrates for arbitrary audio-visual paired-association learning. J Neurosci. 25:6409-6418.
62. van Atteveldt NM, Blau VC, Blomert L, Goebel R. 2010. fMR-adaptation indicates selectivity to audio-visual content congruency in distributed clusters in human superior temporal cortex. BMC Neurosci. 11:11.
63. Wagner AD, Schacter DL, Rotte M, Koutstaal W, Maril A, Dale AM, Rosen BR, Buckner RL. 1998. Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. Science. 281:1188-1191.
64. Wheatley T, Weisberg J, Beauchamp MS, Martin A. 2005. Automatic priming of semantically related words reduces activity in the fusiform gyrus. J Cogn Neurosci. 17:1871-1885.
65. Wheeler ME, Petersen SE, Buckner RL. 2000. Memory's echo: vivid remembering reactivates sensory-specific cortex. Proc Natl Acad Sci USA. 97:11125-11129.