Effects of experience, training and expertise on multisensory perception: Investigating the link between brain and behavior

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7403 LNCS, Issue , 2012, Pages 304-320

  Love, Scott A ^a   Pollick, Frank E ^b   Petrini, Karin ^c  

^a INDIANA UNIVERSITY   (United States)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

audiovisual; drumming; expertise; multisensory; perception

Indexed keywords

AUDIO-VISUAL SPEECH; AUDIOVISUAL; DRUMMING; EXPERTISE; MULTISENSORY; NEURAL MECHANISMS; NON-NATIVE; SENSORY INFORMATION;

AUDITION; SENSORY PERCEPTION;

COGNITIVE SYSTEMS;

EID: 84870354163     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-34584-5_27     Document Type: Conference Paper

References (103)

1. Landy, M.S., et al.: Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Res. 35(3), 389-412 (1995)
2. Ernst, M.O., Banks, M.S.: Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870), 429-433 (2002)
3. Green, C.S., Bavelier, D.: Action-video-game experience alters the spatial resolution of vision. Psychol. Sci. 18(1), 88-94 (2007)
4. Simmons, R.W., Locher, P.J.: Role of extended perceptual experience upon haptic perception of nonrepresentational shapes. Percept. Mot. Skills 48(3 Pt. 1), 987-991 (1979)
5. Kisilevsky, B.S., et al.: Effects of experience on fetal voice recognition. Psychol. Sci. 14(3), 220-224 (2003)
6. Atkins, J.E., Fiser, J., Jacobs, R.A.: Experience-dependent visual cue integration based on consistencies between visual and haptic percepts. Vision Res. 41(4), 449-461 (2001)
7. Jacobs, R.A., Fine, I.: Experience-dependent integration of texture and motion cues to depth. Vision Res. 39(24), 4062-4075 (1999)
8. Powers III, A.R., Hillock, A.R., Wallace, M.T.: Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29(39), 12265-12674 (2009)
9. Mamassian, P., Goutcher, R.: Prior knowledge on the illumination position. Cognition 81(1), B1-B9 (2001)
10. Mamassian, P., Landy, M.S.: Interaction of visual prior constraints. Vision Res. 41(20), 2653-2668 (2001)
11. Mondloch, C.J., et al.: Face perception during early infancy. Psychol. Sci. 10(5), 419-422 (1999)
12. Turati, C.: Why faces are not special to newborns: An alternative account of the face preference. Current Directions in Psychological Science 13(1), 5-8 (2004)
13. Hershber, W.: Attached-Shadow Orientation Perceived as Depth by Chickens Reared in an Environment Illuminated from Below. Journal of Comparative and Physiological Psychology 73(3), 407-&. (1970)
14. Gregory, R.L.: Knowledge in perception and illusion. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences 352(1358), 1121-1127 (1997)
15. Adams, W.J., Graf, E.W., Ernst, M.O.: Experience can change the 'light-from-above' prior. Nat. Neurosci. 7(10), 1057-1058 (2004)
16. Dayan, E., Cohen, L.G.: Neuroplasticity subserving motor skill learning. Neuron 72(3), 443-454 (2011)
17. May, A.: Experience-dependent structural plasticity in the adult human brain. Trends Cogn. Sci. 15(10), 475-482 (2011)
18. Pascual-Leone, A., et al.: The plastic human brain cortex. Annu. Rev. Neurosci. 28, 377-401 (2005)
19. Latinus, M., Crabbe, F., Belin, P.: Learning-induced changes in the cerebral processing of voice identity. Cereb Cortex 21(12), 2820-2828 (2011)
20. Bentin, S., et al.: Electrophysiological Studies of Face Perception in Humans. J. Cogn. Neurosci. 8(6), 551-565 (1996)
21. Rossion, B., et al.: Expertise training with novel objects leads to left-lateralized facelike electrophysiological responses. Psychol. Sci. 13(3), 250-257 (2002)
22. Bukach, C.M., et al.: Does acquisition of Greeble expertise in prosopagnosia rule out a domain-general deficit? Neuropsychologia 50(2), 289-304 (2012)
23. Tanaka, J.W., Curran, T.: A neural basis for expert object recognition. Psychol. Sci. 12(1), 43-47 (2001)
24. Busey, T.A., Vanderkolk, J.R.: Behavioral and electrophysiological evidence for configural processing in fingerprint experts. Vision Res. 45(4), 431-448 (2005)
25. Busey, T.A., Parada, F.J.: The nature of expertise in fingerprint examiners. Psychon. Bull. Rev. 17(2), 155-160 (2010)
26. Calvo-Merino, B., et al.: Action observation and acquired motor skills: an FMRI study with expert dancers. Cereb Cortex 15(8), 1243-1249 (2005)
27. Calvo-Merino, B., et al.: Seeing or doing? Influence of visual and motor familiarity in action observation. Curr. Biol. 16(19), 1905-1910 (2006)
28. Cross, E.S., Hamilton, A.F., Grafton, S.T.: Building a motor simulation de novo: observation of dance by dancers. Neuroimage 31(3), 1257-1267 (2006)
29. Maguire, E.A., et al.: Navigation-related structural change in the hippocampi of taxi drivers. Proc. Natl. Acad. Sci. U S A 97(8), 4398-4403 (2000)
30. Woollett, K., Maguire, E.A.: Acquiring "the Knowledge" of London's layout drives structural brain changes. Curr. Biol. 21(24), 2109-2114 (2011)
31. Hufner, K., et al.: Structural and functional plasticity of the hippocampal formation in professional dancers and slackliners. Hippocampus 21(8), 855-865 (2011)
32. Bezzola, L., et al.: Training-induced neural plasticity in golf novices. J. Neurosci. 31(35), 12444-12448 (2011)
33. Johansen-Berg, H., et al.: Integrity of white matter in the corpus callosum correlates with bimanual co-ordination skills. Neuroimage 36(suppl. 2), T16-T21 (2007)
34. Tuch, D.S., et al.: Choice reaction time performance correlates with diffusion anisotropy in white matter pathways supporting visuospatial attention. Proc. Natl. Acad. Sci. U S A 102(34), 12212-12217 (2005)
35. Draganski, B., et al.: Neuroplasticity: changes in grey matter induced by training. Nature 427(6972), 311-312 (2004)
36. Scholz, J., et al.: Training induces changes in white-matter architecture. Nat. Neurosci. 12(11), 1370-1371 (2009)
37. Bengtsson, S.L., et al.: Extensive piano practicing has regionally specific effects on white matter development. Nat. Neurosci. 8(9), 1148-1150 (2005)
38. Bermudez, P., et al.: Neuroanatomical correlates of musicianship as revealed by cortical thickness and voxel-based morphometry. Cereb Cortex 19(7), 1583-1596 (2009)
39. Gaser, C., Schlaug, G.: Brain structures differ between musicians and non-musicians. J. Neurosci. 23(27), 9240-9245 (2003)
40. Hutchinson, S., et al.: Cerebellar volume of musicians. Cereb Cortex 13(9), 943-949 (2003)
41. Halwani, G.F., et al.: Effects of practice and experience on the arcuate fasciculus: comparing singers, instrumentalists, and non-musicians. Front Psychol. 2, 156 (2011)
42. Imfeld, A., et al.: White matter plasticity in the corticospinal tract of musicians: a diffusion tensor imaging study. Neuroimage 46(3), 600-607 (2009)
43. Schmithorst, V.J., Wilke, M.: Differences in white matter architecture between musicians and non-musicians: a diffusion tensor imaging study. Neurosci. Lett. 321(1-2), 57-60 (2002)
44. Schlaug, G., et al.: In vivo evidence of structural brain asymmetry in musicians. Science 267(5198), 699-701 (1995)
45. Ozturk, A.H., et al.: Morphometric comparison of the human corpus callosum in professional musicians and non-musicians by using in vivo magnetic resonance imaging. J. Neuroradiol. 29(1), 29-34 (2002)
46. Tervaniemi, M.: Musicians-same or different? Ann. N Y Acad. Sci. 1169, 151-156 (2009)
47. Aydin, K., et al.: Quantitative proton MR spectroscopic findings of cortical reorganization in the auditory cortex of musicians. AJNR Am. J. Neuroradiol. 26(1), 128-136 (2005)
48. Elbert, T., et al.: Increased cortical representation of the fingers of the left hand in string players. Science 270(5234), 305-357 (1995)
49. Hyde, K.L., et al.: Musical training shapes structural brain development. J. Neurosci. 29(10), 3019-3025 (2009)
50. Hyde, K.L., et al.: The effects of musical training on structural brain development: a longitudinal study. Ann. N Y Acad. Sci. 1169, 182-186 (2009)
51. Kraus, N., Chandrasekaran, B.: Music training for the development of auditory skills. Nat. Rev. Neurosci. 11(8), 599-605 (2010)
52. Munte, T.F., Altenmuller, E., Jancke, L.: The musician's brain as a model of neuroplasticity. Nat. Rev. Neurosci. 3(6), 473-478 (2002)
53. Bangert, M., Altenmuller, E.O.: Mapping perception to action in piano practice: a longitudinal DC-EEG study. BMC Neurosci. 4, 26 (2003)
54. Magne, C., Schon, D., Besson, M.: Musician children detect pitch violations in both music and language better than nonmusician children: behavioral and electrophysiological approaches. J. Cogn. Neurosci. 18(2), 199-211 (2006)
55. Milovanov, R., Tervaniemi, M.: The Interplay between Musical and Linguistic Aptitudes: A Review. Front Psychol. 2, 321 (2011)
56. Chan, A.S., Ho, Y.C., Cheung, M.C.: Music training improves verbal memory. Nature 396(6707), 128 (1998)
57. Aleman, A., et al.: Music training and mental imagery ability. Neuropsychologia 38(12), 1664-1668 (2000)
58. Brochard, R., Dufour, A., Despres, O.: Effect of musical expertise on visuospatial abilities: evidence from reaction times and mental imagery. Brain Cogn. 54(2), 103-109 (2004)
59. Schmithorst, V.J., Holland, S.K.: The effect of musical training on the neural correlates of math processing: a functional magnetic resonance imaging study in humans. Neurosci. Lett. 354(3), 193-196 (2004)
60. Besson, M., Chobert, J., Marie, C.: Transfer of Training between Music and Speech: Common Processing, Attention, and Memory. Front Psychol. 2, 94 (2011)
61. Patel, A.D.: Why would Musical Training Benefit the Neural Encoding of Speech? The OPERA Hypothesis. Front Psychol. 2, 142 (2011)
62. Wallace, M.T., Stein, B.E.: Sensory and multisensory responses in the newborn monkey superior colliculus. J. Neurosci. 21(22), 8886-8894 (2001)
63. Wallace, M.T., Stein, B.E.: Development of multisensory neurons and multisensory integration in cat superior colliculus. J. Neurosci. 17(7), 2429-2444 (1997)
64. Wallace, M.T., Stein, B.E.: Cross-modal synthesis in the midbrain depends on input from cortex. J. Neurophysiol. 71(1), 429-4232 (1994)
65. Jiang, W., Jiang, H., Stein, B.E.: Neonatal cortical ablation disrupts multisensory development in superior colliculus. J. Neurophysiol. 95(3), 1380-1396 (2006)
66. Haslinger, B., et al.: Transmodal sensorimotor networks during action observation in professional pianists. J. Cogn. Neurosci. 17(2), 282-293 (2005)
67. Hodges, D.A., Hairston, W.D., Burdette, J.H.: Aspects of multisensory perception: the integration of visual and auditory information in musical experiences. Ann. N Y Acad. Sci. 1060, 175-185 (2005)
68. Petrini, K., et al.: Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony. Exp. Brain Res. 198(2-3), 339-352 (2009)
69. Petrini, K., Russell, M., Pollick, F.: When knowing can replace seeing in audiovisual integration of actions. Cognition 110(3), 432-439 (2009)
70. Petrini, K., Holt, S.P., Pollick, F.: Expertise with multisensory events eliminates the effect of biological motion rotation on audiovisual synchrony perception. J. Vis. 10(5), 2 (2010)
71. Petrini, K., et al.: Action expertise reduces brain activity for audiovisual matching actions: an fMRI study with expert drummers. Neuroimage 56, 1480-1492 (2011)
72. Musacchia, G., et al.: Musicians have enhanced subcortical auditory and audiovisual processing of speech and music. Proc. Natl. Acad. Sci. U S A 104(40), 15894-15898 (2007)
73. Schulz, M., Ross, B., Pantev, C.: Evidence for training-induced crossmodal reorganization of cortical functions in trumpet players. Neuroreport 14(1), 157-161 (2003)
74. Lappe, C., et al.: Cortical plasticity induced by short-term unimodal and multimodal musical training. J. Neurosci. 28(39), 9632-9639 (2008)
75. Schlaug, G., et al.: Effects of music training on the child's brain and cognitive development. Ann. N Y Acad. Sci. 1060, 219-230 (2005)
76. Spence, C., Squire, S.: Multisensory integration: maintaining the perception of synchrony. Curr. Biol. 13(13), R519-R521 (2003)
77. Arrighi, R., Alais, D., Burr, D.: Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. J. Vis. 6(3), 260-268 (2006)
78. van Wassenhove, V., Grant, K.W., Poeppel, D.: Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45(3), 598-607 (2007)
79. Vatakis, A., Spence, C.: Audiovisual synchrony perception for music, speech, and object actions. Brain Res. 1111(1), 134-142 (2006)
80. Vatakis, A., Spence, C.: Audiovisual synchrony perception for speech and music assessed using a temporal order judgment task. Neurosci. Lett. 393(1), 40-44 (2006)
81. Dixon, N.F., Spitz, L.: The detection of auditory visual desynchrony. Perception 9(6), 719-721 (1980)
82. Lee, H., Noppeney, U.: Long-term music training tunes how the brain temporally binds signals from multiple senses. Proc. Natl. Acad. Sci. U S A 108(51), E1441-E1450 (2011)
83. Petrini, K., et al.: The music of your emotions: neural substrates involved in detection of emotional correspondence between auditory and visual music actions. PLoS One 6(4), e19165 (2011)
84. Petrini, K., McAleer, P., Pollick, F.: Audiovisual integration of emotional signals from music improvisation does not depend on temporal correspondence. Brain Res. 1323, 139-148 (2010)
85. Hein, G., et al.: Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J. Neurosci. 27(30), 7881-7887 (2007)
86. Kim, R.S., Seitz, A.R., Shams, L.: Benefits of stimulus congruency for multisensory facilitation of visual learning. PLoS One 3(1), e1532 (2008)
87. Munhall, K.G., et al.: Temporal constraints on the McGurk effect. Percept. Psychophys 58(3), 351-362 (1996)
88. Vatakis, A., et al.: Temporal recalibration during asynchronous audiovisual speech perception. Exp. Brain Res. 181(1), 173-181 (2007)
89. Fain, G.L.: Sensory transduction, 340 p. Sinauer Associates, Sunderland (2003)
90. King, A.J.: Multisensory integration: strategies for synchronization. Curr. Biol. 15(9), R339-R3941 (2005)
91. King, A.J., Palmer, A.R.: Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp. Brain Res. 60(3), 492-500 (1985)
92. Waadeland, C.H.: Strategies in empirical studies of swing groove. Musicologia Norvegica 32, 169-191 (2006)
93. Jansson, G., Johansson, G.: Visual perception of bending motion. Perception 2(3), 321-326 (1973)
94. McGurk, H.M., Macdonald, J.: Hearing lips and seeing voices. Nature 264(5588), 746-748 (1976)
95. Vatakis, A., Spence, C.: Crossmodal binding: evaluating the "unity assumption" using audiovisual speech stimuli. Percept. Psychophys 69(5), 744-756 (2007)
96. Drake, C., Jones, M.R., Baruch, C.: The development of rhythmic attending in auditory sequences: attunement, referent period, focal attending. Cognition 77(3), 251-288 (2000)
97. Saygin, A.P., Driver, J., de Sa, V.R.: In the footsteps of biological motion and multisensory perception: judgments of audiovisual temporal relations are enhanced for upright walkers. Psychol. Sci. 19(5), 469-475 (2008)
98. Aschersleben, G., Prinz, W.: Synchronizing actions with events: the role of sensory information. Percept. Psychophys 57(3), 305-317 (1995)
99. Miyake, Y., Onishi, Y., Poppel, E.: Two types of anticipation in synchronization tapping. Acta Neurobiol Exp. (Wars), 64(3), 415-426 (2004)
100. Navarra, J., et al.: Perception of audiovisual speech synchrony for native and non-native language. Brain Res. 1323, 84-93 (2010)
101. Petrini, K., et al.: Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony. Experimental Brain Research 198(2-3), 339-352 (2009)
102. Vatakis, A., Spence, C.: Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. Acta Psychol. (Amst) 127(1), 12-23 (2008)
103. Vatakis, A., Ghazanfar, A.A., Spence, C.: Facilitation of multisensory integration by the "unity effect" reveals that speech is special. J. Vis. 8(9),14. 1-14.11 (2008)