Supramodal processing optimizes visual perceptual learning and plasticity

NeuroImage

Volumn 93, Issue P1, 2014, Pages 32-46

  Zilber, Nicolas ^a,b   Ciuciu, Philippe ^a,b   Gramfort, Alexandre ^a,b,c   Azizi, Leila ^a,d,e   Van Wassenhove, Virginie ^a,d,e  

^a DIF   (France)

^b INRIA   (France)

^c INSTITUT TELECOM   (France)

^d INSERM   (France)

^e UNIVERSITÉ PARIS SACLAY   (France)

Author keywords

Audition; Coherence; Dual stream; MEG; Multisensory; Sensory substitution device

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY DISCRIMINATION; CONTROLLED STUDY; DISCRIMINATION LEARNING; EVENT RELATED POTENTIAL; FEMALE; FRONTAL EYE FIELD; HEARING; HUMAN; HUMAN EXPERIMENT; INFERIOR TEMPORAL CORTEX; INTRAPARIETAL SULCUS; MAGNETOENCEPHALOGRAPHY; MALE; NERVE CELL PLASTICITY; NORMAL HUMAN; PRIORITY JOURNAL; PROCESS OPTIMIZATION; RADIOGRAPHY; RANDOM DOT KINEMATOGRAM; REVERSE HIERARCHY THEORY OF LEARNING; SENSITIVITY ANALYSIS; SENSORY STIMULATION; SIGNAL PROCESSING; SUPERIOR TEMPORAL SULCUS; SUPRAMODAL PROCESSING; SYSTEMS THEORY; TASK PERFORMANCE; VISUAL DISCRIMINATION; ADOLESCENT; BRAIN; BRAIN MAPPING; PHYSIOLOGY; VISION; YOUNG ADULT;

ADOLESCENT; ADULT; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; DISCRIMINATION LEARNING; FEMALE; HUMANS; MAGNETOENCEPHALOGRAPHY; MALE; NEURONAL PLASTICITY; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84896906907     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.02.017     Document Type: Article

References (115)

1. Ahissar M., Hochstein S. Task difficulty and the specificity of perceptual learning. Nature 1997, 387:401-406.
2. Ahissar M., Hochstein S. The reverse hierarchy theory of visual perceptual learning. Trends Cogn. Sci. 2004, 8:457-464.
3. Ahlfors S., Simpson G., Dale A., Belliveau J., Liu A., Korvenoja A., Virtanen J., Huotilainen M., Tootell R., Aronen H., Ilmoniemi R. Spatiotemporal activity of a cortical network for processing visual motion revealed by MEG and fMRI. J. Neurophysiol. 1999, 82:2545-2555.
4. Alais D., Burr D. No direction-specific bimodal facilitation for audiovisual motion detection. Brain Res. Cogn. Brain Res. 2004, 19:185-194.
5. Alink A., Singer W., Muckli L. Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. J. Neurosci. 2008, 28:2690-2697.
6. Amano K., Goda N., Nishida S., Ejima Y., Takeda T., Ohtani Y. Estimation of the timing of human visual perception from magnetoencephalography. J. Neurosci. 2006, 26:3981-3991.
7. Amedi A., Stern W., Camprodon J., Bermpohl F., Merabet L., Rotman S., Hemond C., Meijer P., Pascual-Leone A. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat. Neurosci. 2007, 10:687-689.
8. Aspell J., Tanskanen T., Hurlbert A. Neuromagnetic correlates of visual motion coherence. Eur. J. Neurosci. 2005, 22:2937-2945.
9. Bach-y-Rita P., Kercel S. Sensory substitution and the human-machine interface. Trends Cogn. Sci. 2003, 7:541-546.
10. Bavelier D., Hirshorn E. I see where you're hearing: how cross-modal plasticity may exploit homologous brain structures. Nat. Neurosci. 2010, 13:1309-1311.
11. Beauchamp M., Argall B., Bodurka J., Duyn J., Martin A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 2004, 7:1190-1192.
12. Beauchamp M., Lee K., Argall B., Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41:809-823.
13. Bedny M., Konkle T., Pelphrey K., Saxe R., Pascual-Leone A. Sensitive period for a multimodal response in human visual motion area MT/MST. Curr. Biol. 2010, 20:1900-1906.
14. Benevento L., Fallon J., Davis B., Rezak M. Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp. Neurol. 1977, 57:849-872.
15. Britten K., Shadlen M., Newsome W., Movshon J. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 1992, 12:4745-4765.
16. Bruce C., Desimone R., Gross C. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J. Neurophysiol. 1981, 46:369-384.
17. Chan A. Functional organization and visual representations of human ventral lateral prefrontal cortex. Front. Psychol. 2013, 4:371.
18. Chandrasekaran C., Trubanova A., Stillittano S., Caplier A., Ghazanfar A. The natural statistics of audiovisual speech. PLoS Comput. Biol. 2009, 5:e1000436.
19. Dale A., Fischl B., Sereno M. Cortical surface-based analysis I. Segmentation and surface reconstruction. Neuroimage 1999, 9:179-194.
20. Dale A., Liu A., Fischl B., Buckner R. Dynamic statistical parametric mapping: combining fMRI and MEG for high-resolution imaging of cortical activity. Neuron 2000, 26:55-67.
21. Dienes Z. Subjective measures of unconscious knowledge. Prog. Brain Res. 2008, 168:49-64.
22. Dormal G., Collignon O. Functional selectivity in sensory-deprived cortices. J. Neurophysiol. 2011, 105:2627-2630.
23. Driver J., Spence C. Multisensory perception: beyond modularity and convergence. Curr. Biol. 2000, 10:R731-R735.
24. Evans K., Treisman A. Natural cross-modal mappings between visual and auditory features. J. Vis. 2010, 10:1-12.
25. Fahle M. Perceptual learning: specificity versus generalization. Curr. Opin. Neurobiol. 2005, 15:154-160.
26. Fischl B., Dale A. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc. Natl. Acad. Sci. U. S. A. 2000, 97:11050-11055.
27. Fischl B., Sereno M., Dale A. Cortical surface-based analysis. II: inflation, flattening, and a surface-based coordinate system. Neuroimage 1999, 9:195-207.
28. Fischl B., Sereno M., Tootell R., Dale A. High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 1999, 8:272-284.
29. Frassinetti F., Bolognini N., Bottari D., Bonora A., Làdavas E. Audiovisual integration in patients with visual deficit. J. Cogn. Neurosci. 2005, 17:1442-1452.
30. Freeman E., Driver J. Direction of visual apparent motion driven solely by timing of a static sound. Curr. Biol. 2008, 18:1262-1266.
31. Ghazanfar A., Schroeder C. Is neocortex essentially multisensory?. Trends Cogn. Sci. 2006, 10:278-285.
32. Gilbert C., Sigman M., Crist R. The neural basis of perceptual learning. Neuron 2001, 31:681-697.
33. Gold J., Ding L. How mechanisms of perceptual decision-making affect the psychometric function. Prog. Neurobiol. 2013, 103:98-114.
34. Gold J., Law C., Connolly P., Bennur S. Relationships between the threshold and slope of psychometric and neurometric functions during perceptual learning: implications for neuronal pooling. J. Neurophysiol. 2010, 103:140-154.
35. Goldstone R. Perceptual learning. Annu. Rev. Psychol. 1998, 49:585-612.
36. Gramfort A., Luessi M., Larson E., Engemann D., Strohmeier D., Brodbeck C., Goj R., Jas M., Brooks T., Parkkonen L., Hämäläinen M. MEG and EEG data analysis with MNE-Python. Front. Neuroinform. 2013, 7:42.
37. Gramfort A., Luessi M., Larson E., Engemann D.A., Strohmeier D., Brodbeck C., Goj R., Jas M., Brooks T., Parkkonen L., Hämäläinen M. MEG and EEG data analysis with MNE-Python. Front. Neurosci. 2013, 7:267. 10.3389/fnins.2013.00267.
38. Grant K., Greenberg S. Speech intelligibility derived from asynchronous processing of auditory-visual information. Proc Conf Audio-Visual Speech Processing Aalborg, Denmark 2001, 132-137. (Available at: http://www.isca-speech.org/archive_open/avsp01/av01_132.html [Accessed December 11, 2013].).
39. Grant K., Seitz P. The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am. 2000, 108:1197-1208.
40. Gross J., Baillet S., Barnes G., Henson R., Hillebrand A., Jensen O., Jerbi K., Litvak V., Maess B., Oostenveld R., Parkkonen L., Taylor J., van Wassenhove V., Wibral M., Schoffelen J. Good practice for conducting and reporting MEG research. Neuroimage 2012, 65:349-363.
41. Hackett T., La Mothe L., De Ulbert I., Karmos G., Smiley J., Schroeder C. Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. J. Comp. Neurol. 2007, 502:924-952.
42. Hämäläinen M., Sarvas J. Realistic conductivity geometry model of the human head for interpretation of neuromagnetic data. IEEE Trans. Biomed. Eng. 1989, 36:165-171.
43. Händel B., Lutzenberger W., Thier P., Haarmeier T. Opposite dependencies on visual motion coherence in human area MT+ and early visual cortex. Cereb. Cortex 2007, 17:1542-1549.
44. Hidaka S., Teramoto W., Sugita Y., Manaka Y., Sakamoto S., Suzuki Y. Auditory motion information drives visual motion perception. PLoS One 2011, 6:e17499.
45. Ilg U., Churan J. Motion perception without explicit activity in areas MT and MST. J. Neurophysiol. 2004, 92:1512-1523.
46. Jaekl P., Soto-Faraco S. Audiovisual contrast enhancement is articulated primarily via the M-pathway. Brain Res. 2010, 1366:85-92.
47. Kösem A., van Wassenhove V. Temporal structure in audiovisual sensory selection. PLoS One 2012, 7:e40936.
48. Lam K., Kaneoke Y., Gunji A., Yamasaki H., Matsumoto E., Naito T., Kakigi R. Magnetic response of human extrastriate cortex in the detection of coherent and incoherent motion. Neuroscience 2000, 97:1-10.
49. Levy-Tzedek S., Hanassy S., Abboud S., Maidenbaum S., Amedi A. Fast, accurate reaching movements with a visual-to-auditory sensory substitution device. Restor. Neurol. Neurosci. 2012, 30:313-323.
50. Levy-Tzedek S., Novick I., Arbel R., Abboud S., Maidenbaum S., Vaadia E., Amedi A. Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution. Sci. Rep. 2012, 2:949.
51. Lewis R., Noppeney U. Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J. Neurosci. 2010, 30:12329-12339.
52. Lin F., Belliveau J., Dale A., Hämäläinen M. Distributed current estimates using cortical orientation constraints. Hum. Brain Mapp. 2006, 27:1-13.
53. Lomber S., Meredith M., Kral A. Cross-modal plasticity in specific auditory cortices underlies visual compensations in the deaf. Nat. Neurosci. 2010, 13:1421-1427.
54. Maeda F., Kanai R., Shimojo S. Changing pitch induced visual motion illusion. Curr. Biol. 2004, 14:R990-R991.
55. Mao Y., Hua T., Pallas S. Competition and convergence between auditory and cross-modal visual inputs to primary auditory cortical areas. J. Neurophysiol. 2011, 105:1558-1573.
56. Maris E., Oostenveld R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 2007, 164:177-190.
57. Maruyama K., Kaneoke Y., Watanabe K., Kakigi R. Human cortical responses to coherent and incoherent motion as measured by magnetoencephalography. Neurosci. Res. 2002, 44:195-205.
58. Meijer P. An experimental system for auditory image representations. IEEE Trans. Biomed. Eng. 1992, 39:112-121.
59. Melara R., O'Brien T. Interaction between synesthetically corresponding dimensions. J. Exp. Psychol. Gen. 1987, 116:323-336.
60. Mercier M., Schwartz S., Michel C., Blanke O. Motion direction tuning in human visual cortex. Eur. J. Neurosci. 2009, 29:424-434.
61. Meredith M. On the neuronal basis for multisensory convergence: a brief overview. Brain Res. Cogn. Brain Res. 2002, 14:31-40.
62. Mitchel A., Weiss D. Learning across senses: cross-modal effects in multisensory statistical learning. J. Exp. Psychol. Learn. Mem. Cogn. 2011, 37:1081-1091.
63. Morrone M. Brain development: critical periods for cross-sensory plasticity. Curr. Biol. 2010, 20:R934-R936.
64. Morrone M., Tosetti M., Montanaro D., Fiorentini A., Cioni G., Burr D. A cortical area that responds specifically to optic flow, revealed by fMRI. Nat. Neurosci. 2000, 3:1322-1328.
65. Mosher J., Leahy R., Lewis P. EEG and MEG: forward solutions for inverse methods. IEEE Trans. Biomed. Eng. 1999, 46:245-259.
66. Nakamura H. Human V5 demonstrated by magnetoencephalography using random dot kinematograms of different coherence levels. Neurosci. Res. 2003, 46:423-433.
67. Noesselt T., Rieger J., Schoenfeld M., Kanowski M., Hinrichs H., Heinze H., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27:11431-11441.
68. Noguchi Y., Kaneoke Y., Kakigi R., Tanabe H., Sadato N. Role of the superior temporal region in human visual motion perception. Cereb. Cortex 2005, 15:1592-1601.
69. Oostenveld R., Fries P., Maris E., Schoffelen J. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 2011:156869.
70. Overath T., Kumar S., Stewart L., von Kriegstein K., Cusack R., Rees A., Griffiths T. Cortical mechanisms for the segregation and representation of acoustic textures. J. Neurosci. 2010, 30:2070-2076.
71. Pascual-Leone A., Hamilton R. The metamodal organization of the brain. Prog. Brain Res. 2001, 134:427-445.
72. Pelli D. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 1997, 10:437-442.
73. Poirier C., Collignon O., Devolder A., Renier L., Vanlierde A., Tranduy D., Scheiber C. Specific activation of the V5 brain area by auditory motion processing: an fMRI study. Brain Res. Cogn. Brain Res. 2005, 25:650-658.
74. Poirier C., Collignon O., Scheiber C., Renier L., Vanlierde A., Tranduy D., Veraart C., De Volder A. Auditory motion perception activates visual motion areas in early blind subjects. Neuroimage 2006, 31:279-285.
75. Powers A., Hevey M., Wallace M. Neural correlates of multisensory perceptual learning. J. Neurosci. 2012, 32:6263-6274.
76. Proulx M., Brown D., Pasqualotto A., Meijer P. Multisensory perceptual learning and sensory substitution. Neurosci. Biobehav. Rev 2012, (in presss). 10.1016/j.neubiorev.2012.
77. Rauschecker J., Tian B. Mechanisms and streams for processing of "what" and "where" in auditory cortex. Proc. Natl. Acad. Sci. U. S. A. 2000, 97:11800-11806.
78. Renier L., De Volder A., Rauschecker J. Cortical plasticity and preserved function in early blindness. Neurosci. Biobehav. Rev. 2013, 1-11.
79. Ricciardi E., Bonino D., Pellegrini S., Pietrini P. Mind the blind brain to understand the sighted one! Is there a supramodal cortical functional architecture?. Neurosci. Biobehav. Rev. 2013, (in presss). 10.1016/j.neubiorev.2013.10.006.
80. Ricciardi E., Pietrini P. New light from the dark: what blindness can teach us about brain function. Curr. Opin. Neurol. 2011, 24:357-363.
81. Ricciardi E., Vanello N., Sani L., Gentili C., Scilingo E., Landini L., Guazzelli M., Bicchi A., Haxby J., Pietrini P. The effect of visual experience on the development of functional architecture in hMT+. Cereb. Cortex 2007, 17:2933-2939.
82. Roe A., Pallas S., Hahm J., Sur M. A map of visual space induced in primary auditory cortex. Science 1990, 250(80):818-820.
83. Romanski L. Domain specificity in the primate prefrontal cortex. Cogn. Affect. Behav. Neurosci. 2004, 4:421-429.
84. Romanski L. Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cereb. Cortex 2007, 17:61-69.
85. Romanski L., Hwang J. Timing of audiovisual inputs to the prefrontal cortex and multisensory integration. Neuroscience 2012, 214:36-48.
86. Romanski L., Tian B., Fritz J., Mishkin M., Goldman-Rakic P., Rauschecker J. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci. 1999, 2:1131-1136.
87. Rousseeuw P., Leroy A. Robust Regression and Outlier Detection 1987, Wiley, New York.
88. von Saldern S., Noppeney U. Sensory and striatal areas integrate auditory and visual signals into behavioral benefits during motion discrimination. J. Neurosci. 2013, 33:8841-8849.
89. Sasaki Y., Nanez J., Watanabe T. Advances in visual perceptual learning and plasticity. Nat. Rev. Neurosci. 2010, 11:53-60.
90. Savitzky A., Golay M. Smoothing and differentiation of data by simplified least squares procedures. Anal. Chem. 1964, 36:1627-1639.
91. Saygin A. Superior temporal and premotor brain areas necessary for biological motion perception. Brain 2007, 130:2452-2461.
92. Schafer R. What is a Savitzky-Golay filter?. IEEE Signal Process. Mag. 2011, 28:111-117.
93. Scheef L., Boecker H., Daamen M., Fehse U., Landsberg M., Granath D., Mechling H., Effenberg A. Multimodal motion processing in area V5/MT: evidence from an artificial class of audio-visual events. Brain Res. 2009, 1252:94-104.
94. Schroeder C., Foxe J. Multisensory contributions to low-level, "unisensory" processing. Curr. Opin. Neurobiol. 2005, 15:454-458.
95. Schwartz J., Berthommier F., Savariaux C. Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 2004, 93:B69-B78.
96. Seitz A., Kim R., Shams L. Sound facilitates visual learning. Curr. Biol. 2006, 16:1422-1427.
97. Seitz A., Kim R., van Wassenhove V., Shams L. Simultaneous and independent acquisition of multisensory and unisensory associations. Perception 2007, 36:1445-1453.
98. Seitz A., Watanabe T. A unified model for perceptual learning. Trends Cogn. Sci. 2005, 9:329-334.
99. Shams L., Seitz A.R. Benefits of multisensory learning. Trends Cogn. Sci. 2008, 12:411-417.
100. Song C., Kanai R., Fleming S., Weil R., Schwarzkopf D., Rees G. Relating inter-individual differences in metacognitive performance on different perceptual tasks. Conscious. Cogn. 2011, 20:1787-1792.
101. Stein B., Meredith M. The Merging of the Senses 1993, MIT Press, Cambridge, MA, US.
102. Striem-Amit E., Dakwar O., Reich L., Amedi A. The large-scale organization of "visual" streams emerges without visual experience. Cereb. Cortex 2012, 22:1698-1709.
103. Sugihara T., Diltz M., Averbeck B., Romanski L. Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J. Neurosci. 2006, 26:11138-11147.
104. Talsma D., Senkowski D., Soto-Faraco S., Woldorff M. The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 2010, 14:400-410.
105. Taulu S., Simola J. Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements. Phys. Med. Biol. 2006, 51:1759-1768.
106. Tootell R., Reppas J., Kwong K., Malach R., Born R., Brady T., Rosen B., Belliveau J. Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 1995, 15:3215-3230.
107. Ungerleider L., Mishkin M. Two Cortical Visual Systems 1982, MIT Press, Cambridge, MA, US. D. Ingle, M. Goodale, J. Mansfield (Eds.).
108. Voss P., Zatorre R. Organization and reorganization of sensory-deprived cortex. Curr. Biol. 2012, 22:R168-R173.
109. Watkins K., Shakespeare T., O'Donoghue M., Alexander I., Ragge N., Cowey A., Bridge H. Early auditory processing in area V5/MT+ of the congenitally blind brain. J. Neurosci. 2013, 33:18242-18246.
110. Watson J., Myers R., Frackowiak R., Hajnal J., Woods R., Mazziotta J., Shipp S., Zeki S. Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. Cereb. Cortex 1993, 3:79-94.
111. Weisberg S. Outliers and influence. Applied Linear Regression Hoboken 2005, 194-210. Wiley.
112. Werner S., Noppeney U. Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J. Neurosci. 2010, 30:2662-2675.
113. Wichmann F., Hill N. The psychometric function: I. Fitting, sampling, and goodness of fit. Percept. Psychophys. 2001, 63:1293-1313.
114. Wierzchoń M., Asanowicz D., Paulewicz B., Cleeremans A. Subjective measures of consciousness in artificial grammar learning task. Conscious. Cogn. 2012, 21:1141-1153.
115. Zeki S., Watson J., Frackowiak R. Going beyond the information given: the relation of illusory visual motion to brain activity. Proc. Biol. Sci. 1993, 252:215-222.