Sensory substitution as an artificially acquired synaesthesia

Neuroscience and Biobehavioral Reviews

Volumn 41, Issue , 2014, Pages 26-35

  Ward, Jamie ^a   Wright, Thomas ^a  

^a UNIVERSITY OF SUSSEX   (United Kingdom)

Author keywords

Hearing; Multisensory; Sensory substitution; Synaesthesia synesthesia; Touch; Vision

Indexed keywords

ANIMAL; BRAIN; HEARING; HUMAN; NERVE CELL PLASTICITY; PATHOPHYSIOLOGY; PERCEPTION; PERCEPTUAL DISORDERS; PHYSIOLOGY; SENSORY AID; TOUCH; VISION;

ANIMALS; AUDITORY PERCEPTION; BRAIN; HUMANS; NEURONAL PLASTICITY; PERCEPTION; PERCEPTUAL DISORDERS; SENSORY AIDS; TOUCH PERCEPTION; VISUAL PERCEPTION;

EID: 84898877121     PISSN: 01497634     EISSN: 18737528     Source Type: Journal    
DOI: 10.1016/j.neubiorev.2012.07.007     Document Type: Review

References (87)

1. Afra M., Funke M., Matsuo F. Acquired auditory-visual synesthesia: a window to early cross-modal sensory interactions. Psychology Research and Behavior Management 2009, 2:31-37.
2. Amedi A., Stern W., Camprodon J.A., Bermpohl F., Merabet L., Rotman S., Hemond C., Meijer P., Pascual-Leone A. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neuroscience 2007, 10:687-689.
3. Anastasio T.J., Patton P.E., Belkacem-Boussaid K. Using Bayes' rule to model multisensory enhancement in the superior colliculus. Neural Computation 2000, 12:1165-1187.
4. Arno P., Capelle C., Wanet-Defalque M.C., Catalan-Ahumada M., Veraart C. Auditory coding of visual patterns for the blind. Perception 1999, 28(8):1013-1029.
5. Arno P., De Volder A.G., Vanlierde A., Wanet-Defalque M.-C., Streel E., Robert A., Sanabria-Bohorquez S., Veraart C. Occipital activation by pattern recognition in the early blind using auditory substitution for vision. NeuroImage 2001, 13:632-645.
6. Asher J.E., Lamb J.A., Brocklebank D., Cazier J.B., Maestrini E., Addis L., Sen M., Baron-Cohen S., Monaco A.P. A whole-genome scan and fine-mapping linkage study of auditory-visual synesthesia reveals evidence of linkage to chromosomes 2q24, 5q33, 6p12, and 12p12. American Journal of Human Genetics 2009, 84(2):279-285.
7. Auvray M., Hanneton S., O'Regan J.K. Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with 'The vOICe'. Perception 2007, 36:416-430.
8. Auvray M., Myin E. Perception with compensatory devices: from sensory substitution to sensorimotor extension. Cognitive Science 2009, 33(6):1036-1058.
9. Bach-y-Rita P. Brain Mechanisms in Sensory Substitution 1972, Academic Press, New York.
10. Bach-y-Rita P., Collins C.C., Saunders F.A., White B., Scadden L. Vision substitution by tactile image projection. Nature 1969, 221:963.
11. Bach-y-Rita P., Kaczmarek K.A., Tyler M.E., Garcia-Lara J. Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note. Journal of Rehabilitation Research and Development 1998, 35(4):427-430.
12. Bach-y-Rita P. Sustitucion sensorielle et qualia. Vision and Mind: Selected Readings in the Philosophy of Perception. 1996, MIT Press, Boston, MA, Reprinted (English translation). A. Noë, E. Thompson (Eds.).
13. Bach-y-Rita P., Kercel S.W. Sensory substitution and the human-machine interface. Trends in Cognitive Sciences 2003, 7:541-546.
14. Baron-Cohen S., Harrison J., Goldstein L.H., Wyke M. Coloured speech perception: is synaesthesia what happens when modularity breaks down?. Perception 1993, 22:419-426.
15. Beauchamp M.S., Ro T. Neural substrates of sound-touch synesthesia after a thalamic lesion. Journal of Neuroscience 2008, 28:13696-13702.
16. Bellugi U., Poizner H., Klima E.S. Language, modality and the brain. Trends in Neurosciences 1989, 12:380-388.
17. Block N. Tactile sensation via spatial perception. Trends in Cognitive Neurosciences 2003, 7:285-286.
18. Bologna G., Deville B., Thierry P. On the use of the auditory pathway to represent image scenes in real-time. NeuroComputing 2009, 72:839-849.
19. Brown D.J., Macpherson T., Ward J. Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device. Perception 2011, 40:1120-1135.
20. Cardinali L., Frassinetti F., Brozzoli C., Urquizar C., Roy A.C., Farnè A. Tool-use induces morphological updating of the body schema. Current Biology 2009, 19:R478-R479.
21. Cohen L.G., Celnik P., Pascual-Leone A., Corwell B., Faiz L., Dambrosia J., Honda M., Sadato N., Gerloff C., Catala M.D., Hallett M. Functional relevance of cross-modal plasticity in blind humans. Nature 1997, 389:180-183.
22. Cohen Kadosh R., Walsh V. Cognitive neuroscience: rewired or crosswired brains?. Current Biology 2006, 16:R962-R963.
23. Colizoli O., Murre J.M.J., Rouw R. Pseudo-synesthesia through reading books with colored letters. PLoS ONE 2012, 7:e39799.
24. Collignon O., Lassonde M., Lepore F., Bastien D., Veraart C. Functional cerebral reorganization for auditory spatial processing and auditory substitution of vision in early blind subjects. Cerebral Cortex 2007, 17(2):457-465.
25. Cronly-Dillon J., Persaud K., Gregory R.P.F. The perception of visual images encoded in musical form: a study in cross-modality information transfer. Proceedings of the Royal Society of London. Series B. Biological Sciences 1999, 2427-2433.
26. Cronly-Dillon J., Persaud K.C., Blore R. Blind subjects construct conscious mental images of visual scenes encoded in musical form. Proceedings of the Royal Society of London. Series B. Biological Sciences 2000, 267:2231-2238.
27. Elias L.J., Saucier D.M., Hardie C., Sart G.E. Dissociating semantic and perceptual components of synaesthesia: behavioural and functional neuroanatomical investigations. Cognitive Brain Research 2003, 16:232-237.
28. Farcy R., Damaschini R.M. Guidance-assist system for the blind. SPIE Proceedings 2001, 4158:209.
29. Fernay L., Reby D., Ward J. Visualised voices: a case study of audio-visual synaesthesia. Neurocase 2012, 18:50-56.
30. Froese T., McGann M., Bigge W., Spiers A., Seth A. The enactive torch: a new tool for the science of perception. IEEE Transactions on Haptics 2011, 9(99):1.
31. Gallace A., Spence C. Multisensory synesthetic interactions in the speeded classification of visual size. Perception and Psychophysics 2006, 68:1191-1203.
32. Gilchrist I.D., Brown V., Findlay J.M., Clarke M.P. Using the eye-movement system to control the head. Proceedings of the Royal Society of London. Series B. Biological Sciences 1998, 265:1831-1836.
33. Goller A.I., Otten L.J., Ward J. Seeing sounds and hearing colors: an event-related potential study of auditory-visual synesthesia. Journal of Cognitive Neuroscience 2009, 21:1869-1881.
34. Gray J. How are qualia coupled to functions?. Trends in Cognitive Sciences 2003, 7:192-194.
35. Grice H.P. Some remarks about the senses. Analytical Philosophy, Series I 1962, Oxford, Oxford University Press. R.J. Butler (Ed.).
36. Grossenbacher P.G., Lovelace C.T. Mechanisms of synaesthesia: cognitive and physiological constraints. Trends in Cognitive Sciences 2001, 5:36-41.
37. Guarniero G. Experience of tactile vision. Perception 1974, 3(1):101-104.
38. Hanneton S., Auvray M., Durette B. The Vibe: a versatile vision-to-audition sensory substitution device. Applied Bionics and Biomechanics 2010, 7:269-276.
39. Held R., Freedman S. Plasticity in human sensorimotor control. Science 1963, 142:455-462.
40. Holmes N.P., Calvert G.A., Spence C. Tool use changes multisensory interactions in seconds: evidence from the crossmodal congruency task. Experimental Brain Research 2007, 183:465-476.
41. Hubbard E.M., Ramachandran V.S. Neurocognitive mechanisms of synesthesia. Neuron 2005, 48:509-520.
42. Humphrey N. Seeing Red: A Study in Consciousness 2006, Harvard University Press, Boston, MA.
43. Hurley S., Noe A. Neural plasticity and consciousness. Biology and Philosophy 2003, 18:131-168.
44. Iriki A., Tanaka M., Iwamura Y. Coding of modified body schema during tool use by macaque postcentral neurons. Neuroreport 1996, 7:2325-2330.
45. Iriki A., Tanaka M., Obayashi S., Iwamura Y. Self-images in the video monitor coded by monkey intraparietal neurons. Neuroscience Research 2001, 40:163-173.
46. Jacobs L., Karpik A., Bozian D., Gothgen S. Auditory-visual synesthesia: sound-induced photisms. Archives of Neurology 1981, 38:211-216.
47. Keeley B.L. Making sense of the senses: individuating modalities in humans and other animals. Journal of Philosophy 2002, 99:5-28.
48. Kim J.-K., Zatorre R.J. Generalized learning of visual-to-auditory substitution in sighted individuals. Brain Research 2008, 242:263-275.
49. Koelewijn T., Bronkhorst A., Theeuwes J. Attention and the multiple stages of multisensory integration: a review of audiovisual studies. Acta Psychologica 2010, 134:372-384.
50. Kupers R., Fumal A., de Noordhout A.M., Gjedde A., Schoenen J., Ptito M. Transcranial magnetic stimulation of the visual cortex induces somatotopically organized qualia in blind subjects. Proceedings of the National Academy of Sciences of the United States of America 2006, 103:13256-13260.
51. Marks L.E. Cross-modal interactions in speeded classification. The Handbook of Multisensory Processes 2004, MIT Press, Cambridge, MA. G. Calvert, C. Spence, B.E. Stein (Eds.).
52. Martino G., Marks L.E. Cross-modal interaction between vision and touch: the role of synesthetic correspondence. Perception 2000, 29:745-754.
53. Martino G., Marks L.E. Synesthesia: strong and weak. Current Directions in Psychological Science 2001, 10:61-65.
54. Mattingley J.B. Attention, automaticity and awareness in synesthesia. Year in Cognitive Neuroscience 2009 2009, 1156:141-167.
55. Meier B., Rothen N. Training grapheme-colour associations produces a synaesthetic Stroop effect, but not a conditioned synaesthetic response. Neuropsychologia 2009, 47(4):1208-1211.
56. Meijer P.B.L. An Experimental System for Auditory Image Representations. IEEE Transactions on Biomedical Engineering 1992, 39:112-121.
57. Merabet L.B., Battelli L., Obretenova S., Maguire S., Meijer P., Pascual-Leone A. Functional recruitment of visual cortex for sound encoded object identification in the blind. Neuroreport 2009, 20:132-138.
58. Myles K.M., Dixon M.J., Smilek D., Merikle P.M. Seeing double: the role of meaning in alphanumeric-colour synaesthesia. Brain and Cognition 2003, 53:342-345.
59. Nagel S.K., Carl C., Kringe T., Martin R., Konig P. Beyond sensory substitution-learning the sixth sense. Journal of Neural Engineering 2005, 2(4):R13-R26.
60. Noe A. Action in Perception 2004, MIT Press, Boston, MA.
61. Noe A., Hurley S. The deferential brain in action. Trends in Cognitive Sciences 2003, 7:195-196.
62. Noppeney U. The effects of visual deprivation on functional and structural organization of the human brain. Neuroscience and Biobehavioral Review 2007, 31:1169-1180.
63. O'Regan J.K. Why Red Doesn't Sound Like a Bell: Understanding the Feel of Consciousness 2011, Oxford University Press, Oxford.
64. O'Regan J.K., Noe A. A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 2001, 24:939-1031.
65. Ortiz T., Poch J., Santos J.M., Requena C., Martinez A.M., Ortiz-Teran L., Turrero A., Barcia J., Nogales R., Calvo A., Martinez J.M., Cordoba J.L., Pascual-Leone A. Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness. PLoS ONE 2011, 6(8):11.
66. Poirier C., De Volder A.G., Scheiber C. What neuroimaging tells us about sensory substitution. Neuroscience and Biobehavioral Reviews 2007, 31:1064-1070.
67. Poirier C., De Volder A.G., Tranduy D., Scheiber C. Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects. Neuropsychologia 2007, 45:1108-1121.
68. Proulx M.J., Stoerig P. Seeing sounds and tingling tongues: qualia in synesthesia and sensory substitution. Anthropology and Philosophy 2006, 7:135-151.
69. Ptito M., Moesgaard S.M., Gjedde A., Kupers R. Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain 2005, 128:606-614.
70. Rao A.L., Nobre A.C., Alexander I., Cowey A. Auditory evoked visual awareness following sudden ocular blindness: an EEG and TMS investigation. Experimental Brain Research 2007, 176:288-298.
71. Renier L., Collignon O., Poirier C., Tranduy D., Vanlierde A., Bol A., Veraart C., De Volder A.G. Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 2005, 26(2):573-580.
72. Renier L., Laloyaux C., Collignon O., Tranduy D., Van lierde A., Bruyer R., De Volder A.G. The Ponzo illusion with auditory substitution of vision in sighted and early-blind subjects. Perception 2005, 34:857-867.
73. Rouw R., Scholte H.S., Colizoli O. Brain areas involved in synaesthesia: a review. Journal of Neuropsychology 2011, 5:214-242.
74. Sathian K. The New Handbook of Multisensory Processing 2012, B.E. Stein (Ed.).
75. Serino A., Bassolino M., Farne A., Ladavas E. Extended multisensory space in blind cane users. Psychological Science 2007, 18:642-648.
76. Sharma J., Angelucci A., Sur M. Induction of visual orientation modules in auditory cortex. Nature 2000, 404:841-847.
77. Simner J., Ludwig V.U. The color of touch: a case of tactile-visual synaesthesia. Neurocase 2012, 18:167-180.
78. Sur M., Garraghty P.E., Roe A.W. Experimentally induced visual projections into auditory thalamus and cortex. Science 1988, 242:1437-1441.
79. Tyler M., Danilov Y., Bach-y-Rita P. Closing an open-loop control system: vestibular substitution through the tongue. Journal of Integrative Neuroscience 2003, 2:159-164.
80. Vike J., Jabbari B., Maitland C.G. Auditory-visual synesthesia: report of a case with intact visual pathways. Archives of Neurology 1984, 41:680-681.
81. von Melcher L., Pallas S.L., Sur M. Visual behavior mediated by retinal projections directed to the auditory pathway. Nature 2000, 404:871-876.
82. Walker P., Bremner J.G., Mason U., Spring J., Mattock K., Slater A., Johnson S.P. Preverbal infants' sensitivity to synaesthetic cross-modality correspondences. Psychological Science 2010, 21(1):21-25.
83. Ward J., Banissy M.J., Jonas C.N. Haptic perception and synaesthesia. Handbook of Haptic Perception 2008, Springer-Verlag, Berlin. M. Grunwald (Ed.).
84. Ward J., Huckstep B., Tsakanikos E. Sound-colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all?. Cortex 2006, 42:264-280.
85. Ward J., Meijer P. Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition 2010, 19:492-500.
86. Ward J., Moore S., Thompson-Lake D., Salih S., Beck B. The aesthetic appeal of auditory-visual synaesthetic perceptions in people without synaesthesia. Perception 2008, 37:1285-1296.
87. Wright, T.D., Macallister, G., Ward, J. The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms, submitted for publication.