EyeMusic: Introducing a 'visual' colorful experience for the blind using auditory sensory substitution

Restorative Neurology and Neuroscience

Volumn 32, Issue 2, 2014, Pages 247-257

  Abboud, Sami ^a   Hanassy, Shlomi ^a   Levy Tzedek, Shelly ^a,b   Maidenbaum, Shachar ^a   Amedi, Amir ^a,b  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

Blindness; Color; Pleasant sounds; Rehabilitation; Sensory substitution devices; SSD

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BLINDNESS; CLINICAL ARTICLE; COLOR; CONTROLLED STUDY; FEMALE; HUMAN; MALE; MEDICAL DEVICE; PRIORITY JOURNAL; SENSORY SUBSTITUTION DEVICE; SOUND; VISUAL INFORMATION;

BLINDNESS; COLOR; PLEASANT SOUNDS; REHABILITATION; SENSORY SUBSTITUTION DEVICES; SSD;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; BLINDNESS; EDUCATION OF VISUALLY DISABLED; FEMALE; HUMANS; MALE; MIDDLE AGED; NEURONAL PLASTICITY; SENSATION; VISUALLY IMPAIRED PERSONS;

EID: 84895860666     PISSN: 09226028     EISSN: 18783627     Source Type: Journal    
DOI: 10.3233/RNN-130338     Document Type: Article

References (40)

1. Auvray, M., Hanneton, S. & O'Regan, J.K. (2007). Learning to perceive with a visuo-Auditory substitution system: Localisation and object recognition with 'the vOICe'. Perception, 36, 416-430.
2. Auvray, M. & Myin, E. (2009). Perception with compensatory devices: From sensory substitution to sensorimotor extension. Cogn Sci, 33, 1036-1058.
3. Bach-y-Rita, P., Collins, C.C., Saunders, F.A., White, B. & Scadden, L. (1969). Vision substitution by tactile image projection. Nature, 221, 963-964.
4. Bologna, G., Deville, B., Pun, T. & Vinckenbosch, M. (2007). Transforming 3D Coloured Pixels into Musical Instrument Notes for Vision Substitution Applications. J Image Video Process, 2007, 076204.
5. Bologna, G., Deville, B. & Pun, T. (2009). Bioinspired Applications in Artificial and Natural Computation. Springer Berlin Heidelberg.
6. Bram̃ao, I., Reis, A., Petersson, K.M. & Fáisca, L. (2011). The role of color information on object recognition: A review and metaanalysis. Acta Psychol, 138, 244-253.
7. Brown, D., Macpherson, T. & Ward, J. (2011). Seeing with sound? exploring different characteristics of a visual-To-Auditory sensory substitution device. Perception, 40, 1120-1135.
8. Capelle, C., Trullemans, C., Arno, P. & Veraart, C. (1998). A real-Time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Trans Biomed Eng, 45, 1279-1293.
9. Chebat, D.R., Schneider, F.C., Kupers, R. & Ptito, M. (2011). Navigation with a sensory substitution device in congenitally blind individuals. Neuroreport, 22, 342-347.
10. Chen, M.C., Tsai, P.L., Huang, Y.T. & Lin, K.C. (2013). Pleasant music improves visual attention in patients with unilateral neglect after stroke. Brain Inj, 27, 75-82.
11. Dagnelie, G. (2012). Retinal implants: Emergence of a multidisciplinary field. Current opinion in neurology, 25(1), 67-75.
12. Doel, K. (2003). Soundview: Sensing color images by kinesthetic audio. Proc of ICAD, pp. 303-306.
13. Goffaux, V., Jacques, C., Mouraux, A., Oliva, A., Schyns, P. & Rossion, B. (2005). Diagnostic colours contribute to the early stages of scene categorization: Behavioural and neurophysiological evidence. Vis Cogn, 12, 878-892.
14. Goldreich, D. & Kanics, I.M. (2003). Tactile acuity is enhanced in blindness. J Neurosci, 23, 3439-3445.
15. Gougoux, F., Lepore, F., Lassonde, M., Voss, P., Zatorre, R.J. & Belin, P. (2004). Neuropsychology: Pitch discrimination in the early blind. Nature, 430, 309.
16. Gould, S., Fulton, R. & Koller, D. (2009). Decomposing a scene into geometric and semantically consistent regions. IEEE I Conf Comp Vis, pp. 1-8.
17. Kaczmarek, K.A. (2011). The tongue display unit (TDU) for electrotactile spatiotemporal pattern presentation. Sci Iran, 18, 1476-1485.
18. Kahol, K., French, J., Bratton, L. & Panchanathan, S. (2006). Learning and perceiving colors haptically. Proc of ASSETS, pp. 173-180.
19. Kasten, E., Poggel, D.A.,M̈uller-Oehring, E., Gothe, J., Schulte,T. & Sabel, B.A. (1999). Restoration of vision II: Residual functions and training-induced visual field enlargement in brain-damaged patients. Restorative neurology and neuroscience, 15(2), 273-287.
20. Kumar, S., Forster, H.M., Bailey, P. & Griffiths, T.D. (2008). Mapping unpleasantness of sounds to their auditory representation. J Acoust Soc Am, 124, 3810-3817.
21. Kupers, R., Chebat, D.R., Madsen, K.H., Paulson, O.B. & Ptito, M. (2010). Neural correlates of virtual route recognition in congenital blindness. Proc Natl Acad Sci U S A, 107, 12716-12721.
22. Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S. & Amedi, A. (2012). Fast, accurate reaching movements with a visual-To-Auditory sensory substitution device. Restor Neurol Neurosci, 30, 313-323.
23. Levy-Tzedek, S., Novick, I., Arbel, R., Abboud, S., Maidenbaum, S., Vaadia, E. & Amedi, A. (2012). Cross-sensory transfer of sensory-motor information: Visuomotor learning affects performance on an audiomotor task, using sensory-substitution. Sci Rep, 2, 949.
24. Maidenbaum, S & Amedi, A. (2012). Plasticity in Sensory Systems, Cambridge University Press.
25. Meijer, P.B. (1992). An experimental system for auditory image representations. IEEE Trans Biomed Eng, 39, 112-121.
26. Poirier, C., De Volder, A., Tranduy, D. & Scheiber, C. (2007). Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects. Neuropsychologia, 45, 1108-1121.
27. Proulx, M.J., Stoerig, P., Ludowig, E. & Knoll, I. (2008). Seeing 'where' through the ears: Effects of learning-by-doing and longterm sensory deprivation on localization based on image-Tosound substitution. PLoS One, 3, e1840.
28. Quattoni, A. & Torralba, A. (2009). Recognizing indoor scenes. Proc CVPR IEEE, pp. 413-420.
29. Reich, L., Maidenbaum, S. & Amedi, A. (2012). The brain as a flexible task machine: Implications for visual rehabilitation using noninvasive vs. invasive approaches. Curr Opin Neurol, 25, 86-95.
30. Renier, L. & De Volder, A.G. (2010). Vision substitution and depth perception: Early blind subjects experience visual perspective through their ears. Disabil Rehabil Assist Technol, 5, 175-183.
31. Sarkamo, T. & Soto, D. (2012). Music listening after stroke: Beneficial effects and potential neural mechanisms. Ann N Y Acad Sci, 1252, 266-281.
32. Sathian, K. & Randall S. (2010). Cross-modal plasticity of tactile perception in blindness. Restorative neurology and neuroscience, 28(2), 271-281.
33. Soto, D., Funes, M.J., Guzman-Garcia, A., Warbrick, T., Rotshtein, P. & Humphreys, G.W. (2009). Pleasant music overcomes the loss of awareness in patients with visual neglect. Proc Natl Acad Sci U S A, 106, 6011-6016.
34. Striem-Amit, E., Cohen, L., Dehaene, S & Amedi, A. (2012). Reading with sounds: Sensory substitution selectively activates the visual word form area in the blind, 76, 640-52.
35. Striem-Amit, E., Guendelman, M. & Amedi, A. (2012). 'Visual' acuity of the congenitally blind using visual-To-Auditory sensory substitution. PLoS One, 7, e33136.
36. Tsai, P.L., Chen, M.C., Huang, Y.T., Lin, K.C., Chen, K.L. & Hsu, Y.W. (2013). Listening to classical music ameliorates unilateral neglect after stroke. Am J Occup Ther, 67, 328-335.
37. Ward, J. &Meijer, P. (2010). Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and cognition, 19, 492-500.
38. World Health Organization, (2012) Fact sheet no. 282
39. Wright, T. & Ward, J. (2013). The evolution of a visual-To-Auditory sensory substitution device using interactive genetic algorithms. Q J Exp Psychol, 66, 1620-1638.
40. Yip, A.W. & Sinha, P. (2002). Contribution of color to face recognition. Perception, 31, 995-1003.