The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms

Quarterly Journal of Experimental Psychology

Volumn 66, Issue 8, 2013, Pages 1620-1638

  Wright, Thomas ^a   Ward, Jamie ^a  

^a UNIVERSITY OF SUSSEX   (United Kingdom)

Author keywords

Aesthetics; Blindness; Genetic algorithms; Polyglot; Sensory substitution; Visual impairment

Indexed keywords

ADOLESCENT; ADULT; ALGORITHM; ARTICLE; AUDITORY STIMULATION; CHI SQUARE DISTRIBUTION; CONTROLLED CLINICAL TRIAL; CONTROLLED STUDY; FEEDBACK SYSTEM; FEMALE; HEARING; HUMAN; MALE; PHYSIOLOGY; RANDOMIZED CONTROLLED TRIAL; SENSORY AID; VISION; YOUNG ADULT;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; ALGORITHMS; AUDITORY PERCEPTION; CHI-SQUARE DISTRIBUTION; FEEDBACK; FEMALE; HUMANS; MALE; SENSORY AIDS; VISION, OCULAR; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84882275156     PISSN: 17470218     EISSN: 17470226     Source Type: Journal    
DOI: 10.1080/17470218.2012.754911     Document Type: Article

References (49)

1. Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L. B. Rotman, S. 2007. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neuroscience, 10 (6): 687 - 689. doi:10.1038/nn1912 (doi:10.1038/nn1912)
2. Arno, P., Capelle, C., Wanet-Defalque, M.-C., Catalan-Ahumada, M. and Veraart, C. 1999. Auditory coding of visual patterns for the blind. Perception, 28 (8): 1013 - 1029. doi:10.1068/p2607 (doi:10.1068/p2607)
3. Arno, P., Vanlierde, A., Streel, E., Wanet-Defalque, M.-C., Sanabria-Bohorquez, S. and Veraart, C. 2001. Auditory substitution of vision: Pattern recognition by the blind. Applied Cognitive Psychology, 15 (5): 509 - 519. doi:10.1002/acp.720 (doi:10.1002/acp.720)
4. Auvray, M., Hanneton, S., Lenay, C. and O'Regan, K. 2005. There is something out there: Distal attribution in sensory substitution, twenty years later. Journal of Integrative Neuroscience, 4 (4): 505 - 521. (doi:10.1142/S0219635205001002)
5. Auvray, M., Hanneton, S. and O'Regan, J. K. 2007. Learning to perceive with a visuo-auditory substitution system: Localisation and object recognition with "The vOICe". Perception, 36 (3): 416 - 430. doi:10.1068/p5631 (doi:10.1068/p5631)
6. Bach-Y-Rita, P., Collins, C. C., Saunders, F. A., White, B. and Scadden, L. 1969. Vision substitution by tactile image projection. Nature, 221 (5184): 963 - 964. doi:10.1038/221963a0 (doi:10.1038/221963a0)
7. Bach-Y-Rita, P., Kaczmarek, K. A., Tyler, M. E. and Garcia-Lara, J. 1998. Form perception with a 49-point electrotactile stimulus array on the tongue: A technical note. Journal of Rehabilitation Research and Development, 35 (4): 427 - 430.
8. Baumann, O. and Belin, P. 2010. Perceptual scaling of voice identity: Common dimensions for different vowels and speakers. Psychological Research, 74 (1): 110 - 120. (doi:10.1007/s00426-008-0185-z)
9. Ben-Artzi, E. and Marks, L. E. 1995. Visual-auditory interaction in speeded classification: Role of stimulus difference. Perception & Psychophysics, 57 (8): 1151 - 1162. doi:10.3758/BF03208371 (doi:10.3758/BF03208371)
10. Biles, J. 1994. " GenJam: A genetic algorithm for generating jazz solos ". In Proceedings of the International Computer Music Association 131 - 137. Retrieved from http://hdl.handle.net/2027/spo.bbp2372.1994.033
11. Blickle, T. and Thiele, L. 1996. A comparison of selection schemes used in evolutionary algorithms. Evolutionary Computation, 4 (4): 361 - 394. doi:10.1162/evco.1996.4.4.361 (doi:10.1162/evco.1996.4.4.361)
12. Brown, D., Macpherson, T. and Ward, J. 2011. Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device. Perception, 40 (9): 1120 - 1135. doi:10.1068/p6952 (doi:10.1068/p6952)
13. Cinzia, D. D. and Vittorio, G. 2009. Neuroaesthetics: A review. Current Opinion in Neurobiology, 19 (6): 682 - 687. doi:10.1016/j.conb.2009.09.001 (doi:10.1016/j.conb.2009.09.001)
14. Cronly-Dillon, J., Persaud, K. and Gregory, R. P. F. 1999. The perception of visual images encoded in musical form: A study in cross-modality information transfer. Proceedings of the Royal Society B: Biological Sciences, 266 (1436): 2427 - 2433. (doi:10.1098/rspb.1999.0942)
15. Haupt, R. L. and Haupt, S. E. 2004. Practical genetic algorithms, 2, Hoboken, NJ: Wiley-Blackwell.
16. Hopper, E. and Turton, B. 1999. A genetic algorithm for a 2D industrial packing problem. Computers & Industrial Engineering, 37 (1-2): 375 - 378. doi:16/S0360-8352(99)00097-2 (doi:10.1016/S0360-8352(99)00097-2)
17. International Organization for Standardization. 2003. Acoustics-Normal equal-loudness-level contours (ISO 226:2003), Geneva, Switzerland: Author.
18. Jaworska, N. and Chupetlovska-Anastasova, A. 2009. A review of multidimensional scaling (MDS) and its utility in various psychological domains. Tutorials in Quantitative Methods for Psychology, 5 (1): 1 - 10.
19. Johnston, V. S. and Franklin, M. 1993. Is beauty in the eye of the beholder?. Ethology and Sociobiology, 14 (3): 183 - 199. doi:10.1016/0162-3095(93)90005-3 (doi:10.1016/0162-3095(93)90005-3)
20. Kaczmarek, K. A., Tyler, M. E., & Bach-y-Rita, P. (1997, 30 October-2 November). Pattern identification on a fingertip-scanned electrotactile display. In R. Jaeger (Ed.), Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Vol. 4, pp. 1694-1696), Chicago, IL. doi: 10.1109/IEMBS.1997.757047
21. Kim, J.-K. and Zatorre, R. J. 2008. Generalized learning of visual-to-auditory substitution in sighted individuals. Brain Research, 1242: 263 - 275. doi:10.1016/j.brainres.2008.06.038 (doi:10.1016/j.brainres.2008.06.038)
22. Kumar, S., Forster, H. M., Bailey, P. and Griffiths, T. D. 2008. Mapping unpleasantness of sounds to their auditory representation. The Journal of the Acoustical Society of America, 124: 3810 doi:10.1121/1.3006380 (doi:10.1121/1.3006380)
23. Kupers, R., Chebat, D. R., Madsen, K. H., Paulson, O. B. and Ptito, M. 2010. Neural correlates of virtual route recognition in congenital blindness. Proceedings of the National Academy of Sciences, 107 (28): 12716 - 12721. doi:10.1073/pnas.1006199107 (doi:10.1073/pnas.1006199107)
24. Lisetti, C. L. and Nasoz, F. 2004. Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP Journal on Advances in Signal Processing, 2004 (11): 1672 - 1687. doi:10.1155/S1110865704406192 (doi:10.1155/S1110865704406192)
25. Marks, L. E., Hammeal, R. J., Bornstein, M. H. and Smith, L. B. 1987. Perceiving similarity and comprehending metaphor. Monographs of the Society for Research in Child Development, 52 (1): 1 - 100. doi:10.2307/1166084 (doi:10.2307/1166084)
26. Meijer, P. 1992. An experimental system for auditory image representations. IEEE Transactions on Biomedical Engineering, 39 (2): 112 - 121. (doi:10.1109/10.121642)
27. Melara, R. D. and O'Brien, T. P. 1987. Interaction between synesthetically corresponding dimensions. Journal of Experimental Psychology: General, 116 (4): 323 - 336. (doi:10.1037/0096-3445.116.4.323)
28. Merabet, L. B., Battelli, L., Obretenova, S., Maguire, S., Meijer, P. and Pascual-Leone, A. 2009. Functional recruitment of visual cortex for sound encoded object identification in the blind. Neuroreport, 20 (2): 132 - 138. doi:10.1097/WNR.0b013e32832104dc (doi:10.1097/WNR.0b013e32832104dc)
29. Mesz, B., Trevisan, M. A. and Sigman, M. 2011. The taste of music. Perception, 40 (2): 209 (doi:10.1068/p6801)
30. Nestor, A. and Tarr, M. J. 2008. The segmental structure of faces and its use in gender recognition. Journal of Vision, 8 (7): 1 - 12. (doi:10.1167/8.7.7)
31. O'Regan, J. K. 1992. Solving the "real" mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology/Revue Canadienne de Psychologie, 46 (3): 461 - 488. doi:10.1037/h0084327 (doi:10.1037/h0084327)
32. Ortiz, T., Poch, J., Santos, J. M., Requena, C., Martínez, A. M. Ortiz-Terán, L. 2011. Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness. PLoS ONE, 6 (8): e23264 doi:10.1371/journal.pone.0023264 (doi:10.1371/journal.pone.0023264)
33. Parise, C. V. and Spence, C. 2009. "When birds of a feather flock together": Synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS ONE, 4 (5): e5664 doi:10.1371/journal.pone.0005664 (doi:10.1371/journal.pone.0005664)
34. Poirier, C., De Volder, A., Tranduy, D., & Scheiber, C. (2007). Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects. Neuropsychologia, 45(5), 1108-1121. doi:10.1016/j.neuropsychologia.2006.09.018
35. Pollok, B., Schnitzler, I., Stoerig, P., Mierdorf, T. and Schnitzler, A. 2005. Image-to-sound conversion: Experience-induced plasticity in auditory cortex of blindfolded adults. Experimental Brain Research, 167 (2): 287 - 291. doi:10.1007/s00221-005-0060-8 (doi:10.1007/s00221-005-0060-8)
36. Pratt, C. C. 1930. The spatial character of high and low tones. Journal of Experimental Psychology, 13 (3): 278 - 285. doi:10.1037/h0072651 (doi:10.1037/h0072651)
37. Renier, L., Laloyaux, C., Collignon, O., Tranduy, D., Vanlierde, A. Bruyer, R. 2005. The Ponzo illusion with auditory substitution of vision in sighted and early-blind subjects. Perception, 34 (7): 857 - 867. doi:10.1068/p5219 (doi:10.1068/p5219)
38. Rojas, M., Masip, D., Todorov, A. and Vitria, J. 2011. Automatic prediction of facial trait judgments: Appearance vs. structural models. PloS One, 6 (8): e23323 (doi:10.1371/journal.pone.0023323)
39. Sims, K. 1991. Artificial evolution for computer graphics. Computer Graphics, 25 (4): 319 - 328. (doi:10.1145/127719.122752)
40. Smith, J. R. 1991. " Designing biomorphs with an interactive genetic algorithm ". In Proceedings of the Fourth International Conference on Genetic Algorithms, 535 - 538. Morgan Kaufmann Publishers.
41. Soddu, C. 2002. New naturality: A generative approach to art and design. Leonardo, 35 (3): 291 - 294. (doi:10.1162/002409402760105299)
42. Song, H. J. and Beilharz, K. 2008. " Aesthetic and auditory enhancements for multi-stream information sonification ". In Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts, DIMEA'08, 224 - 231. New York, NY: ACM. doi:10.1145/1413634.1413678
43. Steinberg, J. C. 1937. Positions of stimulation in the cochlea by pure tones. The Journal of the Acoustical Society of America, 8: 176 doi:10.1121/1.1915891 (doi:10.1121/1.1915891)
44. Stevens, S. S. and Volkmann, J. 1940. The relation of pitch to frequency: A revised scale. The American Journal of Psychology, 53 (3): 329 - 353. doi:10.2307/1417526 (doi:10.2307/1417526)
45. Unger, R. and Moult, J. 1993. Genetic algorithms for protein folding simulations. Journal of Molecular Biology, 231 (1): 75 - 81. doi:10.1006/jmbi.1993.1258 (doi:10.1006/jmbi.1993.1258)
46. Walker, P., Bremner, J. G., Mason, U., Spring, J., Mattock, K. Slater, A. 2010. Preverbal infants' sensitivity to synaesthetic cross-modality correspondences. Psychological Science, 21 (1): 21 - 25. doi:10.1177/0956797609354734 (doi:10.1177/0956797609354734)
47. Ward, J. and Meijer, P. 2010. Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition, 19 (1): 492 - 500. doi:10.1016/j.concog.2009.10.006 (doi:10.1016/j.concog.2009.10.006)
48. Ward, J., Moore, S., Thompson-Lake, D., Salih, S. and Beck, B. 2008. The aesthetic appeal of auditory-visual synaesthetic perceptions in people without synaesthesia. Perception, 37 (8): 1285 - 1296. (doi:10.1068/p5815)
49. Zeki, S. 1999. Inner vision: An exploration of art and the brain, Oxford, UK: Oxford University Press.