Deep generative learning of location-invariant visual word recognition

Frontiers in Psychology

Volumn 4, Issue SEP, 2013, Pages

  Di Bono, Maria Grazia ^a   Zorzi, Marco ^a,b  

^a UNIVERSITY OF PADOVA   (Italy)

^b IRCCS San Camillo Neurorehabilitation Hospital   (Italy)

Author keywords

Connectionist modeling; Deep unsupervised learning; Hierarchical generative models; Open bigrams; Orthographic coding

Indexed keywords

EID: 84885334343     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2013.00635     Document Type: Article

References (53)

1. Cohen, L., and Dehaene, S. (2004). Specialization within the ventral stream:the case for the visual word form area. Neuroimage 22, 466-476. doi:10.1016/j.neuroimage.2003.12.049.
2. Cohen, L., Lehéricy, S., Chochon, F., Lemer, C., Rivaud, S., and Dehaene, S. (2002). Language-specific tuning of visual cortex. Functional properties of the visual word form area. Brain 125, 1054-1069. doi:10.1093/brain/awf094.
3. Coltheart, M., Rastle, K., Perry, C., Langdon, R., and Ziegler, J. (2001). DRC:a dual route cascaded model of visual word recognition and reading aloud. Psychol. Rev. 108, 204. doi:10.1037/0033-295X.108.1.204.
4. Dandurand, F., Grainger, J., and Dufau, S. (2010). Learning location-invariant orthographic representations for printed words. Connect. Sci. 22, 25-42. doi:10.1080/09540090903085768.
5. Dandurand, F., Grainger, J., and Duñabeitia, J. A. Granier, J. P. (2011). On coding non-contiguous letter combinations. Front. Psychol. 2:136. doi:10.3389/fpsyg.2011.00136.
6. Davis, C. J. (2010). The spatial coding model of visual word identification. Psychol. Rev. 117, 713-758. doi:10.1037/a0019738.
7. Davis, C. J., and Bowers, J. S. (2006). Contrasting five theories of letter position coding:evidence from orthographic similarity effects. J. Exp. Psychol. Hum. Percept. Perform. 32, 535-557. doi:10.1037/0096-1523.32.3.535.
8. De Filippo De Grazia, M., Cutini, S., Lisi, M., and Zorzi, M. (2012a). Space coding for sensorimotor transformations can emerge through unsupervised learning. Cogn. Process. 13, S141-146. doi:10.1007/s10339-012-0478-4.
9. De Filippo De Grazia, M., Stoianov, I., and Zorzi, M. (2012b). "Parallelization of deep networks," in Proceedings of the 2012 European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning-ESANN (Bruges), 621-626.
10. Dehaene, S., and Cohen, L. (2011). The unique role of the visual word form area in reading. Trends Cogn. Sci. 15, 254-262. doi:10.1016/j.tics.2011.04.003.
11. Dehaene, S., Cohen, L., Sigman, M., and Vinckier, F. (2005). The neural code for written words:a proposal. Trends Cogn. Sci. 9, 335-341. doi:10.1016/j.tics.2005.05.004.
12. Fischer-Baum, S., Charny, J., and McCloskey, M. (2011). Both-edges representation of letter position in reading. Psychon. Bul. Rev. 18, 1083-1089. doi:10.3758/s13423-011-0160-3.
13. Ghahramani, Z., Korenberg, A. T., and Hinton, G. E. (1999). "Scaling in a hierarchical unsupervised network," in Proceedings of the 1999 Ninth International Conference on Artificial Neural Networks-ICANN (Edinburgh), 13-18.
14. Gomez, P., Ratcliff, R., and Perea, M. (2008). The overlap model:a model of letter position coding. Psychol. Rev. 115, 577-600. doi:10.1037/a0012667.
15. Grainger, J. (2008). Cracking the orthographic code:an introduction. Lang. Cogn. Proc. 23, 1-35. doi:10.1080/01690960701578013.
16. Grainger, J., Dufau, S., Montant, M., Ziegler, J. C., and Fagot, J. (2012). Orthographic processing in baboons (Papio papio). Science 336, 245-248. doi:10.1126/science.1218152.
17. Grainger, J., Granier, J. P., Farioli, F., Van Assche, E., and van Heuven, W. (2006). Letter position information and printed word perception:the relative-position priming constraint. J. Exp. Psychol. Hum. Percept. Perform. 32, 865-884. doi:10.1037/0096-1523.32.4.865.
18. Grainger, J., and van Heuven, W. J. B. (2003). "Modeling letter position coding in printed word perception," in The Mental lexicon, ed P. Bonin (NewYork, NY:Nova Science Publishers), 1-23.
19. Grainger, J., and Whitney, C. (2004). Does the huamn mnid raed wrods as awlohe. Trends Cogn. Sci. 8, 58-59. doi:10.1016/j.tics.2003.11.006.
20. Grainger, J., and Ziegler, J. C. (2011). A dual-route approach to orthographic processing. Front. Psychol. 2:54. doi:10.3389/fpsyg.2011.00054.
21. Hannagan, T., Dandurand, F., and Grainger, J. (2011). Broken symmetries in a location-invariant word recognition network. Neural Comput. 23, 251-283. doi:10.1162/NECO_a_00064.
22. Harm, M. W., and Seidenberg, M. S. (1999). Phonology, reading acquisition, and dyslexia:insights from connectionist models. Psychol. Rev. 106, 491-528. doi:10.1037/0033-295X.106.3.491.
23. Hertz, J. A., Krogh, A. S., and Palmer, R. G. (1991). Introduction to the Theory of Neural Computation. Redwood City, CA:Addison-Weasley.
24. Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771-1800. doi:10.1162/089976602760128018.
25. Hinton, G. (2013). Where do features come from. Cogn. Sci. doi:10.1111/cogs.12049. [Epub ahead of print].
26. Hinton, G. E. (2007). Learning multiple layers of representation. Trends Cogn. Sci. 11, 428-434. doi:10.1016/j.tics.2007.09.004.
27. Hinton, G. E. (2012). "A practical guide to training Restricted Boltzmann Machines," in Neural Networks:Tricks of the Trade, eds G. Montavon, G. Orr and K. R. Müller (Berlin; Heidelberg:Springer), 599-619.
28. Hinton, G. E., and Salakhutdinov, R. (2006). Reducing the dimensionality of data with neural networks. Science 313, 504-507. doi:10.1126/science.1127647.
29. Humphreys, G. W., Evett, L. J., and Quinlan, P. T. (1990). Orthographic processing in visual word identification. Cogn. Psychol. 22, 517-560. doi:10.1016/0010-028590012-S.
30. Lee, H., Ekanadham, C., and Ng, A. Y. (2008). Sparse deep belief net models for visual area V2. Adv. Neural Inf. Process. Syst. 20, 873-880.
31. McClelland, J. L., and Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception:part 1. An account of basic findings. Psychol. Rev. 88, 375-407. doi:10.1037/0033-295X.88.5.375.
32. O'Reilly, R. C. (1998). Six principles for biologically based computational models of cortical cognition. Trends Cogn. Sci. 2, 455-462. doi:10.1016/S1364-6613(98)01241-8.
33. Pelli, D. G., Farell, B., and Moore, D. C. (2003). The remarkable inefficiency of word recognition. Nature 423, 752-756. doi:10.1038/nature01516.
34. Perea, M., and Lupker, S. J. (2003). "Transposed-letter confusability effects in masked form priming," in Masked Priming:State of the Art, eds S. Kinoshita and S. J. Lupker (Hove:Psychology Press), 97-120.
35. Peressotti, F., and Grainger, J. (1999). The role of letter identity and letter position in orthographic priming. Percept. Psychophys. 61, 691-706. doi:10.3758/BF03205539.
36. Perry, C., Ziegler, J. C., and Zorzi, M. (2007). Nested incremental modelling in the development of computational theories:the CDP+ model of reading aloud. Psychol. Rev. 114, 273. doi:10.1037/0033-295X.114.2.273.
37. Perry, C., Ziegler, J. C., and Zorzi, M. (2010). Beyond single syllables:large-scale modelling of reading aloud with the connectionist dual process (CDP++) model. Cogn. Psychol. 61, 106-151. doi:10.1016/j.cogpsych.2010.04.001.
38. Perry, C., Ziegler, J. C., and Zorzi, M. (2013). A computational and empirical investigation of graphemes in reading. Cogn. Sci. 37, 800-828. doi:10.1111/cogs.12030.
39. Plaut, D. C., McClelland, J. L., Seidenberg, M. S., and Patterson, K. (1996). Understanding normal and impaired word reading:computational principles in quasi-regular domains. Psychol. Rev. 103, 56. doi:10.1037/0033-295X.103.1.56.
40. Riesenhuber, M., and Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019-1025. doi:10.1038/14819.
41. Rumelhart, D. E., and McClelland, J. L. (1982). An interactive activation model of context effects in letter perception:II. The contextual enhancement effect and some tests and extensions of the model. Psychol. Rev. 89, 60-94. doi:10.1037/0033-295X.89.1.60.
42. Schoonbaert, S., and Grainger, J. (2004). Letter position coding in printed word perception:effects of repeated and transposed letters. Lang. Cogn. Process. 19, 333-367. doi:10.1080/01690960344000198.
43. Seidenberg, M. S., and McClelland, J. L. (1989). A distributed, developmental model of word recognition and naming. Psychol. Rev. 96, 523. doi:10.1037/0033-295X.96.4.523.
44. Stoianov, I., and Zorzi, M. (2012). Emergence of a "visual number sense" in hierarchical generative models. Nat. Neurosci. 15, 194-196. doi:10.1038/nn.2996.
45. Testolin, A., Stoianov, I., De Filippo De Grazia, M., and Zorzi, M. (2013). Deep unsupervised learning on a desktop PC:a primer for cognitive scientists. Front. Psychol. 4:251. doi:10.3389/fpsyg.2013.00251.
46. Vinckier, F., Dehaene, S., Jobert, A., Dubus, J. P., Sigman, M., and Cohen, L. (2007). Hierarchical coding of letter strings in the ventral stream:dissecting the inner organization of the visual word-form system. Neuron 55, 143-156. doi:10.1016/j.neuron.2007.05.031.
47. Whitney, C. (2001). How the brain encodes the order of letters in a printed word:the SERIOL model and selective literature review. Psychon. Bull. Rev. 8, 221-243. doi:10.3758/BF03196158.
48. Yarkoni, T., Balota, D., and Yap, M. (2008). Moving beyond Coltheart's N:a new measure of orthographic similarity. Psychon. Bull. Rev. 15, 971-979. doi:10.3758/PBR.15.5.971.
49. Ziegler, J. C., Perry, C., and Zorzi, M. (in press). Modelling reading development through phonological decoding and self-teaching:implications for dyslexia. Proc. R. Soc. Lond. B.
50. Zorzi, M. (2005). "Computational models of reading," in Connectionist Models in Cognitive Psychology, ed G. Houghton (Hove, UK:Psychology Press), 403-444.
51. Zorzi, M. (2010). The connectionist dual process (CDP) approach to modelling reading aloud. Eur. J. Cogn. Psychol. 22, 836-860. doi:10.1080/09541440903435621.
52. Zorzi, M., Houghton, G., and Butterworth, B. (1998). Two routes or one in reading aloud. A connectionist dual-process model. J. Exp. Psychol. Hum. Percept. Perform. 24, 1131. doi:10.1037/0096-1523.24.4.1131.
53. Zorzi, M., Testolin, A., and Stoianov, I. (2013). Modeling language and cognition with deep unsupervised learning:a tutorial overview. Front. Psychol. 4:515. doi:10.3389/fpsyg.2013.00515.