Neural networks learn highly selective representations in order to overcome the superposition catastrophe

Psychological Review

Volumn 121, Issue 2, 2014, Pages 248-261

  Bowers, Jeffrey S ^a   Vankov, Ivan I ^a   Damian, Markus F ^a   Davis, Colin J ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

Distributed representations; Grandmother cells; Localist representations; Short term memory; Superposition catastrophe

Indexed keywords

ARTIFICIAL NEURAL NETWORK; ATTENTION; BRAIN CORTEX; COGNITION; HUMAN; PERCEPTION; PHYSIOLOGY; PSYCHOPHYSIOLOGY; SHORT TERM MEMORY;

ATTENTION; CEREBRAL CORTEX; COGNITION; HUMANS; MEMORY, SHORT-TERM; NEURAL NETWORKS (COMPUTER); PERCEPTION; PSYCHOPHYSIOLOGY;

EID: 84898852427     PISSN: 0033295X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0035943     Document Type: Article

References (30)

1. Berkeley, I. S. N., Dawson, M. R. W., Medler, D. A., Schopflocher, D. P., & Hornsby, L. (1995). Density plots of hidden unit activations reveal interpretable bands. Connection Science, 7, 167-186. doi:10.1080/ 09540099550039336.
2. Botvinick, M. M., & Plaut, D. C. (2006). Short-term memory for serial order: A recurrent neural network model. Psychological Review, 113, 201-233. doi:10.1037/0033-295X.113.2.201.
3. Botvinick, M. M., & Plaut, D. C. (2009). Empirical and computational support for context-dependent representations of serial order: Reply to Bowers, Damian, and Davis (2009). Psychological Review, 116, 998- 1002. doi:10.1037/a0017113.
4. Bowers, J. S. (2002). Challenging the widespread assumption that connectionism and distributed representations go hand-in-hand. Cognitive Psychology, 45, 413-445. doi:10.1016/S0010-0285(02)00506-6.
5. Bowers, J. S. (2009). On the biological plausibility of grandmother cells: Implications for neural network theories in psychology and neuroscience. Psychological Review, 116, 220-251. doi:10.1037/a0014462.
6. Bowers, J. S. (2010). More on grandmother cells and the biological implausibility of PDP models of cognition: A reply to Plaut and Mc- Clelland (2010) and Quian Quiroga and Kreiman (2010). Psychological Review, 117, 300-306. doi:10.1037/a0018047.
7. Bowers, J. S. (2011). What is a grandmother cell? And how would you know if you found one? Connection Science, 23, 91-95. doi:10.1080/ 09540091.2011.568608.
8. Bowers, J. S., Damian, M. F., & Davis, C. J. (2009). A fundamental limitation of the conjunctive codes learned in PDP models of cognition: Comment on Botvinick and Plaut (2006). Psychological Review, 116, 986-997. doi:10.1037/a0017097.
9. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87-114. doi:10.1017/S0140525X01003922.
10. Elliott, C. J. H., & Susswein, A. J. (2002). Comparative neuroethology of feeding control in molluscs. Journal of Experimental Biology, 205, 877-896.
11. Grossberg, S. (1980). How does a brain build a cognitive code? Psychological Review, 87, 1-51. doi:10.1037/0033-295X.87.1.1.
12. Hummel, J. E. (2000). Localism as a first step toward symbolic representation. Behavioral and Brain Sciences, 23, 480-481. doi:10.1017/ S0140525X0036335X.
13. Hummel, J. E., & Biederman, I. (1992). Dynamic binding in a neural network for shape recognition. Psychological Review, 99, 480-517. doi:10.1037/0033-295X.99.3.480.
14. Hummel, J. E., & Holyoak, K. J. (2003). A symbolic-connectionist theory of relational inference and generalization. Psychological Review, 110, 220-264. doi:10.1037/0033-295X.110.2.220.
15. Lennie, P. (2003). The cost of cortical computation. Current Biology, 13, 493-497. doi:10.1016/S0960-9822(03)00135-0.
16. Logothetis, N. K., Pauls, J., & Poggio, T. (1995). Shape representation in the inferior temporal cortex of monkeys. Current Biology, 5, 552-563. doi:10.1016/S0960-9822(95)00108-4.
17. Marr, D. (1971). Simple memory: A theory for archicortex. Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences, 262, 23-81.
18. McClelland, J. L., St. John, M., & Taraban, R. (1989). Sentence comprehension: A parallel distributed processing approach. Language and Cognitive Processes, 4, 287-335. doi:10.1080/01690968908406371.
19. McCloskey, M., & Cohen, N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 109-165). New York, NY: Academic Press.
20. Olshausen, B. A., & Field, D. J. (2004). Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14, 481-487. doi:10.1016/j.conb .2004.07.007.
21. Page, M. P. A. (2000). Connectionist modeling in psychology: A localist manifesto. Behavioral and Brain Sciences, 23, 443-467. doi:10.1017/ S0140525X00003356.
22. Plaut, D. C., & McClelland, J. L. (2000). Stipulating versus discovering representations. Behavioral and Brain Sciences, 23, 489-491. doi: 10.1017/S0140525X00473358.
23. Plaut, D. C., & McClelland, J. L. (2010). Locating object knowledge in the brain: Comment on Bowers's (2009) attempt to revive the grandmother cell hypothesis. Psychological Review, 117, 284-288. doi:10.1037/ a0017101.
24. Pouget, A., Dayan, P., & Zemel, R. (2000). Information processing with population codes. Nature Reviews Neuroscience, 1, 125-132. doi: 10.1038/35039062.
25. Quian Quiroga, R., & Kreiman, G. (2010). Measuring sparseness in the brain: Comment on Bowers (2009). Psychological Review, 117, 291- 297. doi:10.1037/a0016917.
26. Quian Quiroga, R., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005, June 23). Invariant visual representation by single neurons in the human brain. Nature, 435, 1102-1107. doi:10.1038/nature03687.
27. Rosenblatt, F. (1961). Principles of neurodynamics: Perceptrons and the theory of brain mechanisms. Washington, DC: Spartan Books.
28. Thorpe, S. J. (2011). Grandmother cells, neocortical dark matter and very long term visual memories. Journal of Vision, 11(11), Article 1243. doi:10.1167/11.11.1243
29. Touretzky, D. S., & Hinton, G. E. (1988). A distributed connectionist production system. Cognitive Science, 12, 423- 466. doi:10.1207/ s15516709cog1203_4.
30. Von der Malsburg, C. (1986). Am I thinking assemblies? In G. Palm & A. Aertsen (Eds.), Brain theory (pp. 161-176). Berlin, Germany: Springer.