Learning, memory, and the role of neural network architecture

PLoS Computational Biology

Volumn 7, Issue 6, 2011, Pages

  Hermundstad, Ann M ^a   Brown, Kevin S ^a   Bassett, Danielle S ^a   Carlson, Jean M ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMMERCE; MEMORY ARCHITECTURE; NETWORK ARCHITECTURE; NETWORK LAYERS; NEURAL NETWORKS;

EXTERNAL INFORMATIONS; INFORMATION PROCESSING SYSTEMS; LAYERED NETWORK; LEARNING MEMORY; NEURAL NETWORK ARCHITECTURE; PARALLEL NETWORK; PERFORMANCE; SEQUENTIAL TASK; SYSTEMS ARCHITECTURE; UNDERLYING SYSTEMS;

ERRORS;

ACCURACY; ARTICLE; ARTIFICIAL NEURAL NETWORK; BACK PROPAGATION; INFORMATION RETRIEVAL; INFORMATION STORAGE; LEARNING ALGORITHM; MEMORY CONSOLIDATION; POSITIVE FEEDBACK; TASK PERFORMANCE; VELOCITY; BIOLOGICAL MODEL; BIOLOGY; HUMAN; LEARNING; MEMORY; NERVE CELL NETWORK;

COMPUTATIONAL BIOLOGY; HUMANS; LEARNING; MEMORY; MODELS, NEUROLOGICAL; NERVE NET; NEURAL NETWORKS (COMPUTER);

EID: 79959819850     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1002063     Document Type: Article

References (74)

1. Mountcastle VB, (1997) The columnar organization of the neocortex. Brain 120: 701-722.
2. Jain AK, Murty MN, Flynn PJ, (1999) Data clustering: a review. ACM Comput Surv 31: 264-323.
3. Egmont-Petersen M, de Ridder D, Handels H, (2002) Image processing with neural networks-a review. Pattern Recognit 35: 2279-2301.
4. Zhang G, Patuwo BE, Hu MY, (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14: 35-62.
5. Chittka L, Niven JJ, (2009) Are bigger brains better? Current Biology 19: R99535-R1008.
6. Honey CJ, (2009) Predicting human resting-state functional connectivity from structual connectivity. Proc of the Natl Acad of Sci 106: 2035-2040.
7. Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A, (2003) Spontaneously emerging cortical representations of visual attributes. Nature 425: 954-956.
8. McIntosh AR, Rajah MN, Lobaugh NJ, (2003) Functional connectivity of the medial temporal lobe relates to learning and awareness. J Neurosci 23: 6520-6528.
9. Scholz J, Klein MC, Behrens TEJ, Johansen-Berg H, (2009) Training induces changes in whitematter architecture. Nat Neurosci 12: 1370-1371.
10. Bassett DS, Greenfield DL, Meyer-Lindenberg A, Weinberger DR, Moore SW, et al. (2010) Efficient physical embedding of topologically complex information processing networks in brains and computer circuits. PLoS Comput Biol 6: e1000748.
11. Kaiser M, Hilgetag CC, (2010) Optimal hierarchical modular topologies for producing limited sustained activation of neural networks. Front Neuroinformatics 4: 1-14.
12. Reid AT, Krumnack A, Wanke E, Kotter R, (2009) Optimization of cortical hierarchies with continuous scales and ranges. Neuro Image 47: 611-617.
13. Ress D, Glover GH, Liu J, Wandell B, (2007) Laminar profiles of functional activity in the human brain. Neuroimage 34: 74-84.
14. Atencio CA, Schreiner CE, (2007) Columnar connectivity and laminar processing in cat primary auditory cortex. PLoS ONE 5: e9521.
15. Bakoglu HB, (1990) Circuits, Interconnections, and Packaging for VLSI Boston Addison Wesley.
16. Galushkin AI, (2007) Neural Networks Theory Secaucus, NJ Springer-Verlag New York.
17. Fukushima K, (1988) Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Networks 1: 119-130.
18. Robinson AJ, (1994) An application of recurrent nets to phone probability estimation. IEEE Trans Neural Netw 5: 298-305.
19. McCloskey M, Cohen NJ, (1989) Catastrophic interference in connectionist networks: The sequential learning problem. In: Bower GH, editors. The Psychology of Learning and Motivation Academic Press, volume 24 pp. 109-159.
20. Ratcliff R, (1990) Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychol Rev 97: 285-308.
21. Sharkey NE, Sharkey AJC, (1995) An analysis of catastrophic interference. Conn Sci 7: 301-329.
22. Robins A, (1995) Catastrophic forgetting, rehearsal, and pseudorehearsal. Connection Science 7: 123-146.
23. Robins A, McCallum S, (1998) Catastrophic forgetting and the pseudorehearsal solution in hopfieldtype networks. Conn Sci 10: 121-135.
24. Auer P, Herbster M, Warmuth MK, (1996) Exponentially many local minima for single neurons. In: Mozer M, Touretzky DS, Perrone M, editors. Advances in Neural Information Processing Systems Cambridge, MA MIT Press, volume 8 pp. 315-322.
25. Fukumizu K, Amari S, (2000) Local minima and plateaus in hierarchical structures of multilayer perceptrons. Neural Networks 13: 317-327.
26. Rojas R, (1996) Neural Networks: A Systematic Introduction New York Springer-Verlag.
27. Cohen IL, (1994) An artificial neural network analogue of learning in autism. Biol Psychiatry 36: 5-20.
28. Fletcher R, Reeves CM, (1964) Function minimization by conjugate gradients. Comput J 7: 149-154.
29. Polak E, Ribiere G, (1969) Note sur la convergence de methodes de directions conjugees. Rev Franc Inform Rech Oper 16: 35-43.
30. Powell MJD, (1986) Convergence properties of algorithms for nonlinear optimization. SIAM Rev 28: 487-500.
31. Brown KS, Sethna JP, (2003) Statistical mechanical approaches to models with many poorly known parameters. Phys Rev E 68: 021904.
32. Brown KS, Hill CC, Calero GA, Myers CR, Lee KH, et al. (2004) The statistical mechanics of complex signaling networks: nerve growth factor signaling. Phys Biol 1: 184-195.
33. Fletcher R, (1987) Practical Methods of Optimization New York Wiley-Interscience, 2 edition.
34. Mello PA, Kuma N, (2004) Quantum transport in mesoscopic systems: complexity and statistical fluctuations New York Oxford University Press.
35. Bettencourt LM, Stephens GJ, Ham MI, Gross GW, (2007) Functional structure of cortical neuronal networks grown in vitro. Phys Rev E 75: 021915.
36. Achard S, Salvador R, Whitcher B, Suckling J, Bullmore E, (2006) A resilient, low-frequency, smallworld human brain functional network with highly connected association cortical hubs. J Neurosci 26: 63-72.
37. Hagmann P, Cammoun L, Gigandet X, Meuli R, Honey CJ, et al. (2008) Mapping the structural core of human cerebral cortex. PLoS Biol 6: e159.
38. Kim H, (2010) Dissociating the roles of the default-mode, dorsal, and ventral networks in episodic memory retrieval. Neuroimage 50: 1648-1657.
39. Chen ZJ, He Y, Rosa-Neto P, Germann J, Evans AC, (2008) Revealing modular architecture of human brain structural networks by using cortical thickness from MRI. Cereb Cortex 18: 2374-2381.
40. Meunier D, Achard S, Morcom A, Bullmore E, (2009) Age-related changes in modular organization of human brain functional networks. Neuroimage 44: 715-723.
41. Bassett DS, Brown JA, Deshpande V, Carlson JM, Grafton ST, (2011) Conserved and variable architecture of human white matter connectivity. Neuroimage 54: 1262-1279.
42. Felleman DJ, van Essen DC, (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1: 1-47.
43. Meunier D, Lambiotte R, Bullmore ET, (2010) Modular and hierarchically modular organization of brain networks. Front Neurosci 4.
44. Fu HC, Lee YP, Chiang CC, Pao HT, (2001) Divide-and-conquer learning and modular perceptron networks. IEEE Transactions on Neural Netw 12: 250-263.
45. Ersoy OK, Deng SW, (1995) Parallel, self-organizing, hierarchical neural networks with continuous inputs and outputs. IEEE Trans Neural Netw 6: 1037-1044.
46. Oshima H, Odagaki T, (2007) Storage capacity and retrieval time of small-world neural networks. Phys Rev E 76: 036114.
47. Dominguez D, González M, Serrano E, Rodríguez FB, (2009) Structured information in small-world neural networks. Phys Rev E 79: 021909.
48. Larochelle H, Bengio Y, Louradour J, Lamblin P, (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10: 1-40.
49. Bengio Y, LeCun Y, (2007) Scaling learning algorithms toward AI. Large Scale KernelMachines Cambridge, MA MIT Press pp. 321-360.
50. Bengio Y, (2009) Learning deep architectures for AI. Found Trends Mach Learn 2: 1-127.
51. Atallah HE, Frank MJ, O'Reilly RC, (2004) Hippocampus, cortex, and basal ganglia: Insights from computational models of complementary learning systems. Neurobiol Learn Mem 82: 253-267.
52. Bullmore E, Sporns O, (2009) Complex brain networks: graph theoretical analysis of structural and functional systems. Nat Rev Neurosci 10: 186-198.
53. Tononi G, Sporns O, Edelman GM, (1994) A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc Natl Acad Sci 91: 5033-5037.
54. Turrigiano GG, Leslie KR, Desai NS, Rutherford LC, Nelson SB, (1998) Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391: 892-896.
55. Turrigiano GG, Nelson SB, (2004) Homeostatic plasticity in the developing nervous system. Nat Rev Neurosci 5: 97-107.
56. Markram H, Tsodyks M, (1996) Redistribution of synaptic efficacy between neocortical pyramidal neurons. Nature 382: 807-810.
57. Abbott LF, Nelson SB, (2000) Synaptic plasticity: Taming the beast. Nat Neurosci 3: 1178-1183.
58. Bogacz R, Wagenmakers EJ, Forstmann BU, Nieuwenhuis S, (2009) The neural basis of the speedaccuracy tradeoff. Trends Neurosci 33: 10-16.
59. van Veen V, Krug MK, Carter CS, (2008) The neural and computational basis of controlled speedaccuracy tradeoff during task performance. J Cognitive Neurosci 20: 1952-1965.
60. Xu T, Yu X, Perlik AJ, Tobin WF, Zweig JA, et al. (2009) Rapid formation and selective stabilization of synapses for enduring motor memories. Nature 462: 915-919.
61. Alstott J, Breakspear M, Hagmann P, Cammoun L, Sporns O, (2009) Modeling the impact of lesions in the human brain. PLoS Comp Biol 5: e1000408.
62. Honey CJ, Sporns O, (2008) Dynamical consequences of lesions in cortical networks. Hum Brain Mapp 29: 802-809.
63. Allred RP, Adkins D, Woodlee MT, Husbands LC, Maldonado MA, et al. (2008) The vermicelli handling test: A simple quantitative measure of dexterous forepaw function in rats. J Neurosci Methods 170: 229-244.
64. Cucker F, Smale S, (2001) On the mathematical foundations of learning. Bull Amer Math Soc 39: 1-49.
65. Bousquet O, Boucheron S, Lugosi G, (2004) Introduction to statistical learning theory. Advanced Lectures on Machine Learning Springer Berlin, volume 3176 pp. 169-207.
66. Hinton GE, Osindero S, Teh YW, (2006) A fast learning algorithm for deep belief nets. Neural Comput 18: 1527-1554.
67. Marder E, Abbott LF, Turrigiano GG, Liu Z, Golowasch J, (1996) Memory from the dynamics of intrinsic membrane currents. Proc Natl Acad Sci 93: 13481-13486.
68. Gaiteri C, Rubin JE, (2011) The interaction of intrinsic brain dynamics and network topology in determining network burst synchrony. Front Comput Neurosci 5: 1-14.
69. Bush P, Sejnowski T, (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. J Comput Neurosci 3: 91-110.
70. Roelfsema PR, Engel AK, Konig P, Singer W, (1997) Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature 385: 157-161.
71. Vogels TP, Abbott LF, (2005) Signal propagation and logic gating in networks of integrate-and-fire neurons. J Neurosci 25: 10786-10795.
72. Rubinov M, Sporns O, van Leeuwen C, Breakspear M, (2009) Symbiotic relationship between brain structure and dynamics. BMC Neuroscience 10: 1-18.
73. Sanchez-Vives MV, McCormick DA, (2000) Cellular and network mechanisms of rythmic recurrent activity in the neocortex. Nat Neurosci 3: 1027-1034.
74. Tort ABL, Komorowski RW, Manns JR, Kopell NJ, Eichenbaum H, (2009) Theta-gamma coupling increases during the learning of item-content associations. Proc Natl Acad Sci 106: 20942-20947.