Life-long learning Cell Structures - Continuously learning without catastrophic interference

Neural Networks

Volumn 14, Issue 4-5, 2001, Pages 551-573

  Hamker, Fred H ^a  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Catastrophic interference; Cell Structures; Continuously learning; Incremental learning; Life long learning; Radial Basis Function; Stability Plasticity Dilemma

Indexed keywords

CELLS; COMPUTER SIMULATION; INTERFACES (MATERIALS);

PROTOTYPE PATTERNS;

NEURAL NETWORKS;

ADAPTATION; ALGORITHM; ANALYTICAL ERROR; ARTIFICIAL NEURAL NETWORK; ENVIRONMENT; LEARNING; NERVE CELL NETWORK; PLASTICITY; PRIORITY JOURNAL; REVIEW; THEORY;

EID: 0035008798     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(01)00018-1     Document Type: Article

References (58)

1. Ahrns, I., Bruske, J. & Sommer, G. (1995). On-line learning with Dynamic Cell Structures. In Proceedings of the International Conference on Artificial Neural Networks (pp. 141-146).
2. Amari S. A theory of adaptive pattern classifiers. IEEE Transactions on Electronic Computers. 16:1967;299-307.
3. Berthold, M. R. & Diamond, J. (1995). Boosting the performance of RBF networks with dynamic decay adjustment. In Advances in neural information processing systems (NIPS 7) (pp. 521-528). Cambridge: MIT Press.
4. Broomhead D.S., Lowe D. Multivariable functional interpolation and adaptive networks. Complex Systems. 2:1988;321-355.
5. Bruske J., Sommer G. Dynamic cell structure learns perfectly topology preserving map. Neural Computation. 7:1995;845-865.
6. Bruske J., Hansen M., Riehn L., Sommer G. Adaptive saccade control of a binocular head with Dynamic Cell Structures. In Proceedings of the International Conference on Artificial Neural Networks (ICAN'96). 1996;215-220.
7. Bruske, J. (1998). Dynamische Zellstrukturen. Theorie und Anwendung eines KNN-Modells. PhD Thesis, Technische Fakultät der Christian Albrechts-Universität zu Kiel.
8. Bruske J., Ahrns L., Sommer G. An integrated architecture for learning of reactive behaviors based on dynamic cell structures. Robotics and Autonomous Systems. 22:1998;87-102.
9. Carpenter G.A., Grossberg S. ART2: Self-organisation of stable category recognition codes for analog input patterns. Applied Optics. 26:1987;4919-4930.
10. Carpenter G.A., Grossberg S., Reynolds J.H. ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks. 4:1991;543-564.
11. Carpenter G.A., Grossberg S., Markuzon N., Reynolds J.H., Rosen D.B. Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks. 3:1992;698-713.
12. Chen Y.Q., Thomas D.W., Nixon M.S. Generating-shrinking algorithm for learning arbitrary classification. Neural Networks. 7:1994;1477-1489.
13. Eriksson P.S., Perfilieva E., Bjork-Eriksson T., Alborn A.M., Nordborg C., Peterson D.A., Gage F.H. Neurogenesis in the adult human hippocampus. Nature Medicine. 4:1998;1313-1317.
14. Fahlman, S. E. & Lebiere, C. (1990). The cascade-correlation learning architecture. In Advances in neural information processing systems 2 (pp. 524-532). Los Altos, CA: Morgan Kaufmann Publishers.
15. Freeman J.A.S., Saad D. On-line learning in radial basis function networks. Neural Computation. 9:1997;1601-1622.
16. French R.M. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences. 3:1999;128-135.
17. Fritzke, B. (1992). Wachsende Zellstrukturen - ein selbstorganisierendes neuronales Netzwerkmodell. PhD Thesis, Technische Fakultät der Universität Erlangen-Nürnberg.
18. Fritzke B. Growing cell structures - a self-organizing network for unsupervised and supervised learning. Neural Networks. 7:1994;1441-1460.
19. Fritzke, B. (1995). A growing neural gas network learns topologies. In Advances in neural information processing systems (NIPS 7) (pp. 625-632). Cambridge, MA: MIT Press.
20. Fritzke, B. (1997). A self-organizing network that can follow non-stationary distributions. In Proceedings of the International Conference on Artificial Neural Networks (ICAN'97) (pp. 613-618). Springer.
21. Geman S., Bienenstock E., Doursat R. Neural networks and the bias/variance dilemma. Neural Computation. 4:1992;1-58.
22. Gross, H.-M., Stephan, V. & Boehme, H.-J. (1996). Sensory-based robot navigation using self-organizing networks and Q-learning. In Proceedings of the World Congress on Neural Networks (WCNN'96), San Diego (pp. 94-99).
23. Grossberg S. Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors. Biological Cybernetics. 23:1976;121-134.
24. Grossberg S. Nonlinear neural networks: principles, mechanisms, and architectures. Neural Networks. 1:1988;17-61.
25. Grossberg S., Merrill J.W.L. The hippocampus and cerebellum in adaptively timed learning, recognition and movement. Journal of Cognitive Neuroscience. 8:1996;257-277.
26. Hamker, F. H. & Gross, H.-M. (1997). Task-based representation in lifelong learning incremental neural networks. In VDI Fortschrittberichte, Reihe 8, Nr. 663, Workshop SOAVE'97, Ilmenau (pp. 99-108).
27. Hamker, F. H. & Gross, H.-M. (1998). A lifelong learning approach for incremental neural networks. In Proceedings of the Fourteenth European Meeting on Cybernetics and Systems Research (EMCSR'98), Vienna (pp. 599-604).
28. Hamker, F., Debes, K., Pomierski, T., Gross, H.-M., (1998). Multisensorielles Integriertes Realzeit Inspektions-System MIRIS: Lösung der MIKADO-Sortieraufgabe. Schriftenreihe des FG Neuroinformatik der TU Ilmenau, Report. 2/98.
29. Hamker, F. H. (1999). Visuelle Aufmerksamkeit und lebenslanges Lernen im Wahrnehmungs-Handlungs-Zyklus. PhD Thesis, Technische Universität Ilmenau.
30. Hamker, F.H., Paetz, J., Thöne, S., Brause, R., Hanisch, E., (2000). Erkennung kritischer Zustände von Patienten mit der Diagnose 'Septischer Schock' mit einem RBF-Netz. Report. 4/2000, Franfurt am Main: Institute of Imformatik, JW Goethe Iniversität.
31. Heinke D., Hamker F.H. Comparing neural networks: a benchmark on growing neural gas, growing cell structures, and fuzzy ARTMAP. IEEE Transactions on Neural Networks. 9:1998;1279-1291.
32. Heskes, T. M. & Kappen, B. (1993). On-line learning processes in artificial neural networks. In Mathematical foundations of neural networks (pp. 199-233). Amsterdam: Elsevier Science Publishers.
33. Hetherington, P. & Seidenberg, M. (1989). Is there 'catastrophic interference' in connectionist networks? In Proceedings of the 11th Annual Conference of the Cognitive Science Society (pp. 26-33). Hillsdale, NJ: LEA.
34. Karayiannis N.B., Mi G.W. Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques. IEEE Transactions on Neural Networks. 8:1997;1492-1506.
35. Karayiannis N.B. Reformulated radial basis neural networks trained by gradient descent. IEEE Transactions on Neural Networks. 10:1999;657-671.
36. Kohonen T. Self-organized formation of topologically correct feature maps. Biological Cybernetics. 43:1982;59-69.
37. Lim C.P., Harrison R.F. An incremental adaptive network for on-line supervised learning and probability estimation. Neural Networks. 10:1997;925-939.
38. Martinetz, T. M. & Schulten, K. J. (1991). A 'neural gas' network learns topologies. In Artificial neural networks (vol. I, pp. 397-402). Amsterdam: North Holland.
39. Martinetz T.M., Schulten K.J. Topology representing networks. Neural Networks. 4:1994;507-522.
40. Martinetz T.M., Berkovich S., Schulten K.J. Neural-gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks. 4:1993;558-569.
41. McClelland J., McNaughton B., O'Reilly R. Why there are complementary learning systems in the hippocampus and neocortex: insights from the success and failures of connectionist models of learning and memory. Psychological Review. 102:1995;419-457.
42. McCloskey M., Cohen N. Catastrophic interference in connectionist networks: the sequential learning problem. Bower G.H. The psychology of learning and motivation. vol. 24:1989;109-164 Academic Press, New York.
43. Moody, J. & Darken, C. (1988). Learning with localized receptive fields. In Proceedings of the 1988 Connectionist Models Summer School (pp. 133-143). San Mateo: Morgan Kaufmann.
44. Murata, N., Müller, K.-R., Ziehe, A. & Amari, S. (1997). Adaptive on-line learning in changing environments. In Proceedings of the Conference on Neural Information Processing Systems (NIPS 9) (pp.599-604). Cambridge, MA: MIT Press.
45. Obradovic D. On-line training of recurrent neural networks with continuous topology adaptation. IEEE Transactions on Neural Networks. 7:1996;222-228.
46. Park J., Sandberg I.W. Universal approximation using radial-basis-function networks. Neural Computation. 5:1993;305-316.
47. Platt J. A resource-allocating network for function interpolation. Neural Computation. 3:1991;213-225.
48. Poggio T., Girosi F. Regularization algorithms for learning that are equivalent to multilayer networks. Science. 247:1990;978-982.
49. Prechelt, L. (1994). PROBEN1 a set of neural network benchmark problems and benchmarking rules, Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, (anonymous FTP: on ftp.ira.uka.de/pub/uni-karlsruhe/papers/techreports/1994/1994-21.ps.gz).
50. Ratcliff R. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological Review. 97:1990;285-308.
51. Robbins H., Monro S. A stochastic approximation model. Annual Math. Stat. 22:1951;400-407.
52. Roy A., Govil S., Miranda R. A neural-network learning theory and a polynomial time RBF algorithm. IEEE Transactions on Neural Networks. 8:1997;1301-1313.
53. Rumelhart, D. E. Hinton, G. E. & Williams, R. J. (1986). Learning internal representations by error propagation. In Parallel distributed processing (pp. 318-362). Cambridge, MA: MIT Press.
54. Shadafan R.S., Niranjan M. A dynamic neural network architecture by sequential partitioning of the input space. Neural Computation. 6:1994;1202-1223.
55. Sompolinsky, H., Barkai, N. & Seung, H. S. (1995). On-line learning of dichotomies: algorithms and learning curves. In Neural networks: the statistical mechanics perspective (pp. 105-130). Singapore: World Scientific.
56. Specht D.F. Probabilistic neural networks. Neural Networks. 3:1990;109-118.
57. Whitehead B.A., Choate T.D. Evolving space-filling curves to distribute radial basis functions over an input space. IEEE Transactions on Neural Networks. 5:1994;15-23.
58. Yingwei L., Sundararajan N., Saratchandran P. Performance evaluation of a sequential minimal Radial Basis Function (RBF) neural network learning algorithm. IEEE Transactions on Neural Networks. 9:1998;308-318.