Incremental leaning and model selection for radial basis function network through sleep

IEICE Transactions on Information and Systems

Volumn E90-D, Issue 4, 2007, Pages 722-735

  Yamauchi, Koichiro ^a   Hayami, Jiro ^a  

^a HOKKAIDO UNIVERSITY   (Japan)

Author keywords

Incremental learning; Model selection; RBF; Sleep learning

Indexed keywords

DATABASE SYSTEMS; LEARNING SYSTEMS; NEURAL NETWORKS; RADIAL BASIS FUNCTION NETWORKS;

INCREMENTAL LEARNING; MODEL SELECTION; SLEEP LEARNING;

MATHEMATICAL MODELS;

EID: 34247133096     PISSN: 09168532     EISSN: 17451361     Source Type: Journal    
DOI: 10.1093/ietisy/e90-d.4.722     Document Type: Article

References (31)

1. J.E. Moody, The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems, pp.847-854, Morgan Kaufmann Publishers, 1992.
2. G.E. Hinton and D. van Camp, "Keeping neural networks simple by minimizing the description length of the weights," Sixth ACM Conference on Computational Learning Theory, Santa Cruz, pp.5-13, July 1993.
3. N. Murata, S. Yoshizawa, and S. Amari, "Network information criterion: Determining the number of hidden units for an artificial neural network model," IEEE Trans. Neural Netw., vol.5, no.6, pp.865-872, Nov. 1994.
4. P. van de Laar and T. Heskes, "Pruning using parameter and neuronal metrics," Neural Comput., vol.11, pp.977-993, 1999.
5. D.F. Specht, "A general regression neural network," IEEE Trans. Neural Netw., vol.2, no.6, pp.568-576, Nov. 1991.
6. D. Tomandl and A. Schober, "A modified generalized regression neural network (MGRNN) with a new efficient training algorithm as a robust "black-box"-tool for data analysis," Neural Networks, vol.14, pp.1023-1034, 2001.
7. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, Second ed., Wiley-Interscience Publication, 2001.
8. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE International Conference on Neural Networks, vol.78, no.9, pp.1481-1497, Sept. 1990.
9. L. Yingwei, N. Sundararajan, and P. Saratchandran, "A sequential learning scheme for function approximation using minimal radial basis function neural networks," Neural Comput., vol.9, pp.461-478, 1997.
10. M. Ishikawa, "Structural learning with forgetting," Neural Netw., vol.9, no.3, pp.509-521, 1996.
11. N. Kasabov, "Evolving fuzzy neural networks for supervised/ unsupervised online knowledge-based learning," IEEE Trans. Syst. Man Cybern., vol.31, no.6, pp.902-918, Dec. 2001.
12. S. Schaal and C.G. Atkeson, "Constructive incremental learning from only local information," Neural Comput., vol.10, no.8, pp.2047-2084, Nov. 1998.
13. T. Hoya and J.A. Chambers, "Heuristic pattern correction scheme using adaptively trained generalized regression neural networks," IEEE Trans. Neural Netw., vol.12, no.1, pp.91-100, Jan. 2001.
14. I. Hayashi, H. Maeda, and J.R. Williamson, "A formulation of receptive field type input layer for tam network using gabor function," IEEE International Conference on Fuzzy Systems, #1335, July 2004.
15. J. Moody and C.J. Darken, "Fast learning in neural networks of locally-tuned processing units," Neural Comput., vol.1, pp.281-294, 1989.
16. R.M. French, "Pseudo-recurrent connectionist networks: An approach to the "sensitivity stability" dilemma," Connection Science, vol.9, no.4, pp.353-379, 1997.
17. B. Ans and S. Roussert, "Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting," Connection Science, vol.12, no.1, pp.1-19, 2000.
18. K. Yamauchi, S. Itho, and N. Ishii, "Combination of fast and slow learning neural networks for quick adaptation and pruning redundant cells," IEEE SMC'99, 1999 IEEE System, Man and Cybernetics Conference, vol.III, pp.390-395, Oct. 1999.
19. K. Yamauchi, S. Ito, and N. Ishii, "Wake-sleep learning method for quick adaptation and reduction of redundant cells," ICONIP 2000 7th International Conference on Neural Information Processing, vol.1, pp.559-564, Nov. 2000.
20. K. Yamauchi, "Sequential learning and model selection with sleep," ICONIP 2001 8th International Conference on Neural Information Processing, ed. L. Zhang and F. Gu, vol.1, pp.205-210, Fudan University Press, Nov. 2001.
21. K. Yamauchi and N. Kobayashi, "Incremental learning with sleep-learning of noiseless datasets," International Conference on Neural Information Processing ICONIP2002, vol.1, pp.398-403, Nov. 2002.
22. K. Yamauchi and J. Hayami, "Sleep learning," IEEE World Congress on Computational Intelligence (WCCI2006), pp.6295-6302, July 2006.
23. J. Ma, J. Theiler, and S. Perkins, "Accurate on-line support vector regression," Neural Comput., vol.15, pp.2683-2703, 2003.
24. C. Burges, "Geometory and invariance in kernel based methods," in Advances in Kernel Methods - Support Vector Learning, ed. B. Schölkoph, C. Burges, and A. Smola, pp. 144-152, MIT Press, 1999.
25. M. Umano, Y. Hosoya, Y Uno, K. Seta, and M. Okada, "Incremental learning of fuzzy rules by fuzzy neural network with forgetting facility," 19th Fuzzy System Symposium, pp.665-668, Japan Society for Fuzzy Theory and Intelligent Informatics, 2003.
26. K. Yamauchi, N. Yamaguchi, and N. Ishii, "Incremental learning methods with retrieving interfered patterns," IEEE Trans. Neural Netw., vol.10, no.6, pp.1351-1365, Nov. 1999.
27. H. Akaike, "A new look at the statistical model identification, " IEEE Trans. Autom. Control, vol.AC-19, no.6, pp.716-723, Dec. 1974.
28. J.L. McClelland, R.C. O'Reilly, and B.L. McNaughton, "Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory," Psychological Review, vol.102, no.3, pp.419-457, 1995.
29. K.A. Norman, E.L. Newman, and A.J. Perotte, "Methods for reducing interference in the complementary learning systems mode: Ocillating inhibition and autonomous memory rehearsal," Neural Netw., vol.18, no.9, pp.1212-1228, Nov. 2005.
30. A.S. Dave and D. Margoliash, "Song replay during sleep and computational rules for sensorymotor vocal learning," Sciense, vol.290, pp.812-816, Oct. 2000.
31. K. Yamauchi, "Incremental learning and dimension selection with sleep," International Conference on Neural Information Processing ICONIP2004, LNCS3316, pp.489-494, Nov. 2004.