RKHS-based functional analysis for exact incremental learning

Neurocomputing

Volumn 29, Issue 1-3, 1999, Pages 85-113

  Vijayakumar, Sethu ^a   Ogawa, Hidemitsu ^b  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Functional analysis; Generalization; Incremental learning; Model selection; Reproducing kernel Hilbert space (RKHS)

Indexed keywords

APPROXIMATION THEORY; FUNCTION EVALUATION; MATHEMATICAL MODELS; MATHEMATICAL OPERATORS; NEURAL NETWORKS; ONLINE SYSTEMS; OPTIMAL SYSTEMS; OPTIMIZATION; SET THEORY;

INCREMENTAL LEARNING; MODEL SELECTION; REPRODUCING KERNEL HILBERT SPACES;

LEARNING SYSTEMS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SYSTEM; DATA ANALYSIS; INFORMATION SYSTEM; LEARNING; PRIORITY JOURNAL;

EID: 0032821057     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(99)00112-5     Document Type: Article

References (30)

1. A. Albert, Regression and the Moore-Penrose Pseudoinverse, Academic Press, New York and London, 1972.
2. Aronszajn N. Theory of reproducing kernels. Trans. Amer. Math. Soc. 68:1950;337-404.
3. N. Dunkin, J. Shawe-Taylor, P. Koiran, A new incremental learning technique, in: Eighth Italian Workshop on Neural Nets, WIRN Vietri-96, Springer, Berlin, 1997, 112-118.
4. Girosi F. An equivalence between sparse approximation and support vector machines. Neural Comput. 10(6):1998;1455-1480.
5. Jutten C., Chentouf R. A new scheme for incremental learning. Neural Process. Lett. 2(1):1995;1-4.
6. Kadirkamanathan V., Niranjan M. A function estimation approach sequential learning with neural network. Neural Comput. 5(6):1993;954-975.
7. Kimeldorf G.S., Wahba G. A correspondence between bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Statist. 2:1971;495-502.
8. Luttrell S.P. The use of transinformation in the design of data sampling schemes for inverse problems. Inverse Problems. 1(1):1985;199-218.
9. Mackay D. Information-based objective functions for active data selection. Neural Comput. 4(4):1992;590-604.
10. V.A. Morozov, Methods for Solving Incorrectly Posed Problems, Springer, Berlin, 1984.
11. S. Nakazawa, A study on incremental learning in optimally generalizing neural networks, Master's Thesis, Tokyo Institute of Technology, 1997.
12. H. Ogawa, A unified approach to generalised sampling theorems, Proceedings, IEEE-IEICE-ASJ International Joint Conference on ASSP, IEEE-IEICE-ASJ, 1986, pp. 1657-1660.
13. H. Ogawa, Neural network theory as an inverse problem, J. IEICE Japan 73 (7) (1990) 690-695 (in Japanese).
14. H. Ogawa, Neural network learning, generalization and over learning, Proceedings, ICIIPS'92 (Beijing), vol. 2, 1992, pp. 1-6.
15. H. Ogawa, N. Nakamura, Projection filter restoration of degraded images, Proceedings, IEEE International Conference on Pattern Recognition (Montreal), IEEE, 1984, pp. 601-603.
16. Ogawa H., Oja E. Projection filter, Wiener filter and Karhunen-Loève subspaces in digital image restoration. J. Math. Anal. Appl. 114(1):1986;37-51.
17. E. Parzen, Regression analysis of continuous parameter time series, Proceedings, Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, 1961, pp. 469-489.
18. Poggio T., Girosi F. Networks for approximation and learning. Proc. IEEE. 78(9):1990;1481-1497.
19. S. Saitoh, Integral Transforms, Reproducing Kernels and their Applications, Pitman Research Notes in Mathematics No. 369. Longman Publishers, New York, 1997.
20. D. Schuurmans, A new metric based approach to model selection, Proceedings, Fourteenth National Conference on Artificial Intelligence, AAAI, 1997, 552-557.
21. A.N. Tikhonov, V.Y. Arsenin, Solution of Ill-Posed Problems, Winston, Washington, DC, 1977.
22. Unser M., Aldroubi A., Eden M. Fast B-spline transforms for continuous image representation and interpolation. IEEE Trans. Pattern Anal. Mach. Intell. 13(3):1991;277-285.
23. V. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, 1995.
24. V. Vapnik, S.E. Golowich, A. Smola, Support vector method for function approximation, regression estimation and signal processing, in: M.C. Mozer, M.I. Jordan, T. Petsche (Eds.), Advances in Neural Information Processing Systems, vol. 9, The MIT Press, Cambridge, MA, 1997, pp. 281-287.
25. S. Vijayakumar, H. Ogawa, A functional analytic approach to incremental learning in optimally generalizing neural networks, Proceedings, IEEE International Conference on Neural Networks ICNN'95 (Perth), Perth, Australia, 1995, IEEE, pp. 777-782.
26. S. Vijayakumar, M. Sugiyama, H. Ogawa, Training data selection for optimal generalization and noise variance reduction in neural networks, Proceedings, Tenth Italian Workshop on Neural Nets WIRN'98, Salerno, Italy, 1988, 153-166.
27. G. Wahba, Spline Models for Observational Data, vol. 59 of Series in Applied Mathematics, SIAM, Philadelphia, 1990.
28. C.K.I. Williams, Prediction with Gaussian processes: from linear regression to linear prediction and beyond, in: M.I. Jordan (Ed.), Learning and Inference in Graphical Models, Kluwer Academic Press, Dordrecht, 1998.
29. Yingwei L., Sundararajan N., Saratchandran P. A sequential learning scheme for function approximation using minimum radial basis function neural networks. Neural Comput. 9(9):1997;461-478.
30. B.T. Zhang, An incremental algorithm that optimizes network size and sample size in one trial, Proceedings of the International Conference on Neural Networks ICNN'94, Orlando, Florida, 1994, IEEE, pp. 215-220.