Incremental active learning for optimal generalization

Neural Computation

Volumn 12, Issue 12, 2000, Pages 2909-2940

  Sugiyama, Masashi ^a   Ogawa, Hidemitsu ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; BIOLOGICAL MODEL; COMPUTER SIMULATION; LEARNING; ROBOTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; GENERALIZATION (PSYCHOLOGY); MODELS, NEUROLOGICAL; ROBOTICS;

EID: 0034567857     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300014773     Document Type: Article

References (38)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC-19(6), 716-723.
2. Albert, A. (1972). Regression and the Moore-Penrose pseudoinverse. New York: Academic Press.
3. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68, 337-404.
4. Ben-Israel, A., & Greville, T. N. E. (1974). Generalized inverses: Theory and applications. New York: Wiley.
5. Bergman, S. (1970). The kernel function and conformal mapping. Providence, RI: American Mathematical Society.
6. Cohn, D. A. (1994). Neural network exploration using optimal experiment design. In J. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 679-686). San Mateo, CA: Morgan Kaufmann.
7. Cohn, D. A. (1996). Neural network exploration using optimal experiment design. Neural Networks, 9(6), 1071-1083.
8. Cohn, D. A. (1997). Minimizing statistical bias with queries. In M. C. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 417-423). Cambridge, MA: MIT Press.
9. Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4, 129-145.
10. Efron, B., & Tibshirani, B. (1993). An introduction to bootstrap. New York: Chapman and Hall.
11. Fedorov, V. V. (1972). Theory of optimal experiments. New York: Academic Press.
12. Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 295-301). Cambridge, MA: MIT Press.
13. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1), 1-58.
14. Kaelbling, L. P. (Ed.). (1996). Special issue on reinforcement learning. Machine Learning, 22(1/2/3).
15. Kiefer, J. (1959). Optimum experimental designs. Journal of the Royal Statistics Society, Series B, 21, 272-304.
16. Kiefer, J., & Wolfowitz, J. (1960). The equivalence of two extremum problems. Annals of Mathematical Statistics, 32, 298.
17. MacKay, D. J. C. (1992). Information-based objective functions for active data selection. Neural Computation, 4(4), 590-604.
18. Ogawa, H. (1987). Projection filter regularization of ill-conditioned problem. Proceedings of SPIE, Inverse Problems in Optics, 808, 189-196.
19. Ogawa, H. (1989). Inverse problem and neural networks. In Proceedings of IEICE 2nd Karuizawa Workshop on Circuits and Systems (pp. 262-268). Karuizawa, Japan (in Japanese).
20. Ogawa, H. (1992). Neural network learning, generalization and over-learning. Proceedings of the ICIIPS'92, International Conference on Intelligent Information Processing and System, 2 (pp. 1-6). Beijing, China.
21. Saitoh, S. (1988). Theory of reproducing kernels and its applications. London: Longman Scientific & Technical.
22. Saitoh, S. (1997). Integral transform, reproducing kernels and their applications. London: Longman.
23. Schatten, R. (1970). Norm ideals of completely continuous operators. Berlin: Springer-Verlag.
24. Sollich, P. (1994). Query construction, entropy and generalization in neural network models. Physical Review E, 49, 4637-4651.
25. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, Series B, 36, 111-133.
26. Sugiyama, M., & Ogawa, H. (1999a). Exact incremental projection learning in the presence of noise. In Proceedings of the 11th Scandinavian Conference on Image Analysis (pp. 747-754). Kangerlussuaq, Greenland.
27. Sugiyama, M., & Ogawa, H. (1999b). Incremental projection learning for optimal generalization (Tech. Rep. No. TR99-0007). Tokyo: Department of Computer Science, Tokyo Institute of Technology, Japan. Available online at: ftp://ftp.cs.titech.ac.jp/pub/TR/99/TR99-0007.ps.gz.
28. Sugiyama, M., & Ogawa, H. (1999c). Properties of incremental projection learning (Tech. Rep. No. TR99-0008). Tokyo: Department of Computer Science, Tokyo Institute of Technology, Japan. Available online at: ftp://ftp.cs.titech.ac.jp/pub/TR/99/TR99-0008.ps.gz.
29. Sugiyama, M., & Ogawa, H. (1999d). Functional analytic approach to model selection - subspace information criterion. In Proceedings of 1999 Workshop on Information-Based Induction Sciences (IBIS'99) (pp. 93-98). Izu, Japan. Available online at: ftp://ftp.cs.titech.ac.jp/pub/TR/99/TR99-0009.ps.gz.
30. Sugiyama, M., & Ogawa, H. (2000). Training data selection for optimal generalization in trigonometric polynomial networks. In S. A. Solla, T. K. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems, 12 (pp. 624-630). Cambridge, MA: MIT Press.
31. Takemura, A. (1991). Modern mathematical statistics. Tokyo: Sobunsya (in Japanese).
32. Tikhonov, A. N., & Arsenin, V. Y. (1977). Solutions of ill-posed problems. Washington, DC: V. H. Winston.
33. Vijayakumar, S. (1998). Computational theory of incremental and active learning for optimal generalization. Unpublished doctoral dissertation, Tokyo Institute of Technology, Japan.
34. Vijayakumar, S., & Ogawa, H. (1999). Improving generalization ability through active learning. IEICE Transactions on Information and Systems, E82-D(2), 480-487.
35. Vijayakumar, S., Sugiyama, M., & Ogawa, H. (1998). Training data selection for optimal generalization with noise variance reduction in neural networks. In M. Marinaro & R. Tagliaferri (Eds.), Neural nets WIRN Vietri-98 (pp. 153-166). Berlin: Springer-Verlag.
36. Wahba, H. (1990). Spline model for observational data. Philadelphia: Society for Industrial and Applied Mathematics.
37. Yamashita, Y., & Ogawa, H. (1992). Optimum image restoration and topological invariance. The transactions of the IEICE D-II, J75-D-II(2), 306-313. (in Japanese)
38. Yue, R. X., & Hickernell, F. J. (1999). Robust designs for fitting linear models with misspecification. Statistica Sinica, 9, 1053-1069.