Algebraic analysis for nonidentifiable learning machines

Neural Computation

Volumn 13, Issue 4, 2001, Pages 899-933

  Watanabe, Sumio ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; BIOLOGICAL MODEL;

ALGORITHMS; BAYES THEOREM; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 0035316373     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976601300014402     Document Type: Article

References (40)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. on Automatic Control, 19, 716-723.
2. Akaike, H. (1980). Likelihood and Bayes procedure. In J. M. Bernald (Ed.), Bayesian statistics (pp. 143-166), Valencia, Spain: University Press.
3. Amari, S. (1985). Differential-geometrical methods in statistics. New York: Springer.
4. Amari, S. (1993). A universal theorem on learning curves. Neural Networks, 6, 161-166.
5. Amari, S., Fujita, N., & Shinomoto, S. (1992). Four types of learning curves. Neural Computation, 4(4), 608-618.
6. Amari, S., & Murata, N. (1993). Statistical theory of learning curves under entropic loss. Neural Computation, 5, 140-153.
7. Atiyah, M. F. (1970). Resolution of singularities and division of distributions. Comm. Pure and Appl. Math., 13, 145-150.
8. Barron, A. R. (1994). Approximation and estimation bounds for artificial neural networks. Machine Learning, 14(1), 115-133.
9. Bernstein, I. N. (1972). The analytic continuation of generalized functions with respect to a parameter. Functional Anal. Appl., 6, 26-40.
10. Björk, J. E. (1979). Rings of differential operators. Amsterdam: North-Holland.
11. Chui, C. K. (1992). An introduction to wavelets. Orlando, FL: Academic Press.
12. Cramer, H. (1949). Mathematical methods of statistics. Princeton, N.J.: Princeton University Press.
13. Dacunha-Castelle, D., & Gassiat, E. (1997). Testing in locally conic models, and application to mixture models. Probability and Statistics, 1, 285-317.
14. Fukumizu, K. (1996). A regularity condition of information matrix of a multilayer perceptron network. Neural Networks, 9, 871-879.
15. Fukumizu, K. (1999). Generalization error of linear neural networks in unidentifiable cases. In Lecture Notes on Computer Science, 1720, 51-62.
16. Gel'fand, I. M., & Shilov, G. E. (1964). Generalized functions. Orlando, FL: Academic Press.
17. Hagiwara, K., Kuno, K., & Usui, S. (2000). On the problem in model selection of neural network regression in overrealizable scenario. In Proc. of Int. Joint Conf. on Neural Netork, Italy, Como.
18. Hagiwara, K., Toda, N., & Usui, S. (1993). On the problem of applying AIC to determine the structure of a layered feed-forward neural network. Proc. of IJCNN Nagoya Japan, 3, 2263-2266.
19. Hironaka, H. (1964). Resolution of singularities of an algebraic variety over a field of characteristic zero. Annals of Mathematics, 79, 109-326.
20. Kashiwara, M. (1976). B-functions and holonomic systems. Inventions Math., 38, 33-53.
21. Knowles, M., & Siegmund, D. (1989). On Hotelling's approach to testing for a nonlinear parameter in regression. International Statistical Review, 57, 205-220.
22. Levin, E., Tishby, N., & Solla, S. A. (1990). A statistical approaches to learning and generalization in layered neural networks. Proc. of IEEE, 78(10), 1568-1674.
23. Mackay, D. J. (1992). Bayesian interpolation. Neural Computation, 4(2), 415-447.
24. Mhaskar, H. N. (1996). Neural networks for optimal approximation of smooth and analytic functions. Neural Computation, 8, 164-177.
25. Murata, N., Yoshizawa, S., & Amari, S. (1994). Network information criterion-determining the number of hidden units for an artificial neural network model. IEEE Transactions on Neural Networks, 5(6), 797-807.
26. Oaku, T. (1997). Algorithms for the b-function and D-modules associated with a polynomial. Journal of Pure and Applied Algebra, 117, 495-518.
27. Oka, K. (1962). Sur les fonctions analytiques de plusieurs variables. Tokyo: Iwanamishoten.
28. Opper, M., & Haussler, D. (1995). Bounds for predictive errors in the statistical mechanics of supervised learning. Physical Review letters, 75(20), 3772-3775.
29. Rissanen, J. (1986). Stochastic complexity and modeling. Annals of Statistics, 14, 1080-1100.
30. Sato, M., & Shintani, T. (1974). On zeta functions associated with prehomogeneous vector space. Annals of Mathematics, 100, 131-170.
31. Schwarz, G. (1974). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464.
32. Shibata, R. (1981). An optimal model selection of regression variables. Biometrika, 68, 45-54.
33. Watanabe, S. (1998a). Inequalities of generalization errors for layered neural networks in Bayesian learning. Proc. of Int. Conf. on Neural. Inform. Process, 1, 59-62.
34. Watanabe, S. (1998b). On the generalization error by a layered statistical model with Bayesian estimation. IEICE Trans., J81-A, 1442-1452. (English version is in Elect. and Comm. in Japan, (2000), 95-104.)
35. Watanabe, S. (1998b). On the generalization error by a layered statistical model with Bayesian estimation. IEICE Trans., J81-A, 1442-1452. (English version is in Elect. and Comm. in Japan, (2000), 95-104.)
36. Watanabe, S. (1999). Algebraic analysis for singular statistical estimation. Lecture Notes in Computer Science, 1720, 39-50.
37. Watanabe, S. (2000a). Algebraic analysis for non-regular learning machines. Advances in Neural Information Processing, 12, 356-362.
38. Watanabe, S. (2000b). Mathematical foundation for redundant statistical estimation. Proc. of 31st ISCIE Int. Symp. on Stochastic Systems Theory and Its Applications (pp. 119-124).
39. Watanabe, S., & Fukumizu, K. (1995). Probabilistic design of layered neural networks based on their unified framework. IEEE Trans. on Neural Networks, 6(3), 691-702.
40. Yamanishi, K. (1998). A decision-theoretic extension of stochastic complexity and its applications to learning. IEEE Trans. on Information Theory, 44(4), 1424-1439.