Generalization error of linear neural networks in an empirical bayes approach

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2005, Pages 804-810

  Nakajima, Shinichi ^a,b   Watanabe, Sumio ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b NIKON CORPORATION   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

BAYES ESTIMATION; COMPUTATIONAL COSTS; EMPIRICAL BAYES APPROACH; GENERALIZATION ERROR; GENERALIZATION PERFORMANCE; LINEAR NEURAL NETWORK; POSTERIOR DISTRIBUTIONS; SHRINKAGE ESTIMATIONS;

ESTIMATION; MAXIMUM LIKELIHOOD ESTIMATION;

NEURAL NETWORKS;

EID: 33645770499     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. H. Akaike. A new look at statistical model. IEEE Trans. on Automatic Control, 19:716-723, 1974.
2. H. Akaike. Likelihood and bayes procedure. In J. M. Bernald, Bayesian statistics, pages 143-166, Valencia, Italy, 1980. University Press.
3. M. Aoyagi and S. Watanabe. The generalization error of reduced rank regression in bayesian estimation. In Proc. of ISITA, pages 1068-1073, Parma, Italy, 2004.
4. H. Attias. Inferring parameters and structure of latent variable models by variational bayes. In Proc. of UAI, 1999.
5. P. F. Baldi and K. Hornik. Learning in linear neural networks: a survey. IEEE Trans. on Neural Networks, 6:837-858, 1995.
6. B. Efron and C. Morris. Stein's estimation rule and its competitors-an empirical bayes approach. J. of Am. Stat. Assoc., 68:117-130, 1973.
7. K. Fukumizu. Generalization error of linear neural networks in unidentifiable cases. In Proc. of ALT, pages 51-62. Springer, 1999.
8. Z. Ghahramani and M. J. Beal. Graphical models and variational methods. In Advanced Mean Field Methods. MIT Press, 2000.
9. G. E. Hinton and D. van Camp. Keeping neural networks simple by minimizing the description length of the weights. In Proc. of COLT, 1993.
10. W. James and C. Stein. Estimation with quadratic loss. In Proc. of the 4th Berkeley Symp. on Math. Stat. and Prob., pages 361-379, Berkeley:University of California, 1961.
11. R. E. Kass and D. Steffey. Approximate bayesian inference in conditionally independent hierarchical models (parametric empirical bayes models). J. of the Am. Stat. Assoc., 84:717-726, 1989.
12. D. J. C. MacKay. Developments in probabilistic modeling with neural networks-ensemble learning. In Proc. of the 3rd Ann. Symp. on Neural Networks, pages 191-198, 1995.
13. S. Nakajima and S. Watanabe. Generalization performance of subspace bayes approach in linear neural networks. Submitted to IEICE Trans., 2005.
14. S. Nakajima and S. Watanabe. Variational bayes solution of linear neural networks and its generalization error and training error. Submitted to ICML, 2005.
15. G. C. Reinsel and R. P. Velu. Multivariate Reduced-Rank Regression. Springer, 1998.
16. S. Watanabe and S. Amari. Learning coefficients of layered models when the true distribution mismatches the singularities. Neural Computation, 15:1013-1033, 2003. (Pubitemid 37049800)
17. S. Watanabe. Algebraic information geometry for learning machines with singularities. Advances in NIPS, 13:329-336, 2001.
18. K.W.Watcher. The strong limits of random matrix spectra for sample matrices of independent elements. Ann. Prob., 6:1-18, 1978.
19. K. Yamazaki and S. Watanabe. Singularities in mixture models and upper bounds of stochastic complexity. Neural Networks, 16(7):1029-1038, 2003. (Pubitemid 38147281)