Likelihood ratio of unidentifiable models and multilayer neural networks

Annals of Statistics

Volumn 31, Issue 3, 2003, Pages 833-851

  Fukumizu, Kenji ^a  

^a INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

Author keywords

Likelihood ratio; Locally conic model; Multilayer neural networks; Unidentifiable model

Indexed keywords

EID: 0042311289     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1056562464     Document Type: Article

References (19)

1. BICKEL, P. and CHERNOFF, H. (1993). Asymptotic distribution of the likelihood ratio statistic in a prototypical nonregular problem. In Statistics and Probability: A Raghu Raj Bahadur Festschrift (J. K. Ghosh, S. K. Mitra, K. R. Parthasarathy and B. P. Rao, eds.) 83-96. Wiley Eastern, New Delhi.
2. CRAMÉR, H. (1946). Mathematical Methods of Statistics. Princeton Univ. Press.
3. CSÖRGO, M. and HORVATH, L. (1997). Limit Theorems in Change-Point Analysis. Wiley, New York.
4. DACUNHA-CASTELLE, D. and GASSIAT, E. (1997). Testing in locally conic models and application to mixture models. ESAIM Probab. Statist. 1 285-317.
5. DACUNHA-CASTELLE, D. and GASSIAT, E. (1999). Testing the order of a model using locally conic parametrization: Population mixtures and stationary ARMA processes. Ann. Statist. 27 1178-1209.
6. FUKUMIZU, K. (1996). A regularity condition of the information matrix of a multilayer perception network. Neural Networks 9 871-879.
7. FUKUMIZU, K. (1999). Generalization error of linear neural networks in unidentifiable cases. In Algorithmic Learning Theory. Lecture Notes in Artificial Intelligence (O. Watanabe and T. Yokomori, eds.) 1720 51-62. Springer, Berlin.
8. FUKUMIZU, K. and AMARI, S. (2000). Local minima and plateaus in hierarchical structures of multilayer perceptions. Neural Networks 13 317-327.
9. GASSIAT, E. and KÉRIBIN, C. (2000). The likelihood ratio test for the number of components in a mixture with Markov regime. ESAIM Probab. Statist. 4 25-52.
10. HAGIWARA, K., KUNO, K. and USUI, S. (2000). On the problem in model selection of neural network regression in overrealizable scenario. In Proc. IEEE-INNS-ENNS International Joint Conference on Neural Networks 6 461-466. IEEE Comput. Soc., Los Alamitos, CA.
11. HARTIGAN, J. A. (1985). A failure of likelihood asymptotics for normal mixtures. In Proc. Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (L. M. Le Cam and R. A. Olshen, eds.) 2 807-810. Wadsworth, Belmont, CA.
12. HAYASAKA, T., TODA, N., USUI, S. and HAGIWARA, K. (1996). On the least square error and prediction square error of function representation with discrete variable basis. In Proc. Conference on Neural Networks for Signal Processing 6 72-81. IEEE, New York.
13. LE CAM, L. (1970). On the assumptions used to prove asymptotic normality of maximum likelihood estimates. Ann. Math. Statist. 41 802-828.
14. LIU, X. and SHAO, Y. (2001). Asymptotic distribution of the likelihood ratio test in a two-component normal mixture model. Technical report, Dept. Statistics, Columbia Univ.
15. RUMELHART, D. E., HINTON, G. E. and WILLIAMS, R. J. (1986). Learning internal representations by error propagation. In Parallel Distributed Processing (D. E. Rumelhart, J. L. McClelland and the PDP Research Group, eds.) 1 318-362. MIT Press, Cambridge, MA.
16. SAZONOV, V. V. (1968). On the multi-dimensional central limit theorem. Sankhyā Ser. A 30 181-204.
17. SUSSMANN, H. J. (1992). Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks 5 589-593.
18. VERES, S. (1987). Asymptotic distributions of likelihood ratios for overparameterized ARMA processes. J. Time Ser. Anal. 8 345-357.
19. WALD, A. (1949). Note on the consistency of the maximum likelihood estimate. Ann. Math. Statist. 20 595-601.