Bayesian information criteria and smoothing parameter selection in radial basis function networks

Biometrika

Volumn 91, Issue 1, 2004, Pages 27-43

  Konishi, Sadanori ^a   Ando, Tomohiro ^a   Imoto, Seiya ^b  

^a KYUSHU UNIVERSITY   (Japan)

^b UNIVERSITY OF TOKYO   (Japan)

Author keywords

Bayes approach; Maximum penalised likelihood; Model selection; Neural network; Nonlinear regression

Indexed keywords

EID: 3242882795     PISSN: 00063444     EISSN: None     Source Type: Journal    
DOI: 10.1093/biomet/91.1.27     Document Type: Article

References (47)

1. AKAIKE, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd Int. Symp. on Information Theory, Ed. B. N. Petrov and F. Csaki, pp. 267-81. Budapest: Akademiai Kiado.
2. ANDO, T., IMOTO, S. & KONISHI, S. (2001). Estimating nonlinear regression models based on radial basis function networks (in Japanese). Jap. J. Appl. Statist. 30, 19-35.
3. ANDRIEU, C., DE FREITAS, N. & DOUCET, A. (2001). Robust full Bayesian learning for radial basis networks. Neural Comp. 13, 2359-407.
4. BISHOP, C. M. (1995). Neural Networks for Pattern Recognition. Oxford University Press.
5. BROOMHEAD, D. S. & LOWE, D. (1988). Multivariable functional interpolation and adaptive networks. Complex Syst. 2, 321-35.
6. CLARKE, B. S. & BARRON, A. R. (1994). Jeffreys' proir is asymptotically least favorable under entropy risk. J. Statist. Plan. Infer. 41, 37-60.
7. DAVISON, A. C. (1986). Approximate predictive likelihood. Biometrika 73, 323-32.
8. EFRON, B. (1986). How biased is the apparent error rate of a prediction rule? J. Am. Statist. Assoc. 81, 461-70.
9. GOOD, I. J. & GASKINS, R. A. (1971). Nonparametric roughness penalites for probability densities. Biometrika 58, 255-77.
10. GREEN, P. J. & SILVERMAN, B. W. (1994). Nonparametric Regression and Generalized Linear Models. London: Chapman and Hall.
11. GREEN, P. J. & YANDELL, B. (1985). Semi-parametric generalized linear models. In Generalized Linear Models, Ed. R. Gilchrist, B. J. Francis and J. Whittaker, Lecture Notes in Statistics, 32, pp. 44-55. Berlin: Springer.
12. HASTIE, T. J. & TIBSHIRANI, R. J. (1990). Generalized Additive Models. London: Chapman and Hall.
13. HOLMES, C. C. & MALLICK, B. K. (1998). Bayesian radial basis functions of variable dimension. Neural Comp. 10, 1217-33.
14. HURVICH, C. M. & TSAI, C.-L. (1989). Regression and time series model selection in small samples. Biometrika 76, 297-307.
15. HURVICH, C. M., SIMONOFF, J. S. & TSAI, C.-L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Statist. Soc. B 60, 271-93.
16. KASS, R. E. & RAFTERY, A. E. (1995). Bayes factors. J. Am. Statist. Assoc. 90, 773-95.
17. KASS, R. E. & WASSERMAN, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Am. Statist. Assoc. 90, 928-34.
18. KASS, R. E., TIERNEY, L. & KADANE, J. B. (1990). The validity of posterior expansions based on Laplace's method. In Essays in Honor of George Barnard, Ed. S. Geisser, J. S. Hodges, S. J. Press and A. Zellner, pp. 473-88. Amsterdam: North-Holland.
19. KONISHI, S. (1999). Statistical model evaluation and information criteria. In Multivariate Analysis, Design of Experiments and Surrey Sampling, Ed. S. Ghosh, pp. 369-99. New York: Marcel Dekker.
20. KONISHI, S. & KITAGAWA, G. (1996). Generalised information criteria in model selection. Biometrika 83, 875-90.
21. KULLBACK, S. & LEIBLER, R. A. (1951). On information and sufficiency. Ann. Math. Statist. 22, 79-86.
22. LANTERMAN, A. D. (2001). Schwarz, Wallace, and Rissanen: Intertwining themes in theories of model selection. Int. Statist. Rev. 69, 185-212.
23. MACKAY, D. J. C. (1992). A practical Bayesian framework for backpropagation networks. Neural Comp. 4, 448-72.
24. MCCULLAGH, P. & NELDER, J. A. (1989). Generalized Linear Models, 2nd ed. London: Chapman and Hall.
25. MOODY, J. (1992). The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems. In Advances in Neural Information Processing Systems 4, Ed. J. E. Moody, S. J. Hanson and R. P. Lippmann, pp. 847-54. San Mateo, CA: Morgan Kaufmann.
26. MOODY, J. & DARKEN, C. J. (1989). Fast learning in networks of locally-tuned processing units. Neural Comp. 1, 281-94.
27. MURATA, N., YOSHIZAWA, S. & AMARI, S. (1994). Network information criterion-determining the number of hidden units for an artificial neural network model. IEEE Trans. Neural Networks 5, 865-72.
28. NEAL, R. M. (1996). Bayesian Learning for Neural Networks, Lecture Notes in Statistics 118. New York: Springer-Verlag.
29. NEATH, A. A. & CAVANAUGH, J. E. (1997). Regression and time series model selection using variants of the Schwarz information criterion. Commun. Statist. A 26, 559-80.
30. NELDER, J. A. & WEDDERBURN, R. W. M. (1972). Generalized linear models. J. R. Statist. Soc. A 135, 370-84.
31. O'HAGAN, A. (1995). Fractional Bayes factors for model comparison (with Discussion). J. R. Statist. Soc. B 57, 99-138.
32. O'SULLIVAN, F., YANDELL, B. S. & RAYNOR, W. J. (1986). Automatic smoothing of regression functions in generalized linear models. J. Am. Statist. Assoc. 81, 96-103.
33. PAULER, D. (1998). The Schwarz criterion and related methods for normal linear models. Biometrika 85, 13-27.
34. POGGIO, T. & GIROSI, F. (1990). Networks for approximation and learning. Proc. IEEE 78, 1484-7.
35. RIOS INSUA, D. & MÜLLER, P. (1998). Feedforward neural networks for nonparametric regression. In Practical Nonparametric and Semiparametric Bayesian Statistics, Ed. D. K. Dey, P. Müller and D. Sinha, pp. 181-91. New York: Springer Verlag.
36. RIPLEY, B. D. (1996). Pattern Recognition and Neural Networks. Cambridge University Press.
37. SCHWARZ, G. (1978). Estimating the dimension of a model. Ann. Statist. 6, 461-4.
38. SILVERMAN, B. W. (1985). Some aspects of the spline smoothing approach to nonparametric regression curve fitting (with Discussion). J. R. Statist. Soc. B 47, 1-52.
39. SPIEGELHALTER, D. J., BEST, N. G., CARLIN, B. P. & VAN DER LINDE, A. (2002). Bayesian measures of model complexity and fit (with Discussion). J. R. Statist. Soc. E 64, 583-639.
40. SUGIURA, N. (1978). Further analysis of the data by Akaike's information criterion and the finite corrections. Commun. Statist. A 7, 13-26.
41. TAKEUCHI, K. (1976). Distributions of information statistics and criteria for adequacy of models (in Japanese). Math. Sci. 153, 12-8.
42. TANABE, K. & TANAKA, T. (1983). Estimation of curve and surface by Bayesian model (in Japanese). Chikyu 5, 179-86.
43. TIERNEY, L. & KADANE, J. B. (1986). Accurate approximations for posterior moments and marginal densities. J. Am. Statist. Assoc. 81, 82-6.
44. TIERNEY, L., KASS, R. E. & KADANE, J. B. (1989). Fully exponential Laplace approximations to expectations and variances of nonpositive functions. J. Am. Statist. Assoc. 84, 710-6.
45. WAHBA, G. (1990). Spline Models for Observational Data. Philadelphia: Society for Industrial and Applied Mathematics.
46. WEBB, A. (1999). Statistial Pattern Recognition. London: Arnold.
47. WHITTAKER, E. (1923). On a new method of graduation. Proc. Edin. Math. Soc. 41, 63-75.