Subspace information criterion for model selection

Neural Computation

Volumn 13, Issue 8, 2001, Pages 1863-1889

  Sugiyama, Masashi ^a   Ogawa, Hidemitsu ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPARATIVE STUDY; LEARNING; MATHEMATICS; PSYCHOLOGICAL MODEL; REGRESSION ANALYSIS; STATISTICAL MODEL;

LEARNING; LEAST-SQUARES ANALYSIS; MATHEMATICS; MODELS, NEUROLOGICAL; MODELS, PSYCHOLOGICAL; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER);

EID: 0035434818     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/08997660152469387     Document Type: Article

References (74)

1. Abramowitz, M., & Segun, I. A. (Eds.). (1964). Handbook of mathematical functions with formulas, graphs, and mathematical tables. New York: Dover.
2. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC-19(6), 716-723.
3. Akaike, H. (1980). Likelihood and the Bayes procedure. In N. J. Bernardo, M. H. DeGroot, D. V. Lindley, & A. F. M. Smith (Eds.), Bayesian statistics (pp. 141-166). Valencia: University Press.
4. Akaike, H., & Kitagawa, G. (Eds.). (1994). The practice of time series analysis I. Tokyo: Asakura Syoten. (In Japanese)
5. Akaike, H., & Kitagawa, G. (Eds.). (1995). The practice of time series analysis II. Tokyo: Asakura Syoten. (In Japanese)
6. Albert, A. (1972). Regression and the Moore-Penrose pseudoinverse. New York: Academic Press.
7. Allen, D. (1974). The relationship between variable selection and data augmentation and a method for prediction. Technometrics, 16, 125-127.
8. Amari, S. (1967). Theory of adaptive pattern classifiers. IEEE Transactions on Electronic Computers, EC-16(3), 299-307.
9. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68, 337-404.
10. Ben-Israel, A., & Greville, T. N. E. (1974). Generalized inverses: Theory and applications. New York: Wiley.
11. Bergman, S. (1970). The kernel function and conformal mapping. Providence, RRI: American Mathematical Society.
12. Bozdogan, H. (Ed.). (1994). Proceedings of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach. Norwell, MA: Kluwer.
13. Cavanaugh, J. E., & Shumway, R. H. (1997). A bootstrap variant of AIC for state space model selection. Statistica Sinica, 7, 473-496.
14. Cherkassky, V., Shao, X., Mulier, F. M., & Vapnik, V. N. (1999). Model complexity control for regression using VC generalization bounds. IEEE Transactions on Neural Networks, 10(5), 1075-1089.
15. Craven, P., & Wahba, G. (1979). Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik, 31, 377-403.
16. Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their application. Cambridge: Cambridge University Press.
17. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7(1), 1-26.
18. Efron, B. (1986). How biased is the apparent error rate of a prediction rule? Journal of the American Statistical Association, 81(394), 461-470.
19. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
20. Fedorov, V. V. (1972). Theory of optimal experiments. New York: Academic Press.
21. Freud, G. (1966). Orthogonal polynomials. Oxford: Pergamon Press.
22. Fujikoshi, Y., & Satoh, K. (1997). Modified AIC and CP in multivariate linear regression. Biometrika, 84, 707-716.
23. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1), 1-58.
24. Gorman, J. W., & Toman, R. J. (1966). Selection of variables for fitting equations to data. Technometrics, 8(1), 27-51.
25. Hurvich, C. M., Simonoff, J. S., & Tsai, C. L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society, Series B, 60, 271-293.
26. Hurvich, C. M., & Tsai, C. L. (1989). Regression and time series model selection in small samples. Biometrika, 76, 297-307.
27. Hurvich, C. M., & Tsai, C. L. (1991). Bias of the corrected AIC criterion for underfitted regression and time series models. Biometrika, 78, 499-509.
28. Hurvich, C. M., & Tsai, C. L. (1993). A corrected Akaike information criterion for vector autoregressive model selection. Journal of Time Series Analysis, 14, 271-279.
29. Ishiguro, M., Sakamoto, Y., & Kitagawa, G. (1997). Bootstrapping log likelihood and EIC, an extension of AIC. Annals of the Institute of Statistical Mathematics, 49, 411-434.
30. Kitagawa, G., & Gersch, W. (1996). Smoothness priors analysis of time series. New York: Springer-Verlag.
31. Konishi, S., & Kitagawa, G. (1996). Generalized information criterion in model selection. Biometrika, 83, 875-890.
32. Li, K. (1986). Asymptotic optimality of CL and generalized cross-validation in ridge regression with application to spline smoothing. Annals of Statistics, 14(3), 1101-1112.
33. MacKay, D. (1992). Bayesian interpolation. Neural Computation, 4(3), 415-447.
34. Mallows, C. L. (1964). Choosing variables in a linear regression: A graphical aid. Paper presented at the Central Regional Meeting of the Institute of Mathematical Statistics, Manhattan, KS.
35. p. Technometrics, 15(4), 661-675.
36. McQuarrie, A. D. R., & Tsai, C. L. (1998). Regression and time series model selection. Singapore: World Scientific.
37. Mosteller, F., & Wallace, D. (1963). Inference in an authorship problem. A comparative study of discrimination methods applied to the authorship of the disputed Federalist Papers. Journal of the American Statistical Association, 58, 275-309.
38. Murata, N., Yoshizawa, S., & Amari, S. (1994). Network information criterion -Determining the number of hidden units for an artificial neural network model. IEEE Transactions on Neural Networks, 5(6), 865-872.
39. Nakashima, A., & Ogawa, H. (1999). How to design a regularization term for improving generalization. In Proceedings of ICONIP'99, the 6th International Conference on Neural Information Processing, 1 (pp. 222-227). Perth, Australia.
40. Noda, K., Miyaoka, E., & Itoh, M. (1996). On bias correction of the Akaike information criterion in linear models. Communications in Statistics: Theory and Methods, 25, 1845-1857.
41. Ogawa, H. (1987). Projection filter regularization of ill-conditioned problem. In Proceedings of SPIE, Inverse Problems in Optics, 808 (pp. 189-196).
42. Ogawa, H. (1992). Neural network learning, generalization and over-learning. In Proceedings of the ICIIPS'92, International Conference on Intelligent Information Processing and System, 2 (pp. 1-6). Beijing, China.
43. Oja, E., & Ogawa, H. (1986). Parametric projection filter for image and signal restoration. IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-34(6), 1643-1653.
44. Onoda, T. (1995). Neural network information criterion for the optimal number of hidden units. In Proceedings of IEEE International Conference on Neural Networks, ICNN'95 (pp. 275-280). Perth, Australia.
45. Platt, J. (1991). A resource-allocating network for function interpolation. Neural Computation, 3(2), 213-225.
46. Poggio, T., & Girosi, F. (1990). Networks for approximation and learning. Proceedings of the IEEE, 78(9), 1481-1497.
47. Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14, 465-471.
48. Rissanen, J. (1987). Stochastic complexity. Journal of the Royal Statistical Society, Series B, 49(3), 223-239.
49. Rissanen, J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, IT-42(1), 40-47.
50. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986a). Learning representations by back-propagating errors. Nature, 323, 533-536.
51. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986b). Learning internal representations by error propagation. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, 1 (pp. 318-362). Cambridge, MA: MIT Press.
52. Saitoh, S. (1988). Theory of reproducing kernels and its applications. London: Longman Scientific & Technical.
53. Saitoh, S. (1997). Integral transform, reproducing kernels and their applications. London: Longman.
54. Sato, K., Kobayashi, M., & Fujikoshi, Y. (1997). Variable selection for the growth curve model. Journal of Multivariate Analysis, 60, 277-292.
55. Savage, L. J. (1954). The foundation of statistics. New York: Wiley.
56. Schatten, R. (1970). Norm ideals of completely continuous operators. New York: Springer-Verlag.
57. Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1998). Advances in kernel methods: Support vector machines. Cambridge, MA: MIT Press.
58. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
59. Shibata, R. (1989). Statistical aspects of model selection. In J. C. Willems (Ed.), From data to model (pp. 375-394). New York: Springer-Verlag.
60. Shibata, R. (1997). Bootstrap estimate of Kullback-Leibler information for model selection. Statistica Sinica, 7, 375-394.
61. Simonoff, J. S. (1998). Three sides of smoothing: Categorical data smoothing, nonparametric regression, and density estimation. International Statistical Review, 66, 137-156.
62. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, Series B, 36, 111-147.
63. Stone, M. (1977). An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. Journal of the Royal Statistical Society, Series B, 39, 44-47.
64. Sugiura, N. (1978). Further analysis of the data by Akaike's information criterion and the finite corrections. Communications in statistics. Theory and Methods, 7(1), 13-26.
65. Szegö, G. (1939). Orthogonal polynomials. Providence, RI: American Mathematical Society.
66. Takemura, A. (1991). Modern mathematical statistics. Tokyo: Sobunsya. (In Japanese)
67. Takeuchi, K. (1976). Distribution of information statistics and validity criteria of models. Mathematical Science, 153, 12-18. (In Japanese)
68. Takeuchi, K. (1983). On the selection of statistical models by AIC. Journal of the Society of Instrument and Control Engineering, 22(5), 445-453. (In Japanese)
69. Tikhonov, A. N., & Arsenin, V. Y. (1977). Solutions of ill-posed problems. Washington, DC: V. H. Winston.
70. Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer-Verlag.
71. Wada, Y., & Kawato, M. (1991). Estimation of generalization capability by combination of new information criterion and cross validation. Transactions of the IEICE, J74-D-II(7), 955-965. (In Japanese)
72. Wahba, H. (1990). Spline model for observational data. Philadelphia: Society for Industrial and Applied Mathematics.
73. Wong, W. (1983). A note on the modified likelihood for density estimation. Journal of the American Statistical Association, 78(382), 461-463.
74. Yamanishi, K. (1998). A decision-theoretic extension of stochastic complexity and its application to learning. IEEE Transactions on Information Theory, IT-44, 1424-1439.