Model selection for regression with continuous kernel functions using the modulus of continuity

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 2607-2633

  Koo, Imhoi ^a   Kil, Rhee Man ^b  

^a Korea Institute of Oriental Medicine (KIOM)   (South Korea)

^b Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

Author keywords

Information criteria; Model selection; Modulus of continuity; Multilayer perceptrons; Regression models

Indexed keywords

CYBERNETICS; ELECTRIC LOADS; FORECASTING; MULTILAYERS; NEURAL NETWORKS; PATTERN RECOGNITION SYSTEMS; POLYNOMIAL APPROXIMATION; PROBABILITY DENSITY FUNCTION; RISK PERCEPTION;

INFORMATION CRITERIA; MODEL SELECTION; MODULUS OF CONTINUITY; MULTILAYER PERCEPTRONS; REGRESSION MODELS;

REGRESSION ANALYSIS;

EID: 57249114552     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (22)

1. H. Akaike. Information theory and an extension of the maximum likelihood principle. In Proceedings of the Second International Symposium on Information Theory, pages 267-281, 1973.
2. G. Anastassiou and S. Gal. Approximation Theory: Moduli of Continuity and Global Smoothness Preservation. Birkhäuser, Boston, 2000.
3. A. Barron, J. Rissanen, and B. Yu. The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory, 44:2743-2760, 1998.
4. C. Chang and C. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
5. O. Chapelle, V. Vapnik, and Y. Bengio. Model selection for small sample regression. Machine Learning, 48:315-333, 2002.
6. V. Cherkassky and Y. Ma. Comparison of model selection for regression. Neural Computation, 15:1691-1714, 2003.
7. V. Cherkassky, X. Shao, F. Mulier, and V. Vapnik. Model complexity control for regression using VC generalization bounds. IEEE Transactions on Neural Networks, 10:1075-1089, 1999.
8. S. Cohen and N. Intrator. On different model selection criteria in a forward and backward regression hybrid network. International Journal of Pattern Recognition and Artificial Intelligence, 18:847-865, 2004.
9. D. Donoho and I. Johnstone. Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90:1200-1224, 1995.
10. D. Foster and E. George. The risk inflation criterion for multiple regression. Annals of Statistics, 22:1947-1975, 1994.
11. T. Hastie, R. Tibshirani, and J. Friedman. Note on "comparison of model selection for regression" by V. Cherkassky and Y. Ma. Neural Computation, 15:1477-1480, 2003.
12. G. Hinton and D. van Camp. Keeping neural networks simple by minimizing the description length of the weights. In Proceedings of the Sixth Annual ACM Conference on Computational Learning Theory, pages 5-13, 1993.
13. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:13-30, 1963.
14. M. Karpinski and A. Macintyre. Polynomial bounds for VC dimension of sigmoidal neural networks. In Proceedings of the Twenty-Seventh Annual ACM Symposium on Theory of Computing, pages 200-208, 1995.
15. I. Koo and R. Kil. Nonlinear model selection based on the modulus of continuity. In Proceedings of World Congress on Computational Intelligence, pages 3552-3559, 2006.
16. G. Lorentz. Approximation of Functions. Chelsea Publishing Company, New York, 1986.
17. J. Rissanen. Stochastic complexity and modeling. Annals of Statistics, 14:1080-1100, 1986.
18. A. Sakurai. Polynomial bounds for the VC dimension of sigmoidal, radial basis function, and sigmapi networks. In Proceedings of the World Congress on Neural Networks, pages 58-63, 1995.
19. G. Schwartz. Estimating the dimension of a model. Annals of Statistics, 6:461-464, 1978.
20. A. Timan. Theory of Approximation of Functions of a Real Variable. English translation 1963, Pergaman Press, Russian original published in Moscow by Fizmatgiz in 1960.
21. V. Vapnik. Statistical Learning Theory. J. Wiley, 1998.
22. G. Wahba, G.Golub, and M. Heath. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21:215-223, 1979.