Adaptive model selection using empirical complexities

Annals of Statistics

Volumn 27, Issue 6, 1999, Pages 1830-1864

  Lugosi, Gábor ^a   Nobel, Andrew B ^b  

^a UNIVERSITAT POMPEU FABRA   (Spain)

^b University of North Carolina at Chapel Hill   (United States)

Author keywords

Classification; Complexity regularization; Curve fitting; Minimum description length; Pattern recognition; Regression estimation

Indexed keywords

EID: 0033234644     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1017939241     Document Type: Article

References (48)

1. AKAIKE, H. (1974). A new look at the statistical model identification. IEEE Trans. on Automat. Control 19 716-723.
2. BARRON, A. R. (1985). Logically smooth density estimation. Technical Report 56, Dept. Statistics, Stanford Univ.
3. BARRON, A. R. (1991). Complexity regularization with application to artificial neural networks. In Nonparametric Functional Estimation and Related Topics (G. Roussas, ed.) 561-576. Kluwer, Dordrecht.
4. BARRON, A., BIRGÉ, L. and MASSART, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413.
5. BARRON, A. R. and COVER, T. M. (1991). Minimum complexity density estimation. IEEE Trans. Inform. Theory 37 1034-1054.
6. BERNSTEIN, S. N. (1946). The Theory of Probabilities. Gastehizdat, Moscow.
7. BIRGÉ, L. and MASSART, P. (1997). From model selection to adaptive estimation. In Festschrift for Lucien Le Cam: Research papers in Probability and Statistics (D. Pollard, E. Torgersen and G. Yang, eds.) 55-87. Springer, New York.
8. BIRGÉ, L. and P. MASSART, (1998). Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli 4 329-375.
9. BREIMAN, L. FRIEDMAN, J. H., OLSHEN, R. A. and STONE, C. J. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
10. BUESCHER, K. L. and KUMAR, P. R. (1996a). Learning by canonical smooth estimation I: simultaneous estimation. IEEE Trans. on Automat. Control 41 545-556.
11. BUESCHER, K. L. and KUMAR, P. R. (1996b). Learning by canonical smooth estimation II: learning and choice of model complexity. IEEE Trans. Automat. Control 41 557-569.
12. CHAUDHURI, P. HUANG, M.-C., LOH, W.-Y. and YAO, R. (1994). Piecewise polynomial regression trees. Statist. Sinica 4 143-167.
13. DEVROYE, L. (1988). Automatic pattern recognition: a study of the probability of error. IEEE Trans. Pattern Analysis and Machine Intelligence 10 530-543.
14. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
15. DUDLEY, R. M. (1978). Central limit theorems for empirical measures. Ann. Probab. 6 899-929.
16. GALLANT, A. R. (1987). Nonlinear Statistical Models. Wiley, New York.
17. GEMAN, S. and HWANG, C.-R. (1982). Nonparametric maximum likelihood estimation by the method of sieves. Ann. Statist. 10 401-414.
18. GORDON, L. and OLSHEN, R. (1984). Almost sure consistent nonparametric regression from recursive partitioning schemes. J. Multivariate Anal. 15 147-163.
19. GRENANDER, U. (1981). Abstract Inference. Wiley, New York.
20. HAUSSLER, D. (1992). Decision theoretic generalizations of the PAC model for neural net and other learning applications. Inform. and Comput. 100 78-150.
21. HOEFFDING, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30.
22. KEARNS, M., MANSOUR, Y., NG, A. Y. and RON, D. (1995). An experimental and theoretical comparison of model selection methods. In Proceedings of the Eighth Annual ACM Workshop on Computational Learning Theory 21-30. ACM, New York.
23. KRZYZAK, A. and LINDER, T. (1998). Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Networks 9 247-256.
24. LUGOSI, G. and ZEGER, K. (1995). Nonparametric estimation via empirical risk minimization. IEEE Trans. Inform. Theory 41 677-678.
25. LUGOSI, G. and ZEGER, K. (1996). Concept learning using complexity regularization. IEEE Trans. Inform. Theory 42 48-54.
26. p. Technometrics 15 661-675.
27. MEIR, R. (1997). Performance bounds for nonlinear time series prediction. In Proceedings of the Tenth Annual ACM Workshop on Computational Learning Theory. ACM, New York.
28. MODHA, D. S. and MASRY, E. (1996). Minimum complexity regression estimation with weakly dependent observations. IEEE Trans. Inform. Theory 42 2133-2145.
29. NEMIROVSKII, A. S., POLYAK, B. T. and TSYBAKOV, A. B. (1985). Rate of convergence of nonparametric estimates of maximum-likelihood type. Problems Inform. Transmission 21 258-272.
30. NICOLERIS, T. and YATRACOS, Y. (1997). Rates of convergence of estimates, Kolmogorov's entropy and the dimensionality reduction principle in regression. Ann. Statist. 25 2493-2511.
31. NOBEL, A. B. (1996). Histogram regression estimation using data-dependent partitions. Ann. Statist. 24 1084-1105.
32. POLLARD, D. (1984). Convergence of Stochastic Processes. Springer, New York.
33. POLLARD, D. (1986). Rates of uniform almost sure convergence for empirical processes indexed by unbounded classes of functions. Unpublished manuscript.
34. POLLARD, D. (1989). Asymptotics via empirical processes. Statist. Sci. 4 341-366.
35. RISSANEN, J. (1983). A universal prior for integers and estimation by minimum description length. Ann. Statist. 11 416-431.
36. SCHWARZ, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461-464.
37. SHAWE-TAYLOR, J., BARTLETT, P. L., WILLIAMSON, R. C. and ANTHONY, M. (1988). Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inform. Theory 44 1926-1940.
38. SHEN, X. and WONG, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580-615.
39. STONE, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040-1053.
40. VAPNIK, V. N. (1982). Estimation of Dependencies Based on Empirical Data. Springer, New York.
41. VAPNIK, V. N. and CHERVONENKIS, A. YA. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 264-280.
42. VAPNIK, V. N. and CHERVONENKIS, A. YA. (1974). Theory of Pattern Recognition. Nauka, Moscow. (in Russian). [German translation, Theorie der Zeichenerkennung. Akademie Berlin (1979).]
43. VAPNIK, V. N. and CHERVONENKIS, A. YA. (1974). Theory of Pattern Recognition. Nauka, Moscow. (in Russian). [German translation, Theorie der Zeichenerkennung. Akademie Berlin (1979).]
44. WALLACE, C. S. and FREEMAN, P. R. (1987). Estimation and inference by compact coding. J. Roy. Statist. Soc. Ser. B 49 240-265.
45. WHITE, H. (1990). Connectionist nonparametric regression: multilayer feedforward networks can learn arbitrary mappings. Neural Networks 3 535-549.
46. WONG, W. H. and SHEN, X. (1992). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Technical Report 346, Dept. Statistics, Univ. Chicago.
47. YANG, Y. and BARRON, A. R. (1998). An asymptotic property of model selection criteria. IEEE Trans. Inform. Theory 44 95-116.
48. YANG, Y. and BARRON, A. R. (1999). Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 1564-1599.