Adaptive regression by mixing

Journal of the American Statistical Association

Volumn 96, Issue 454, 2001, Pages 574-588

  Yang, Yuhong ^a  

^a Department of Statistics Center for Statistics and Applications in Forensic Evidence Iowa State University 2438 Osborn Dr Ames   (United States)

Author keywords

Adaptive estimation; Combining procedures; Nonparametric regression

Indexed keywords

EID: 1542573460     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1198/016214501753168262     Document Type: Article

References (45)

1. Akaike, H. (1973), “Information Theory and an Extension of the Maximum Likelihood Principle,” in Proceedings of the 2nd International Symposium on Information Theory, eds. B. N. Petrov and F. Csaki, Budapest: Akademia Kiado, pp. 267–281.
2. Barron, A. R. (1987), “Are Bayes Rules Consistent in Information?,” in Open Problems in Communication and Computation, eds. T. M. Cover and B. Gopinath, Berlin: Springer-Verlag, pp. 85–91.
3. Barron, A. R., Birgé, L., and Massart, P. (1999),“Risk Bounds for Model Selection via Penalization,” Probability Theory and Related Fields, 113, 301–413.
4. Barron, A. R., and Cover, T. M. (1991), “Minimum Complexity Density Estimation,” IEEE Transactions on Information Theory, 37, 1034–1054.
5. Barron, A. R., Rissanen, J., and Yu, B. (1998), “The Minimum Description Length Principle in Coding and Modeling,” IEEE Transactions on Information Theory, 44, 2743–2760.
6. Breiman, L. (1996a), “Stacked Regressions,” Machine Learning, 24, 49–64.
7. Breiman, L. (1996b), “Bagging predictors,” Machine Learning, 24, 123–140.
8. Buckland, S. T., Burnham, K. P., and Augustin, N. H. (1995), “Model Selection: An Integral Part of Inference,” Biometrics, 53, 603–618.
9. Catoni, O. (1997), “The Mixture Approach to Universal Model Selection,” Technical Report LIENS-97-22, Ecole Normale Superieure, Paris, France.
10. Cesa-Bianchi, N., and Lugosi, G. (1999), “On Prediction of Individual Sequences,” The Annals of Statistics, 27, 1865–1895.
11. Cesa-Bianchi, N., Freund, Y., Haussler, D. P., Schapire, R., and Warmuth, M. K. (1997), “How to Use Expert Advice?,” Journal of the Association for Computing Machinery, 44, 427–485.
12. Clemen, R. T. (1989), “Combining Forecasts: A Review and Annotated Bibliography,” International Journal of Forecasting, 5, 559–583.
13. Cleveland, W. S. (1993), Visualizing Data, Summit, NJ: Hobart Press.
14. Foster, D. P., and Vohra, R. V., (1993), “A Randomization Rule for Selecting Forecasts,” Operations Research, 41, 704–709.
15. Hart, J. D. (1997), Nonparametric Smoothing and Lack-of-Fit Tests, Berlin: Springer-Verlag.
16. Haussler, D., Kivinen, J., and Warmuth, M. K. (1998), “Sequential Prediction of Individual Sequences Under General Loss Functions,” IEEE Transactions on Information Theory, 44, 1906–1925.
17. Hoeting, J. A., Madigan, D., Raftery, A. E., and Volinsky, C. T. (1999), “Bayesian Model Averaging: A Tutorial,” Statistical Science, 14, 382–417. Correction, 15, 193–195.
18. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., and Hinton, G. E. (1991), “Adaptive Mixtures of Local Experts,” Neural Computations, 3, 79–87.
19. Jiang, W., and Tanner, M. A. (1999), “Hierarchical Mixtures-of-Experts for Exponential Family Regression Models: Approximation and Maximum Likelihood Estimation,” The Annals of Statistics, 27, 987–1011.
20. Jiang, W., and Tanner, M. A. (2000), “On the Asymptotic Normality of Hierarchical Mixtures-of-Experts for Generalized Linear Models,” IEEE Transactions on Information Theory, 46, 1005–1013.
21. Jordan, M. I., and Jacobs, R. A. (1994), “Hierarchical Mixtures of Experts and the EM Algorithm,” Neural Computations, 6, 181–214.
22. Juditsky, A., and Nemirovski, A. (2000), “Functional Aggregation for Nonparametric Estimation,” The Annals of Statistics, 28, 681–712.
23. Kivinen, J., and Warmuth, M. K. (1999), “Averaging Expert Predictions,” Eurocolt 99.
24. LeBlanc, M., and Tibshirani, R (1996), “Combining Estimates in Regression and classification,” Journal of the American Statistical Association, 91, 1641–1650.
25. Littlestone, N., and Warmuth, M. K. (1994), “The Weighted Majority Algorithm,” Information and Computation, 108, 212–261.
26. Loader, C. R. (1999), “Bandwidth Selection: Classical or Plug-In?,” The Annals of Statistics, 27, 415–438.
27. Lugosi, G., and Nobel, A. (1999), “AdaptiveModel Selection Using Empirical Complexities. The Annals of Statistics, 27, 1830–1864.
28. Merhav, N., and Feder, M. (1998), “Universal prediction,” IEEE Transactions on Information Theory, 44, 2124–2147.
29. Olkin, I., and Spiegelman, C. H. (1987), “A Semiparametric Approach to Density Estimation,” Journal of the American Statistical Association, 82, 858–865.
30. Rice, J. (1984), “Bandwidth Choice for Nonparametric Regression,” The Annals of Statistics, 12, 1215–1230.
31. Ruppert, D., Wand, M. P., Holst, U., and Hössjer, O. (1997), “Local Polynomial Variance-Function Estimation,” Journal of the American Statistical Association, 39, 262–273.
32. Schwartz, G. (1978), “Estimating the Dimension of a Model,” The Annals of Statistics, 6, 461–464.
33. Simonoff, J. S. (1996), Smoothing Methods in Statistics, Berlin: Springer-Verlag.
34. Stone, M. (1974), “Cross-Validatory Choice and Assessment of Statistical Predictions” (with discussion), Journal of the Royal Statistical Society, Ser. B, 36, 111–147.
35. Venable and Ripley (1997). Modern Applied Statistics with S-Plus, 2nd edition, Springer-Verlag, New York.
36. Vovk, V. G. (1990), “Aggregating Strategies,” in Proceedings of the 3rd Annual Workshop on Computational Learning Theory, pp. 372–383.
37. Vovk, V. G. (1998), “A Game of Prediction With Expert Advice,” Journal of Computer and System Sciences, 56 153–173.
38. Wolpert, D. (1992), “Stacked Generalization,” Neural Networks, 5, 241–259.
39. Yang, Y. (1999a), “Model Selection for Nonparametric Regression,” Statistica Sinica, 9, 475–499.
40. Yang, Y. (1999b), “Regression With Multiple Candidate Models: Selecting or Mixing?” Technical Report 8, Iowa State University, Dept. of Statistics.
41. Yang, Y. (2000a), “Mixing Strategies for Density Estimation,” The Annals of Statistics, 28, 75–87.
42. Yang, Y. (2000b), “Combining Different Procedures for Adaptive Regression,” Journal of Multivariate Analysis, 74, 135–161.
43. Yang, Y. (2000c), “Adaptive Estimation in Pattern Recognition by Combining Different Procedures,” Statistica Sinica, 10, 1069–1089.
44. Yang, Y., and Barron, A. R. (1998), “An Asymptotic Property of Model Selection Criteria,” IEEE Transactions on Information Theory, 44, 95–116.
45. Yang, Y., and Barron, A. R. (1999), “Information-Theoretic Determination of Minimax Rates of Convergence,” The Annals of Statistics, 27, 1564–1599.