PAC-Bayesian stochastic model selection

Machine Learning

Volumn 51, Issue 1, 2003, Pages 5-21

  McAllester, David A ^a  

^a AT AND T LABS RESEARCH   (United States)

Author keywords

Gibbs distribution; Model averaging; PAC learning; PAC Bayesian learning; Posterior distribution

Indexed keywords

LEARNING ALGORITHMS; MATHEMATICAL MODELS; PROBABILITY DISTRIBUTIONS; RANDOM PROCESSES;

BAYESIAN LEARNING; GIBBS DISTRIBUTION; POSTERIOR DISTRIBUTION;

LEARNING SYSTEMS;

EID: 0037399538     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1021840411064     Document Type: Article

References (23)

1. Barron, A. R. (1991). Complexity regularization with application to artificial neural networks. In G. Roussas (Ed.), Nonparametric functional estimation and related topics (pp. 561-576). Dordrecht: Kluwer Academic Publishers.
2. Barron, A. R., & Cover, T. M. (1991). Minimum complexity density estimation. IEEE Transactions on Information Theory, 37, 1034-1054.
3. Buntine, W. (1992). Learning classification trees. Statistics and Computing, 2, 63-73.
4. Catoni, O. (To appear a) Gibbs estimators. Probability Theory and Related Fields.
5. Catoni, O. (To appear b) Universal aggregation rules with sharp oracle inequalities. Annals of Statistics.
6. Cesa-Bianchi, N., Freund, Y., Helmbold, D. P., Haussler, D., Schpire, R. E., & Warmuth, M. K. (1997). How to use expert advices. JACM, 44:3, 427-485.
7. Freund, Y., & Schapire, R. E. (1999). Adaptive game playing using multiplicative weights. Games and Economic Behavior, 79-103.
8. Freund, Y., Schapire, R. E., Singer, Y., & Warmuth, M. K. (1997). Using and combining predictors that specialize. In COLT-97.
9. Helmbold, D. P., & Schapire, R. E. (1997). Predicting nearly as well as the best pruning of a decision tree. Machine Learning, 27:1, 51-68.
10. Kearns, M., & Mansour, Y. (1998). A fast, bottom-up decision tree pruning algorithm with near-optimal generalization. In Proceedings of the 15th International Conference on Machine Learning. San Mateo, CA: Morgan Kaufmann.
11. Kearns, M., Mansour, Y., Ng, A., & Ron, D. (1995). An experimental and theoretical comparison of model selection methods. In Proceedings of the Eighth ACM Conference on Computational Learning Theory (pp. 21-30). New York: ACM Press.
12. Linial, N., Mansour, Y., & Rivest, R. (1991). Results on learnability and the Vapnik-Chervonenski dimension. Information and Computation, 90, 33-49.
13. Littlestone, N., & Warmuth, M. (1994). The weighted majority algorithm. Information and Computation, 108:2, 212-261. An extended abstract appeared in COLT-89.
14. Lugosi, G., & Zeger, K. (1996). Concept learning using complexity regularization. IEEE Transactions on Information Theory, 42, 48-54.
15. McAllester, D. (1998). Some pac-bayesian theorems. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory (pp. 230-234).
16. McAllester, D. (1999). Pac-bayesian model averaging. In COLT-99.
17. Oliver, J. J., & Hand, D. (1995). On pruning and averaging decision trees. In Proceedings of the Twelfth International Conference on Machine Learning.
18. Pereira, F. C., & Singer, Y. (1997). An efficient extension to mixture techniques for prediction and decision trees. In Proceedings of the Tenth Annual Conference on Computational Learning Theory (pp. 114-121). Long version to appear in Machine Learning.
19. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann.
20. Shawe-Taylor, J., & Williamson, R. (1997). A pac analysis of a bayesian estimator. In Proceedings of the Tenth Annual Conference on Computational Learning Theory. New York: ACM Press.
21. Yamanishi, K. (1992). Learning non-parametric densities in terms of finite-dimensional parametric hypotheses. IEICE Trans. Inf. and Syst., E75-D(4).
22. Yang, Y. (2000a). Adaptive estimation in pattern recognition by combining different procedures. Statistica Sinica, 10, 1069-1089.
23. Yang, Y. (2000b). Mixing strategies for density estimation. Annals of Statistics, 28, 75-87.