Generalization bounds for averaged classifiers

Annals of Statistics

Volumn 32, Issue 4, 2004, Pages 1698-1722

  Freund, Yoav ^a,d   Mansour, Yishay ^b   Schapire, Robert E ^c,e  

^a Och Spine at New York Presbyterian Hospitals   (United States)

^b TEL AVIV UNIVERSITY   (Israel)

^c PRINCETON UNIVERSITY   (United States)

^d Columbia University ^*  (United States)

^e Princeton University   (United States)

Author keywords

Averaging; Bayesian methods; Classification; Ensemble methods; Generalization bounds

Indexed keywords

EID: 24344500472     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053604000000058     Document Type: Article

References (23)

1. ALLWEIN, E. L., SCHAPIRE, R. E. and SINGER, Y. (2000). Reducing multiclass to binary: A unifying approach for margin classifiers. J. Mach. Learn. Res. 1 113-141.
2. BLUMER, A., EHRENFEUCHT, A., HAUSSLER, D. and WARMUTH, M. K. (1987). Occam's razor. Inform. Process. Lett. 24 377-380.
3. BOUSQUET, O. and ELISSEEFF, A. (2002). Stability and generalization. J. Mach. Learn. Res. 2499-526.
4. BREIMAN, L. (1996). Bagging predictors. Machine Learning 24 123-140.
5. BREIMAN, L. (1996). Heuristics of instability and stabilization in model selection. Ann. Statist. 24 2350-2383.
6. CESA-BIANCHI, N., FREUND, Y., HAUSSLER, D., HELMBOLD, D. P., SCHAPIRE, R. E. and WARMUTH, M. K. (1997). How to use expert advice. J. ACM 44 427-485.
7. DE BRUIJN, N. G. (1981). Asymptotic Methods in Analysis, 3rd ed. Dover, New York.
8. FREUND, Y. (2003). Predicting a binary sequence almost as well as the optimal biased coin. Inform. and Comput. 182 73-94.
9. FREUND, Y. and MASON, L. (1999). The alternating decision tree learning algorithm. In Proc. Sixteenth International Conference on Machine Learning 124-133. Morgan Kaufmann, San Francisco.
10. FREUND, Y. and SCHAPIRE, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119-139.
11. FRIEDMAN, J. H. (1997). On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Min. Knowl. Discov. 1 55-77.
12. HELMBOLD, D. P. and SCHAPIRE, R. E. (1997). Predicting nearly as well as the best pruning of a decision tree. Machine Learning 27 51-68.
13. HOEFFDING, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30.
14. LITTLESTONE, N. and WARMUTH, M. K. (1994). The weighted majority algorithm. Inform. and Comput. 108 212-261.
15. MACKAY, D. J. C. (1991). Bayesian methods for adaptive models. Ph.D dissertation, California Institute of Technology.
16. MCALLESTER, D. A. (1999). Some PAC-Bayesian theorems. Machine Learning 37 355-363.
17. MCDIARMID, C. (1989). On the method of bounded differences. In Surveys in Combinatorics 1989 148-188. Cambridge Univ. Press.
18. SCHAPIRE, R. E., FREUND, Y., BARTLETT, P. and LEE, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651-1686.
19. SHAWE-TAYLOR, J., BARTLETT, P. L., WILLIAMSON, R. C. and ANTHONY, M. (1998). Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inform. Theory 44 1926-1940.
20. SHAWE-TAYLOR, J. and WILLIAMSON, R. C. (1997). A PAC analysis of a Bayesian estimator. In Proc. Tenth Annual Conference on Computational Learning Theory 2-9. ACM Press, New York.
21. VAPNIK, V. N. (1998). Statistical Learning Theory. Wiley, New York.
22. VOVK, V. (2001). Transductive confidence machines. Unpublished manuscript.
23. WILLEMS, F. M. J., SHTARKOV, Y. M. and TJALKENS, T. J. (1995). The context-tree weighting method: Basic properties. IEEE Trans. Inform. Theory 41 653-664.