Boosting with diverse base classifiers

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

Volumn 2777, Issue , 2003, Pages 273-287

  Dasgupta, Sanjoy ^a   Long, Philip M ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b GENOME INSTITUTE OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CLASSIFIERS; ERROR ANALYSIS; LEARNING SYSTEMS; PROBLEM SOLVING; THEOREM PROVING;

ADABOOST; BASE CLASSIFIERS; BOOST-BY-MAJORITY ALGORITHM; GENERALIZATION; CROSS VALIDATION; GENERALIZATION ERROR; LARGE DATASETS; LEARNING METHODS;

DATABASE SYSTEMS; ADAPTIVE BOOSTING;

EID: 9444224951     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/978-3-540-45167-9_21     Document Type: Conference Paper

References (51)

1. K. M. Ali and M. J. Pazzani. Error reduction through learning multiple descriptions. Machine Learning, 24:173-202, 1996.
2. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler. Scale-sensitive dimensions, uniform convergence, and learnability. Journal of the Association for Computing Machinery, 44(4):616-631, 1997.
3. Y. Amit and G. Blanchard. Multiple randomized classifiers: MRCL, 2001. Manuscript.
4. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
5. P. L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536, 1998.
6. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. Technical Report 638, Department of Statistics, U.C. Berkeley, 2003.
7. G. Blanchard, G. Lugosi, and N. Vayatis. On the rate of convergence of regularized boosting methods, 2003. Manuscript.
8. L. Breiman. Prediction games and arcing algorithms. Neural Computation, 11(7), 1999.
9. L. Breiman. Some infinity theory for predictor ensembles. Technical Report 577, Statistics Department, UC Berkeley, 2000.
10. Leo Breiman. Arcing classifiers. The Annals of Statistics, 1998.
11. P. Bülmann and B. Yu. Boosting with the 12 loss: regression and classification. Journal of the American Statistical Association, to appear.
12. D. Dubhashi and D. Ranjan. Balls and bins: A study in negative dependence. Random Structures & Algorithms, 13(2):99-124, Sept 1998.
13. S. Dudoit, J. Fridlyand, and T. P. Speed. Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association, 97(457):77-87, 2002.
14. Y. Freund. Boosting a weak learning algorithm by majority. Information and Computation, 121(2):256-285, 1995.
15. Y. Freund and R. Schapire. Experiments with a new boosting algorithm. Proceedings of the Thirteenth International Conference on Machine Learning, 1996.
16. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Proceedings of the Second European Conference on Computational Learning Theory, pages 23-37, 1995.
17. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. The Annals of Statistics, 38(2):337-407, 2000.
18. D. Gavinsky. Optimally-smooth adaptive boosting and application to agnostic learning. Proceedings of the 13th International Workshop on Algorithmic Learning Theory, 2002.
19. A.J. Grove and D. Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In Proceedings of the Fifteenth National Conference on Artifical Intelligence, 1998.
20. A. Hajnal, W. Maass, P. Pudlák, M. Szegedy, and G. Turán. Threshold circuits of bounded depth. Journal of Computer and System Sciences, 46:129-154, 1993.
21. Russell Impagliazzo. Hard-core distributions for somewhat hard problems. In IEEE Symposium on Foundations of Computer Science, pages 538-545, 1995.
22. W. Jiang. Process consistency for AdaBoost. Annals of Statistics, to appear.
23. Adam Klivans and Rocco A. Servedio. Boosting and hard-core sets. In IEEE Symposium on Foundations of Computer Science, pages 624-633, 1999.
24. V. Koltchinskii and D. Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics, 30(1), 2002.
25. V. Koltchinskii and D. Panchenko. Complexities of convex combinations and bounding the generalization error in classification, 2003. Manuscript.
26. J. Langford and J. Shawe-Taylor. PAC-bayes and margins. NIPS, 2002.
27. J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer, 2001.
28. J. S. Liu and R. Chen. Sequential Monte Carlo methods for dynamic systems. Journal of the American Statistical Association, 93(443): 1032-1044, 1998.
29. P. M. Long. Minimum majority classification and boosting, Proceedings of the The Eighteenth National Conference on Artificial Intelligence, 2002.
30. P. M. Long and V. B. Vega. Boosting and microarray data. Machine Learning, to appear.
31. G. Lugosi and N. Vayatis. On the bayes-risk consistency of regularized boosting methods. Annals of Statistics, 2004. Preliminary version in COLT'02.
32. S. Mannor, R. Meir, and S. Mendelson. The consistency of boosting algorithms. Manuscript, 2001.
33. S. Mannor, R. Meir, and T. Zhang. The consistency of greedy algorithms for classification. Proc. Fifteenth Annual Conference on Computational Learning Theory, 2002.
34. Dragos D. Margineantu and Thomas G. Dietterich. Pruning adaptive boosting. In Proc. 14th International Conference on Machine Learning, pages 211-218. Morgan Kaufmann, 1997.
35. L. Mason, J. Baxter, P. L. Bartlett, and M. Frean. Boosting algorithms as gradient descent. In Advances in Neural Information Processing Systems 12, pages 512-518. MIT Press, 2000.
36. Llew Mason, Peter L. Bartlett, and Jonathan Baxter. Improved generalization through explicit optimization of margins. Machine Learning, 38(3):243-255, 2000.
37. D. McAllester. Simplified PAC-Bayesian margin bounds. Proceedings of the 2003 Conference on Computational Learning Theory, 2003.
38. David A. McAllester. PAC-Bayesian model averaging. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 164-170. ACM Press, New York, NY, 1999.
39. David A. McAllester. Some PAC-Bayesian theorems. Machine Learning, 37(3):355-363, 1999.
40. R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, 1995.
41. -niyogi/decorrelation.ps.
42. G. Pisier. Remarques sur un resultat non publi'e de B. Maurey. Sem. d'Analyse Fonctionelle, 1(12): 1980-81, 1981.
43. J. Quinlan. Bagging, boosting and c4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pages 725-730. AAAI/MIT Press, 1996.
44. G. Rätsch and M. K. Warmuth. Marginal boosting. Proceedings of the Annual Conference on Computational Learning Theory, 2002.
45. S. Rosset, J. Zhu, and T. Hastie. Boosting as a regularized path to a maximum margin classifier. NIPS, 2002.
46. R. Schapire. The strength of weak learnability. Machine Learning, 5:197-227, 1990.
47. Robert E. Schapire, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5):1651-1686, 1998.
48. F. Southey, D. Schuurmans, and A. Ghodsi. Regularized greedy importance sampling. NIPS'02.
49. M. West, et al. Predicting the clinical status of human breast cancer by using gene expression profiles. Proc. Natl. Acad. Sci. USA, 98(20):11462-11467, 2001.
50. T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. Annals of Statistics, to appear.
51. T. Zhang and B. Yu. Boosting with early stopping: convergence and consistency. Technical Report 635, Statistics Department, UC Berkeley, 2003.