Boosted classification trees and class probability/quantile estimation

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 409-439

  Mease, David ^a   Wyner, Abraham J ^b   Buja, Andreas ^b  

^a SAN JOSE STATE UNIVERSITY   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

AdaBoost; Boosting algorithms; Class probability estimation; Data jittering; LogitBoost; Over sampling; Stratification; Under sampling

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); DATA REDUCTION; ERROR ANALYSIS; PROBABILITY; SAMPLED DATA CONTROL SYSTEMS;

ADABOOST; BOOSTING ALGORITHMS; CLASS PROBABILITY ESTIMATION; DATA JITTERING; LOGITBOOST; OVER-SAMPLING; UNDER-SAMPLING;

DECISION TREES;

EID: 33947284406     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. P. J. Bickel, Y. Ritov, and A. Zakai. Some theory for generalized boosting algorithms. Journal of Machine Learning Research, 7:705-732, 2006.
2. G. Blanchard, G. Lugosi, and N. Vayatis. On the rate of convergence of regularized boosting classifiers. Journal of Machine Learning Research, 4:861-894, 2003.
3. A. Buja, W. Stuetzle, and Y. Shen. Loss functions for binary class probability estimation and classification: Structure and applications. 2006.
4. P. Chan and S. Stolfo. Toward scalable learning with non-uniform class and cost distributions: A case study in credit card fraud detection. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pages 164-168, 1998.
5. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. Smote: Synthetic minority over-sampling technique. Journal of 'Artificial Intelligence Research, 16:321-357, 2002.
6. N. V. Chawla, A. Lazarevic, L. O. Hall, and K. W. Bowyer. Smoteboost: Improving prediction of the minority class in boosting. In Proceedings of 7th European Conference on Principles and Practice of Knowledge Discovery in Databases, pages 107-119, 2003.
7. W. W. Cohen and Y. Singer. A simple, fast, and effective rule learner. In Proceedings of the 16th National Conference on Artificial Intelligence (AAAI), pages 335-342, 1999.
8. M. Collins, R. E. Schapire, and Y. Singer. Logistic regression, adaboost and bregman distances. In Computational Learing Theory, pages 158-169, 2000.
9. M. Dettling and P. Buhlmann. Boosting for tumor classification with gene expression data. Bioinformatics, 19:1061-1069, 2003.
10. N. Duffy and D. Helmbold. Potential boosters? In Advances in Neural Information Processing Systems, pages 258-264, 1999.
11. C. Elkan. The foundations of cost-sensitive learning. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI), pages 973-978, 2001.
12. A. Estabrooks, T. Jo, and N. Japkowicz. A multiple resampling method for learning from imbalanced data sets. Computational Intelligence, 20:18-36, 2004.
13. W. Fan, S. Stolfo, J. Zhang, and P. Chan. Adacost: Misclassification cost-sensitive boosting. In Proceedings of the 16th International Conference on Machine Learning, pages 97-105, 1999.
14. D. P. Foster and R. A. Stine. Variable selection in data mining: Building a predictive model for bankruptcy. Journal of the American Statistical Association, 99:303-313, 2004.
15. Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pages 148-156, 1996.
16. Y. Freund and R. E. Schapire. Discussion of three papers regarding the asymptotic consistency of boosting. Annals of Statistics, 32:113-117, 2004.
17. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28:337-374, 2000.
18. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2001.
19. W. Jiang. Process consistency for adaboost. Annals of Statistics, 32:13-29, 2004.
20. W. Jiang. Does boosting overfit: Views from an exact solution. Technical Report 00-03, Department of Statistics, Northwestern University, 2000.
21. M. Joshi, V. Kumar, and R. Agarwal. Evaluating boosting algorithms to classify rare classes: Comparison and improvements. In Proceedings of the First IEEE International Conference on Data Mining (ICDM), pages 257-264, 2001.
22. G. Lebanon and J. Lafferty. Boosting and maximum likelihood for exponential models. In Advances in Neural Information Processing Systems, 2001.
23. Y. Liu, Y. Yang, and J. Carbonell. Boosting to correct inductive bias in text classification. In Proceedings of the Eleventh International Conference on Information and Knowledge Management, pages 348-355, 2002.
24. G. Lugosi and N. Vayatis. On the bayes-risk consistency of regularized boosting methods. Annals of Statistics, 32:30-55, 2004.
25. D. Mease and A. Wyner. Evidence contrary to the statistical view of boosting. 2007.
26. S. Rosset, J. Zhu, and T. Hastie. Boosting as a regularized path to a maximum margin classifier. Journal of Machine Learning Research, 5:941-973, 2004.
27. L. J. Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66:783-801, 1973.
28. K. M. Ting. A comparative study of cost-sensitive boosting algorithms. In Proceedings of the 17th International Conference on Machine Learning, pages 983-990, 2000.
29. B. Zadrozny and C. Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 609-616, 2001.
30. T. Zhang and B. Yu. Boosting with early stopping: Convergence and consistency. Annals of Statistics, 33:1538-1579, 2005.