Information theoretic regularization for semi-supervised boosting

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2009, Pages 1017-1025

  Zheng, Lei ^a   Wang, Shaojun ^a   Liu, Yan ^a   Lee, Chi Hoon ^b  

^a WRIGHT STATE UNIVERSITY   (United States)

^b YAHOO RESEARCH   (United States)

Author keywords

Ensemble method; Semi supervised learning

Indexed keywords

BEARING LOSS; BOOSTING ALGORITHM; ENSEMBLE METHOD; FUNCTIONAL GRADIENT DESCENT; GRADIENT DESCENT OPTIMIZATION; LABELED DATA; LINEAR COMBINATIONS; REAL-WORLD TASK; REGULARIZATION FRAMEWORK; SEMI-SUPERVISED; SEMI-SUPERVISED LEARNING; TRAINING DATA; UNLABELED DATA; WEAK CLASSIFIERS;

ALGORITHMS; INFORMATION THEORY; SEQUENTIAL SWITCHING;

SUPERVISED LEARNING;

EID: 71049137802     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1557019.1557129     Document Type: Conference Paper

References (34)

1. K. Benett, A. Demiriz and R. Maclin. Exploiting unlabeled data in ensemble methods. The 8th International Conference on Knowledge Discovery and Data Mining, 289-296, 2002.
2. D. Bertsekas. Nonlinear Programming, 2nd Edition, Athena Scientific, 1999.
3. A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. The Workshop on Computational Learning Theory, 92-100, 1998.
4. S. Boyd and L. Vandenberghe. Convex Optimization, Cambridge University Press, 2004.
5. L. Breiman. Prediction games and arcing classifiers. Neural Computation, 11:1493-1517, 1999.
6. V. Castelli and T. Cover. The relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter. IEEE Trans. on Information Theory, 42(6):2102-2117, 1996.
7. G. Celeux and G. Govaert. A classification EM algorithm for clustering and two stochastic versions. Computational Statistics and Data Analysis, 14:315-332, 1992.
8. O. Chapelle, B. Scholköpf and A. Zien. Semi-Supervised Learning, MIT Press, 2006.
9. K. Chen and S. Wang. Regularized boost for semi-supervised learning. Advances in Neural Information Processing Systems 20, 2007.
10. I. Cohen and F. Cozman. Risks of semi-supervised learning. Semi-Supervised Learning, O. Chapelle, B. Scholköpf and A. Zien, 55-70, MIT Press, 2006.
11. M. Collins, R. Schapire and Y. Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1-3):253-285, 2002.
12. A. Corduneanu and T. Jaakkola. Data dependent regularization. Semi-Supervised Learning, O. Chapelle, B. Scholköpf and A. Zien, 163-182, MIT Press, 2006.
13. T. Cover and J. Thomas. Elements of Information Theory, John Wiley & Sons, 1991.
14. F. d'Alché-Buc, Y. Grandvalet and C. Ambroise. Semi-supervised marginBoost. Advances in Neural Information Processing Systems 14, 553-560, 2002.
15. S. Della Pietra, V. Della Pietra and J. Lafferty. Inducing features of random fields. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(4):380-393, 1997.
16. Y. Freund and R. Schapire. Experiments with a new boosting algorithm. The 13th International Conference on Machine Learning, 148-156, 1996.
17. Y. Freund and R. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-139, 1997.
18. J. Friedman, T.Hastie and R. Tibshirani. Additive logistic regression: A statistical view of boosting. The Annals of Statistics, 28(2):337-407, 2000.
19. Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems, 17:529-536, 2004.
20. G. Haffari, Y. Wang, S. Wang, G. Mori and F. Jiao. Boosting with incomplete information. The 25th International Conference on Machine Learning, 368-375, 2008.
21. T. Hastie, R. Tibshirani, J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition, Springer, 2009.
22. J. Janodet, R. Nock, M. Sebban and H. Suchier. Boosting grammatical inference with confidence oracles. The 21st International Conference on Machine Learning, 54-61, 2004.
23. F. Jiao, S. Wang, C. Lee, R. Greiner and D. Schuurmans. Semi-supervised conditional random fields for improved sequence segmentation and labeling. The Joint 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, 209-216, 2006.
24. G. Lebanon and J. Lafferty. Boosting and maximum likelihood for exponential models. Advances in Neural Information Processing Systems 14, 447-454, 2002.
25. C. Lee, S. Wang, F. Jiao, D. Schuurmans and R. Greiner. Learning to model spatial dependency: Semi-supervised discriminative random fields. Advances in Neural Information Processing, 19, 793-800, 2007.
26. L. Mason, J. Baxter, P. Bartlett and M. Frean. Functional gradient techniques for combining hypotheses. In Advances in Large Margin Classifiers, A. Smola, P. Bartlett, B. Scholköpf and D. Schuurmans, editors, 221-246, MIT Press, 2000.
27. K. Nigam, A. McCallum, S. Thrun and T. Mitchell. Text classification from labeled and unlabeled documents using EM. Machine Learning. 39(2/3):135-167, 2000.
28. S. Roberts, R. Everson and I. Rezek. Maximum certainty data partitioning. Pattern Recognition, 33(5):833-839, 2000.
29. R. Schapire. The strength of weak learnability. Machine Learning, 5(2):197-227, 1990.
30. H. Valizadegan, R. Jin and A. Jain. Semi-supervised boosting for multi-class classification. The European Conference on Machine Learning and Knowledge Discovery in Databases, 522-537, 2008.
31. Y. Wang, G. Haffari, S. Wang and G. Mori. Rate distortion based semi-supervised discriminative learning. Technical Report, 2009.
32. D. Zhou, O. Bousquet, T. Navin Lal, J. Weston and B. Schölkopf. Learning with local and global consistency. Advances in Neural Information Processing Systems, 16:321-328, 2004.
33. J. Zhu, S. Rosset, H. Zhou and T. Hastie. Multiclass AdaBoost. Technical Report, 2005.
34. X. Zhu, Z. Ghahramani and J. Lafferty. Semi-supervised learning using Gaussian fields and harmonic functions. The 20th International Conference on Machine Learning, 912-919, 2003.