Support vector machinery for infinite ensemble learning

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 285-312

  Lin, Hsuan Tien ^a   Li, Ling ^a  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Boosting; Ensemble learning; Kernel; Support vector machine

Indexed keywords

ALGORITHMS; GAUSSIAN DISTRIBUTION; PARAMETER ESTIMATION; RADIAL BASIS FUNCTION NETWORKS;

BOOSTING; ENSEMBLE LEARNING; KERNEL;

SUPPORT VECTOR MACHINES;

EID: 41549137738     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (41)

1. Annalisa Barla, Francesca Odone, and Alessandro Verri. Histogram intersection kernel for image classification. In Proceedings of the 2003 Conference on Image Processing, volume 3, pages 513-516, 2003.
2. Brad J. C. Baxter. Conditionally positive functions and p-norm distance matrices. Constructive Approximation, 7(1):427-440, 1991.
3. Christian Berg, Jens P. R. Christensen, and Paul Ressel. Harmonic Analysis on Semigroups: Theory of Positive Definite and Related Functions. Springer-Verlag, 1984.
4. Sabri Boughorbel, Jean-Philippe Tarel, and Nozha Boujemaa. Generalized histogram intersection kernel for image recognition. In Proceedings of the 2005 Conference on Image Processing, volume 3, pages 161-164, 2005.
5. Leo Breiman. Prediction games and arcing algorithms. Neural Computation, 11(7): 1493-1517, 1999.
6. Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines, 2001a. Software available at http://www.csie.ntu.edu.tw/ ~cjlin/libsvm.
7. Chih-Chung Chang and Chih-Jen Lin. Training v-support vector classifiers: Theory and algorithms. Neural Computation, 13(9):2119-2147, 2001b.
8. Olivier Chapelle, Patrick Haffner, and Vladimir N. Vapnik. Support vector machines for histogrambased image classification. IEEE Transactions on Neural Networks, 10(5): 1055-1064, 1999.
9. Ayhan Demiriz, Kristin P. Bennett, and John Shawe-Taylor. Linear programming boosting via column generation. Machine Learning, 46(1-3):225-254, 2002.
10. Thomas G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40(2):139-157, 2000.
11. François Fleuret and Hichem Sahbi. Scale-invariance of support vector machines based on the triangular kernel. In Proceedings of the Third International Workshop on Statistical and Computational Theories of Vision, 2003.
12. Yoav Freund and Robert E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pages 148-156, 1996.
13. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1): 119-139, 1997.
14. Yoav Freund and Robert E. Schapire. A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence, 14(5):771-780, 1999.
15. Thore Graepel, Ralf Herbrich, and John Shawe-Taylor. PAC-Bayesian compression bounds on the prediction error of learning algorithms for classification. Machine Learning, 59(1-2):55-76, 2005.
16. Kristen Grauman and Trevor Darrell. The pyramid match kernel: Discriminative classification with sets of image features. In Proceedings of the Tenth IEEE International Conference on Computer Vision, pages 1458-1465, 2005.
17. Trevor Hastie and Robert Tibshirani. Generalized Additive Models. Chapman and Hall, 1990.
18. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, 2001.
19. Simon Haykin. Neural Networks: A Comprehensive Foundation. Prentice Hall, second edition, 1999.
20. Seth Hettich, Catherine L. Blake, and Christopher J. Merz. UCI repository of machine learning databases, 1998. Downloadable at http://www.ics.uci.edu/ ~mlearn/MLRepository.html.
21. Robert C. Holte. Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11(1):63-91, 1993.
22. Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin. A practical guide to support vector classification. Technical report, National Taiwan University, 2003.
23. S. Sathiya Keerthi and Chih-Jen Lin. Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation, 15(7): 1667-1689, 2003.
24. Ling Li and Hsuan-Tien Lin. Optimizing 0/1 loss for perceptrons by random coordinate descent. In Proceedings of the 2007 International Joint Conference on Neural Networks, pages 749-754, 2007.
25. Hsuan-Tien Lin. Infinite ensemble learning with support vector machines. Master's thesis, California Institute of Technology, 2005.
26. Hsuan-Tien Lin and Ling Li. Novel distance-based SVM kernels for infinite ensemble learning. In Proceedings of the 12th International Conference on Neural Information Processing, pages 761-766, 2005a.
27. Hsuan-Tien Lin and Ling Li. Analysis of SAGE results with combined learning techniques. In P. Berka and B. Crémilleux, editors, Proceedings of the ECML/PKDD 2005 Discovery Challenge, pages 102-113, 2005b.
28. Ron Meir and Gunnar Rätsch. An introduction to boosting and leveraging. In S. Mendelson and A. J. Smola, editors, Advanced Lectures on Machine Learning, pages 118-183. Springer-Verlag, 2003.
29. Charles A. Micchelli. Interpolation of scattered data: Distance matrices and conditionally positive definite functions. Constructive Approximation, 2(1): 11-22, 1986.
30. John C. Piatt. Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods: Support Vector Learning, pages 185-208. MIT Press, 1999.
31. J. Ross Quinlan. Induction of decision trees. Machine Learning, 1(1):81-106, 1986.
32. Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
33. Gunnar Ratsch, Takashi Onoda, and Klaus-Robert Müller. Soft margins for AdaBoost. Machine Learning, 42(3):287-320, 2001.
34. Gunnar Ratsch, Ayhan Demiriz, and Kristin P. Bennett. Sparse regression ensembles in infinite and finite hypothesis spaces. Machine Learning, 48(1-3): 189-218, 2002.
35. Gunnar Ratsch, Sebastian Mika, Bernhard Schölkopf, and Klaus-Robert Müller. Constructing boosting algorithms from SVMs: an application to one-class classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9): 1184-1199, 2002.
36. Michael Reed and Barry Simon. Functional Analysis. Academic Press, revised and enlarged edition, 1980.
37. Saharon Rosset, Ji Zhu, and Trevor Hastie. Boosting as a regularized path to a maximum margin classifier. Journal of Machine Learning Research, 5:941-973, 2004.
38. 1 regularization in infinite dimensional feature spaces. In N. H. Bshouty and C. Gentile, editors, Learning Theory: 20th Annual Conference on Learning Theory, volume 4539 of Lecture Notes in Computer Science, pages 544558. Springer-Verlag, 2007.
39. Bernhard Schölkopf and Alex J. Smola. Learning with Kernels. MIT Press, 2002.
40. Vladimir N. Vapnik. Statistical Learning Theory. John Wiley & Sons, 1998.
41. Christopher K. I. Williams. Computation with infinite neural networks. Neural Computation, 10 (5): 1203-1216, 1998.