Generalization bounds and complexities based on sparsity and clustering for convex combinations of functions from random classes

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  Jaeger, Savina Andonova ^a  

^a HARVARD MEDICAL SCHOOL   (United States)

Author keywords

Complexities of classifiers; Generalization bounds; Random classes; SVM; Voting classifiers

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; DATA PROCESSING; FUNCTIONS; RADIAL BASIS FUNCTION NETWORKS; SET THEORY; STATISTICAL METHODS; VECTORS;

COMPLEXITIES OF CLASSIFIERS; GENERALIZATION BOUNDS; RANDOM CLASSES; SVM; VOTING CLASSIFIERS;

RANDOM PROCESSES;

EID: 21844440024     PISSN: 15337928     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (24)

1. Andonova, S. Theoretical and experimental analysis of the generalization ability of some statistical learning algorithms. PhD thesis, Department of Mathematics and Statistics, Boston University, Boston, MA, 2004.
2. Anthony, M., Shawe-Taylor, J. A result of Vapnik with applications. Discrete Applied Mathematics, 47(3): 207-217, 1993.
3. Bartlett, P., Shawe-Taylor, J. Generalization performance of support vector machines and other pattern classifiers. Advances in Kernel Methods. Support Vector Learning. Schoïlkopf, Burges and Smola (Eds.), 1, The MIT Press, Cambridge, 1999.
4. Borowkow, A. A. Wahrscheinlichkeits-theorie. Birkhäuser Verlag, 1976.
5. Cannon, A., Ettinger, M., Hush, D., Scovel, C. Machine Learning with Data Dependent Hypothesis Classes. Journal of Machine Learning Research, 2: 335-358, 2002.
6. Breiman, L. Prediction games and arcing algorithms. Neural Computation, 11(7): 1493-1517, 1999.
7. Dudley, R. M. Uniform Central Limit Theorems. Cambridge University Press, 1999.
8. Feller, W. An Introduction to probability theory and its applications, volume II. John Wiley, 1966.
9. Gat, Y. A bound concerning the generalization ability of a certain class of learning algorithms. Technical Report 548, University of California, Berkeley, CA, 1999.
10. Heisele, B., Serre, T., Mukherjee, S., Poggio, T. Feature Reduction and Hierarchy of Classifiers for Fast Object Detection in Video Images. In. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2: 18-24, 2001.
11. Kohonen, T. The self-organizing map. In. Proceedings of IEEE, 78: 1464-1479, 1990.
12. Koltchinskii, V., Panchenko, D. Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1), 2002.
13. Koltchinskii, V., Panchenko, D. Complexities of convex combinations and bounding the generalization error in classification, submitted, 2004.
14. Koltchinskii, V., Panchenko, D., Lozano, F. Bounding the generalization error of convex combinations of classifiers: balancing the dimensionality and the margins. The Annals of Applied Probability, 13(1), 2003a.
15. Koltchinskii, V., Panchenko, D., Andonova, S. Generalization bounds for voting classifiers based on sparsity and clustering. In Proceedings of the Annual Conference on Computational Learning Theory, Lecture Notes in Artificial Intelligence, M. Warmuth and B. Schoelkopf (eds.). Springer, New York, 2003b.
16. Littlestone, N., Warmuth, M. Relating data compression and learnability. Technical Report, University of California, Santa Cruz, CA, 1986.
17. Panchenko, D. Some extensions of an inequality of Vapnik and Cervonenkis. Electronic Communications in Probability, 7: 55-65, 2002.
18. Schapire, R., Freund, Y., Bartlett, P., Lee, W. S. Boosting the margin: A new explanation of effectiveness of voting methods. The Annals of Statistics, 26: 1651-1687, 1998.
19. Schapire, R., Singer, Y. Improved Boosting Algorithms using Confidence-Rated Predictions. Machine Learning, 37: 297-336, 1999.
20. Steinwart, I. Sparseness of Support Vector Machines. Journal of Machine Learning Research, 2: 1071-1105, 2003.
21. Vapnik, V. N., Červonenkis, A. Ya. On the uniform convergence of relative frequencies of event to their probabilities. Soviet Math. Dokl, 9: 915-918, 1968.
22. Vapnik, V. N., Červonenkis A. Ya. Theory of Pattern Recognition. Nauka, Moscow (in Russian), 1974.
23. Vapnik, V. Statistical Learning Theory. John Wiley & Sons, New York, 1998.
24. Vapnik, V. Estimation of Dependencies Based on Empirical Data. SpringerVerlag, New York, 1982.