On robustness properties of convex risk minimization methods for pattern recognition

Journal of Machine Learning Research

Volumn 5, Issue , 2004, Pages 1007-1034

  Christmann, Andreas ^a   Steinwart, Ingo ^b  

^a TU DORTMUND UNIVERSITY   (Germany)

^b LOS ALAMOS NATIONAL LABORATORY   (United States)

Author keywords

AdaBoost loss function; Influence function; Kernel logistic regression; Robustness; Sensitivity curve; Statistical learning; Support vector machine; Total variation

Indexed keywords

ADAPTIVE BOOSTING; ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; METHOD OF MOMENTS; PATTERN RECOGNITION; REGRESSION ANALYSIS; ROBUSTNESS (CONTROL SYSTEMS); SUPPORT VECTOR MACHINES;

INFLUENCE FUNCTIONS; KERNEL LOGISTIC REGRESSION; LOSS FUNCTIONS; SENSITIVITY CURVES; STATISTICAL LEARNING; TOTAL VARIATION;

SENSITIVITY ANALYSIS;

EID: 18244390064     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (48)

1. R. Akerkar. Nonlinear Functional Analysis. Narosa Publishing House, New Dehli, 1999.
2. P. L. Bartlett, O. Bousquet, and S. Mendelson. Local Rademacher complexities. http://www.stat.berkeley.edu/~bartlett/papers/bbm-lrc-02b.pdf, 2002.
3. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. http://stat-www.berkeley.edu/tech-reports/638.pdf, 2003.
4. P. L. Bartlett and S. Mendelson. Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463-482, 2002.
5. H. Bauer. Maß- und Integrationstheorie. De Gruyter, Berlin, 1990.
6. G. Blanchard, G. Lugosi, and N. Vayatis. On the rate of convergence of regularized boosting classifiers. Journal of Machine Learning Research, 4:861-894, 2003.
7. O. Bousquet and A. Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2:499-526, 2002.
8. A. Brown and C. Pearcy. Introduction to Operator Theory I. Springer, New York, 1977.
9. D.-R. Chen, Q. Wu, Y. Ying, and D.-X. Zhou. Support vector machine soft margin classifiers. City University of Hong Kong, 2003.
10. W. Cheney. Analysis for Applied Mathematics. Springer, New York, 2001.
11. A. Christmann. Least median of weighted squares in logistic regression with large strata. Biometrika, 81:413-417, 1994.
12. A. Christmann. On positive breakdown point estimators in regression models with discrete response variables. 1998. Habilitation thesis, University of Dortmund, Department of Statistics.
13. A. Christmann. On a strategy to develop robust and simple tariffs from motor vehicle insurance data. Technical Report 16/04, University of Dortmund, SFB-475, 2004. http://www.statistik. uni-dortmund.de/download/mitarbeiter/christmann/Christmann-insurance04.pdf.
14. A. Christmann, P. Fischer, and T. Joachims. Comparison between various regression depth methods and the support vector machine to approximate the minimum number of misclassifications. Computational Statistics, 17:273-287, 2002.
15. A. Christmann and P. J. Rousseeuw. Measuring overlap in logistic regression. Computational Statistics and Data Analysis, 37:65-75, 2001.
16. J. Diestel and J. J. Uhl. Vector Measures. American Mathematical Society, Providence, RI, 1977.
17. D. L. Donoho and P. J. Huber. The notion of breakdown point. In P. J. Bickel, K. A. Doksum, and J. L. Hodges Jr, editors, A Festschrift for Erich L. Lehmann, pages 157-184, Belmont, California, Wadsworth, 1983.
18. R. M. Dudley. Real Analysis and Probability. Cambridge University Press, 2002.
19. Y. Freund and R. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the 13th International Conference, pages 148-156. Morgan Kauffman Publishers, San Francisco, 1996.
20. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting (with discussion). Annals of Statistics, 28:337-407, 2000.
21. F. R. Hampel. The influence curve and its role in robust estimation. Journal of the American Statistical Association, 69:383-393, 1974.
22. F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel. Robust Statistics. The approach based on influence functions. Wiley, New York, 1986.
23. D. Hand, H. Mannila, and P. Smyth. Principles of Data Mining. MIT Press, Cambridge, Massachusetts, 2001.
24. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, New York, 2001.
25. J. Hipp, U. Güntzer, and U. Grimmer. Data quality mining - making a virtue of necessity. Workshop on Research Issues in Data Mining and Knowledge Discovery DMKD, Santa Barbara, CA, http://www.cs.cornell.edu/johannes/papers/dmkd2001-papers/p5-hipp.pdf, 2001.
26. K. U. Höffgen, H.-U. Simon, and K. S. Van Horn. Robust trainability of single neurons. Journal Computer and System Sciences, 50:114-125, 1995.
27. P. J. Huber. Robust estimation of a location parameter. Annals of Mathematical Statistics, 35:73-101, 1964.
28. P. J. Huber. Robust statistics. Wiley, New York, 1981.
29. T. Joachims. Making large-scale SVM learning practical. In B. Schölkopf, C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 41-56, MIT Press, Cambridge, Massachusetts, 1999.
30. J. Lindenstrauss and L. Tzafriri. Classicial Banach spaces I. Springer, Berlin, 1977.
31. G. K. Pedersen. Analysis Now. Springer, New York, 1989.
32. H. Rieder. Robust Asymptotic Statistics. Springer, New York, 1994.
33. P. J. Rousseeuw and M. Hubert. Regression depth. Journal of the American Statistical Association, 94:388-433, 1999.
34. B. Schölkopf and A. J. Smola. Learning with Kernels Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, Massachusetts, 2002.
35. J. C. Scovel and I. Steinwart. Fast rates for support vector machines. Los Alamos Technical Report LA-UR-03-9117, http://www.c3.lanl.gov/~ingo/publications/ann-03.ps, 2003.
36. I. Steinwart. On the influence of the kernel on the consistency of support vector machines. Journal of Machine Learning Research, 2:67-93, 2001.
37. I. Steinwart. Consistency of support vector machines and other regularized kernel machine. IEEE Transactions on Information Theory, to appear, 2002a.
38. I. Steinwart. Support vector machines are universally consistent. Journal of Complexity, 18:768-791, 2002b.
39. I. Steinwart. Sparseness of support vector machines. Journal of Machine Learning Research, 4:1071-1105, 2003.
40. I. Steinwart. Sparseness of support vector machines - some asymptotically sharp bounds. Proceedings of Neural Information Processing Systems, in press, 2004.
41. J. A. K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle. Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing, 48:85-105, 2002.
42. A. B. Tsybakov. Optimal aggregation of classifiers in statistical learning. Annals of Statistics, 32:135-166, 2004.
43. J. W. Tukey. Exploratory Data Analysis. Addison-Wesley, Reading, Massachusetts, 1977.
44. V. N. Vapnik. Statistical Learning Theory. Wiley, New York, 1998.
45. G. Wahba. Support vector machines, reproducing kernel Hilbert spaces and the randomized GACV. In B. Schölkopf, C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 69-88, MIT Press, Cambridge, Massachusetts, 1999.
46. th edition, 1974.
47. E. Zeidler. Nonlinear Functional Analysis and its Applications I. Springer, New York, 1986.
48. T. Zhang. Statistical behaviour and consistency of classification methods based on convex risk minimization. Annals of Statistics, 32:56-134, 2004.