Generalization bounds for the area under the ROC curve

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  Agarwal, Shivani ^a   Graepel, Thore ^b   Herbrich, Ralf ^b   Har Peled, Sariel ^a   Roth, Dan ^a  

^a University of Illinois at Urbana Champaign   (United States)

^b MICROSOFT RESEARCH   (United Kingdom)

Author keywords

Area under the ROC curve; Generalization bounds; Large deviations; Ranking; Uniform convergence

Indexed keywords

COMBINATORIAL MATHEMATICS; ERROR ANALYSIS; ERRORS; FUNCTIONS; PROBABILITY;

AREA UNDER THE ROC CURVE; GENERALIZATION BOUNDS; LARGE DEVIATIONS; RANKING; UNIFORM CONVERGENCE;

CONVERGENCE OF NUMERICAL METHODS;

EID: 21844440820     PISSN: 15337928     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (26)

1. Shivani Agarwal, Thore Graepel, Ralf Herbrich, and Dan Roth. A large deviation bound for the area under the ROC curve. In Advances in Neural Information Processing Systems 17. MIT Press, 2005a.
2. Shivani Agarwal, Sariel Har-Peled, and Dan Roth. A uniform convergence bound for the area under the ROC curve. In. Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, 2005b.
3. Martin Anthony and Peter Bartlett. Learning in Neural Networks: Theoretical Foundations. Cambridge University Press, 1999.
4. Z. W. Birnbaum and O. M. Klose. Bounds for the variance of the Mann-Whitney statistic. Annals of Mathematical Statistics, 38, 1957.
5. R. C. Buck. Partition of space. American Mathematical Monthly, 50:2541-544, 1943.
6. William W. Cohen, Robert E. Schapire, and Yoram Singer. Learning to order things. Journal of Artificial Intelligence Research, 10:243-270, 1999.
7. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms. MIT Press, second edition, 2001.
8. Corinna Cortes and Mehryar Mohri. AUC optimization vs. error rate minimization. In Sebastian Thrun, Lawrence Saul, and Bernhard Schölkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, 2004.
9. Corinna Cortes and Mehryar Mohri. Confidence intervals for the area under the ROC curve. In Advances in Neural Information Processing Systems 17. MIT Press, 2005.
10. Koby Crammer and Yoram Singer. Pranking with ranking. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 641-647. MIT Press, 2002.
11. D. Van Dantzig. On the consistency and power of Wilcoxon's two sample test. In Koninklijke Nederlandse Akademie van Weterschappen, Series A, volume 54, 1915.
12. Luc Devroye. Exponential inequalities in nonparametric estimation. In G. Roussas, editor, Non-parametric Functional Estimation and Related Topics, NATO ASI Series, pages 31-44. Kluwer Academic Publishers, 1991.
13. Luc Devroye, László Györfi, and Gábor Lugosi. A Probabilistic Theory of Pattern Recognition. Springer-Verlag, New York, 1996.
14. Bradley Efron and Robert Tibshirani. An Introduction to the Bootstrap. Chapman and Hall, 1993.
15. James P. Egan. Signal Detection Theory and ROC Analysis. Academic Press, 1975.
16. Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003.
17. Thore Graepel, Ralf Herbrich, and John Shawe-Taylor. PAC-Bayesian compression bounds on the prediction error of learning algorithms for classification. Machine Learning, 2005. To appear.
18. James A. Hanley and Barbara J. McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143:29-36, 1982.
19. Ralf Herbrich, Thore Graepel, and Klaus Obermayer. Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers, pages 115-132, 2000.
20. Simon I. Hill, Hugo Zaragoza, Ralf Herbrich, and Peter J. W. Rayner. Average precision and the problem of generalisation. In Proceedings of the ACM SIGIR Workshop on Mathematical and Formal Methods in Information Retrieval, 2002.
21. Erich L. Lehmann. Nonparametrics: Statistical Methods Based on Ranks. Holden-Day, San Francisco, California, 1975.
22. Nick Littlestone and Manfred Warmuth. Relating data compression and learnability. Technical report, University of California Santa Cruz, 1986.
23. Jiři Matoušek. Lectures on Discrete Geometry. Springer-Verlag, New York, 2002.
24. Colin McDiarmid. On the method of bounded differences. In Surveys in Combinatorics 1989, pages 148-188. Cambridge University Press, 1989.
25. Saharon Rosset. Model selection via the AUG. In Proceedings of the 21st International Conference on Machine Learning, 2004.
26. Vladimir N. Vapnik. Estimation of Dependences Based on Empirical Data. Springer-Verlag, New York, 1982.