Test error bounds for classifiers: A survey of old and new results

IEEE SSCI 2011 - Symposium Series on Computational Intelligence - FOCI 2011: 2011 IEEE Symposium on Foundations of Computational Intelligence

Volumn , Issue , 2011, Pages 80-87

  Anguita, Davide ^a   Ghelardoni, Luca ^a   Ghio, Alessandro ^a   Ridella, Sandro ^a  

^a UNIVERSITY OF GENOA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

GENERALIZATION ERROR; NEW RESULTS; TEST ERRORS; TEST SETS;

ERROR ANALYSIS; PATTERN RECOGNITION; SURVEYS;

ARTIFICIAL INTELLIGENCE;

EID: 79961151398     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FOCI.2011.5949469     Document Type: Conference Paper

References (26)

1. C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.
2. C. Cortes, V. Vapnik, "Support-Vector Networks", Machine Learning, Vol. 20, pp. 273-297, 1995.
3. L. Roberge, S. B. Long, D. B. Burnham, "Data Warehouses and Data Mining tools for the legal profession: using information technology to raise the standard of practice", Syracuse Law Review, vol. 52, pp. 1281- 1292, 2002.
4. P. Bartlett, S. Boucheron, G. Lugosi, "Model Selection and Error Estimation", Machine Learning, Vol. 48, pp. 85-113, 2002. (Pubitemid 34247574)
5. L. Devroye, L. Gyorfi, G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer Verlag, 1996.
6. V.N. Vapnik, A.Y. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities", Theory of Probability and its Applications, Vol. 16, pp. 264-280, 1971.
7. O. Gascuel, G. Caraux, "Distribution-free performance bounds with the resubstitution error estimate", Pattern Recognition Letters, Vol. 13, pp. 757-764, 1992.
8. C.J.C. Burges, "A tutorial on Support Vector Machines for classification", Data Mining and Knowledge Discovery, vol. 2, pp. 121-167, 1998. (Pubitemid 128695475)
9. C.-W. Su, C.-C. Chang, C.-J. Lin, "A practical guide to Support Vector classification", Technical report, Dept. of Computer Science, National Taiwan University, 2003.
10. D. Anguita, A. Boni, S. Ridella, F. Rivieccio, D. Sterpi, "Theoretical and practical model selection methods for Support Vector classifiers", in "Support Vector Machines: Theory and Applications", edited by L. Wang, Springer, 2005.
11. D. Anguita, A. Ghio, S. Ridella, D. Sterpi, "K-Fold Cross Validation for Error Rate Estimate in Support Vector Machines", Proc. of the Int. Conf. on Data Mining (DMIN'09), pp. 291-297, 2009.
12. M. Anthony, S. B. Holden, "Cross-Validation for binary classification by real-valued functions: theoretical analysis", Proc. of the 11th Conf. on Computational Learning Theory, pp. 218-229, 1998.
13. J.C. Platt, "Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods", Advances in Large Margin Classifiers, pp. 61-74, 1999.
14. P. Bartlett, A. Tewari, "Sparseness vs Estimating Conditional Probabilities: Some Asymptotic Results", Journal of Machine Learning Research, Vol. 8, pp. 775-790, 2007. (Pubitemid 46677047)
15. L.D. Brown, T.T. Cai, A. DasGupta, "Interval estimation for a binomial proportion", Statistical Science, Vol. 16, pp. 101-133, 2001. (Pubitemid 33636344)
16. E.B. Wilson, "Probable inference, the law of succession, and statistical inference", Journal of the American Statistical Association, Vol. 22, pp. 209-212, 1927.
17. C.J. Clopper, E.S. Pearson, "The use of confidence intervals for fiducial limits illustrated in the case of the binomial", Biometrika, Vol. 26, pp. 404-413, 1934.
18. J. Langford, "Tutorial on practical prediction theory for classification", Journal of Machine Learning Research, Vol. 6, pp. 273-306, 2005.
19. A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd ed., McGraw-Hill, 1991.
20. L. Guttman, "A distribution-free confidence interval for the mean", The Annals of Mathematical Statistics, Vol. 19, pp. 410-413, 1948.
21. S.N. Bernstein, Sobranie sochinenie, Tom IV. Teoriya veroyatnostei, Matematicheskaya statistika (1911-1946) (Collected works, Vol. IV. The theory of probability, Mathematical statistics (1911-1946)), Izdat. Nauka, 1964.
22. J.Y. Audibert, R. Munos, C. Szepesvari, "Exploration-exploitation tradeoff using variance estimates in multi-armed bandits", Theoretical Computer Science, Vol. 410, pp. 1876-1902, 2009.
23. A. Maurer, M. Pontil, "Empirical Bernstein Bounds and Sample Variance Penalization", Proc. of the Int. Conference on Learning Theory (COLT), 2009.
24. W. Hoeffding, "Probability inequality for sum of bounded random variables", Journal of the American Statistical Association, Vol. 58, pp. 13-30, 1963.
25. V. Bentkus, "An Inequality for Large Deviation Probabilities of Sums of Bounded i.i.d. Random Variables", Lithuanian Mathematical Journal, Vol. 41, pp. 112-119, 2001. (Pubitemid 33815240)
26. V. Bentkus, "On Hoeffdings inequalities", Annals of Probability, Vol. 32, pp. 1650-1673, 2004.