On the dangers of cross-validation. An experimental evaluation

Society for Industrial and Applied Mathematics - 8th SIAM International Conference on Data Mining 2008, Proceedings in Applied Mathematics 130

Volumn 2, Issue , 2008, Pages 588-596

  Rao, R Bharat ^a   Fung, Glenn ^a   Rosales, Romer ^a  

^a IKM CKS Siemens Medical Solutions   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ESTIMATION; LEARNING ALGORITHMS; RISK PERCEPTION;

CLASSIFICATION ALGORITHM; CLOSED FORM SOLUTIONS; COMPUTATIONAL POWER; CROSS VALIDATION; DATA DIMENSIONALITY; EXPERIMENTAL EVALUATION; LEARNING MODELS; MODEL SELECTION;

DATA MINING;

EID: 52649131369     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1137/1.9781611972788.54     Document Type: Conference Paper

References (28)

1. Breiman, L., &, Spector, P. (1992), Submodel selection and evaluation in regression: The x-random case, International Statistical Review, 60, 291-319.
2. Cawley, G. C., & Talbot, N. L. C. (2003). Efficient leave-one-out cross-validation of kernel fisher discriminant classifiers. citeseer.ist.psu. edu/cawley03efficieiit.litnil.
3. Efron, B., & Tibshirani, R. (1993). An introduction to the bootstrap. London, UK: Chapman and Hall.
4. Fung, G., Dundar, M., Krishnapuram, B., & Rao, R. B. (2006), Multiple instance learning for computer aided diagnosis. NIPS 2006: Advances in Neural Information Processing Systems.
5. Fung, G., & Mangasarian, O. L. (2001). Proximal support vector machine classifiers. Proceedings KDD-2001: Knowledge Discovery and Data Mining, August 26-29, 2001, San Francisco, CA (pp. 77-86). New York: Asscociation for Computing Machinery. ftp://ftp.cs.wisc.edu/pub/dmi/tech- reports/01-02.ps.
6. Golub, G. H., & Loan, C. F. V. (1996). Matrix computations. Baltimore: The Johns Hopkins University Press.
7. Joachims, T. (2006). Training linear svms in linear time. KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 217-226). New York, NY, USA: ACM Press.
8. Johnson, R. A., & Wichern, D. W. (2002). Applied multivariate statistical analysis. New Jercey: Prentice Hall.
9. Kearns, M. J., & Ron, D. (1997). Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Computational Learing Theory (pp. 152-162).
10. Kohavi, R. (1995a). A study of cross-validation and bootstrap for accuracy estimation and model selection. IJCAI (pp. 1137-1145).
11. Kohavi, R. (1995b). A study of cross-validation and bootstrap for accuracy estimation and model selection. International Joint Conference on Artificial Intelligence IJCAI (pp. 1137-1145).
12. Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273-324.
13. Kohavi, R., &, Wolpert, D. H. (1996). Bias plus variance decomposition for zero-one loss functions. Machine Learning: Proceedings of the Thirteenth International Conference (pp. 275-283). Morgan Kaufmann.
14. Mika, S., Rätsch, G., Weston, J., Schölkopf, B., & Müller, K.-R. (1999). Fisher discriminant analysis with kernels. Neural Networks for Signal Processing IX (pp. 41-48). IEEE.
15. Newman, C. B. D., & Merz, C. (1998). UCI repository of machine learning databases.
16. Ng, A. Y. (1997). Preventing overfitting of cross-validation data. Proc. 14th International Conference on Machine Learning (pp. 245-253). Morgan Kaufmann.
17. Ng, A. Y. (1998). On feature selection: learning with exponentially many irrelevant features as training examples. Proc. 15th International Conf. on Machine Learning (pp. 404-412). Morgan Kaufmann, San Francisco, CA.
18. Raykar, V. C., & Duraiswarni, R. (2005). The improved fast gauss transform with applications to machine learning. NIPS 2005 workshop on Large scale kernel machines.
19. Reunanen, J. (2003). Overfitting in making comparisons between variable selection methods. Journal of Machine Learning Research, 3, 1371-1382.
20. Reunanen, J. (2004). A pitfall in determining the optimal feature subset size. Proc. of the 4th Int. Workshop on Pattern Recognition in Information Systems (PRIS 2004) (pp. 176-185). Porto, Portugal.
21. Saunders, G., Garnmerman, A., & Vovk, V. (1998). Ridge regression learning algorithm in dual variables. Proc. 15th International Conf. on Machine Learning (pp. 515-521). Morgan Kaufmann, San Francisco, CA.
22. Stone, M. (1977). Asymptotics for and against cross-validation. Biometrika, 64, 29-35.
23. Suykens, J., & Vandewalle, J. (1999). Least squares support vector machine classifiers. Neural Processing Letters, 9, 293-300.
24. Vapnik, V. N. (1982). Estimation of dependencies based on empirical data. New York: Springer-Verlag.
25. Vapnik, V. N. (1998). Statistical learning theory. New York: John Wiley &, Sons, Inc.
26. Weiss, S. M., & Kulikowski, C. A. (1991). Computer systems that learn. Morgan Kaufmann.
27. Xu, J., Zhang, X., & Li, Y. (2001). Kernel MSE algorithm: a unified framework for KFD, LS-SVM and KRR. Proc. of IJCNN-01 (pp. 1486-1491).
28. Zhang, T. (2003). Leave-one-out bounds for kernel methods. Neural Computation, 15, 1397-1437.