Algorithmic stability and sanity-check bounds for leave-one-out cross-validation

Neural Computation

Volumn 11, Issue 6, 1999, Pages 1427-1453

  Kearns, Michael ^a   Ron, Dana ^b  

^a AT AND T LABS RESEARCH   (United States)

^b TEL AVIV UNIVERSITY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK;

ALGORITHMS; NEURAL NETWORKS (COMPUTER);

EID: 0033566418     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976699300016304     Document Type: Article

References (17)

1. Devroye, L., Gyöyfi, L., & Lugosi, G. (1996). A probabilistic theory of pattern recognition. Berlin: Springer-Verlag.
2. Devroye, L. P., & Wagner, T. J. (1979a). Distribution-free inequalities for the deleted and holdout error estimates. IEEE Transactions on Information Theory, IT-25(2), 202-207.
3. Devroye, L. P., & Wagner, T. J. (1979b). Distribution-free performance bounds for potential function rules. IEEE Transactions on Information Theory, IT-25(5), 601-604.
4. Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: Wiley.
5. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741.
6. Haussler, D. (1992). Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation, 100(1), 78-150.
7. Haussler, D., Kearns, M., Seung, H. S., & Tishby, N. (1996). Rigorous learning curve bounds from statistical mechanics. Machine Learning, 25, 195-236.
8. Holden, S. B. (1996a). Cross-validation and the PAC learning model (Research Note RN/96/64). London: University College, London.
9. Holden, S. B. (1996b). PAC-like upper bounds for the sample complexity of leave-one-out cross validation. In Proceedings of the Ninth Annual ACM Workshop on Computational Learning Theory (pp. 41-50).
10. Kearns, M. (1996). A bound on the error of cross validation, with consequences for the training-test split. D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 183-189). Cambridge, MA: MIT Press.
11. Kearns, M. J., Mansour, Y., Ng, A., & Ron, D. (1995). An experimental and theoretical comparison of model selection methods. In Proceedings of the Eighth Annual ACM Workshop on Computational Learning Theory (pp. 21-30).
12. Kearns, M., Schapire, R., & Sellie, L. (1994). Toward efficient agnostic learning. Machine Learning, 17, 115-141.
13. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In International Joint Conference on Artificial Intelligence.
14. Miller, A. J. (1990). Subset selection in regression. London: Chapman and Hall.
15. Rogers, W. H., & Wagner, T. J. (1978). A finite sample distribution-free performance bound for local discrimination rules. Annals of Statistics, 6(3), 506-514.
16. Seung, H. S., Sompolinsky, H., & Tishby, N. (1992). Statistical mechanics of learning from examples. Physical Review, A45, 6056-6091.
17. Vapnik, V. N. (1982). Estimation of dependences based on empirical data. New York: Springer-Verlag.