A comparison of nonparametric error rate estimation methods in classification problems

Biometrical Journal

Volumn 46, Issue 1, 2004, Pages 35-47

  Wehberg, Sonja ^a   Schumacher, Martin ^b  

^a UNIVERSITY OF SOUTHERN DENMARK   (Denmark)

^b UNIVERSITY OF FREIBURG   (Germany)

Author keywords

.632+ estimator; Bootstrap; Efron's .632 estimator; Misclassification error rate; Prediction problem; Stability

Indexed keywords

COVARIANCE MATRIX; ERRORS; INTELLIGENT SYSTEMS; MEDICAL APPLICATIONS; MONTE CARLO METHODS; NORMAL DISTRIBUTION;

.632+ ESTIMATOR; BOOTSTRAP; EFRON 632 ESTIMATOR; ERROR RATE; MISCLASSIFICATION ERROR; MISCLASSIFICATION ERROR RATE; PERFORMANCE; PREDICTION PROBLEM; PREDICTION RULES; TRUE ERROR;

FORECASTING;

EID: 26844497970     PISSN: 03233847     EISSN: None     Source Type: Journal    
DOI: 10.1002/bimj.200410011     Document Type: Article

References (32)

1. Ambroise, C., and McLachlan, G. (2002). Selection bias in gene expression on the basis of microarray gene-expression data, Proceedings of the National Academy of Sciences, 99, 6562-6566.
2. Breiman, L. (1996). Heuristics in instability and stabilization in model selection, The Annals of Statistics, 24, 2350-2383.
3. Breiman, L., Friedman, J., Olshen, R., and Stone, C. (1984). Classification and Regression Trees, Wadsworth.
4. Chatterjee, S., and Chatterjee, S. (1983). Estimation of misclassification probabilities, Communications in Statistics: Simulation and Computation, 12, 645-656.
5. Chernick, M., Murthy, V., and Nealy, C. (1985). Application of bootstrap and other resampling methods: evaluation of classifier performance, Pattern Recognition Letters, 3, 167-178.
6. Chemick, M., Murthy, V., and Nealy, C. (1986). Correction note to application of bootstrap and other resampling methods: evaluation of classifier performance, Pattern Recognition Letters, 4, 133-142.
7. Chernick, M., Murthy, V., and Nealy, C. (1988). Estimation of error rate for linear discriminant functions by resampling: non-gaussian populations, Computers and Mathematics with Applications, 15, 29-37.
8. Davison, A., and Hinkley, D. (1997). Bootstrap Methods and Their Application, Cambridge University Press.
9. Davison, A., and Hall, P. (1992), On the bias and variability of bootstrap and crossvalidation estimates of error rates in discrimination problems, Biometrica, 79, 279-284.
10. Efron, B. (1982). The Jacknife, the Bootstrap, and Other Resampling Plans, SIAM, Society for Industrial and Applied Mathematics, Philadelphia.
11. Efron, B. (1983). Estimating the error rate of a prediction rule: some improvements on cross-validation, Journal of the American Statistical Association, 78, 316-331.
12. Efron, B., and Tibshirani, R. (1993). An Introduction to the Bootstrap, Chapman & Hall.
13. Efron, B., and Tibshirani, R. (1997). Improvements on cross-validation: the 632+ bootstrap method, Journal of the American Statistical Association, 92, 548-560.
14. Fitzmaurice, G., Krzanowski, W., and Hand, D. (1991). A Monte Carlo study of the 632 bootstrap estimator of error rate, Journal of Classification, 8, 239-250.
15. Glick, N. (1978). Additive estimators for probabilities of correct classification, Pattern Recognition, 10, 211-222.
16. Hand, D. (1986). Recent advances in error rate estimation, Pattern Recognition Letters, 4, 335-346.
17. Hand, D. (1997). Construction and Assessment of Classification Rules, John Wiley & Sons.
18. Hirst, D. (1996). Error-rate estimation in multiple-group linear discriminant analysis, Technometrics, 38, 389-399.
19. Jain, D., Dubes, R., and Chen, C. (1987). Bootstrap techniques for error estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9, 628-633.
20. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy assessment and model selection, Proceedings of the International Joint Conference on Artificial Intelligence, 1137-1143.
21. Krzanowski, W., and Hand, D. (1997). Assessing error rate estimators: the leave-one-out method reconsidered, Australian Journal of Statistics, 39, 35-46.
22. Mardia, K, Kent, J., and Bibby, J. (1979). Multivariate Analysis, Academic Press.
23. McLachlan, G. (1977). A note on the choice of a weighting function to give an efficient method for estimating the probability of misclassification, Pattern Recognition, 9, 147-149.
24. McLachlan, G. (1992). Discriminant Analysis and Statistical Pattern Recognition, John Wiley & Sons.
25. Merler, S., and Furlanello, C. (1997). Selection of tree-based classifiers with the bootstrap 632+ rule, Biometrical Journal, 39, 369-382.
26. Schiavo, R., and Hand, D. (2000). Ten more years of error rate research,. International Statistical Review, 68, 295-310.
27. Snapinn, S., and Knoke, J. (1988). Bootstrapped and smoothed classification error rate estimators, Communications in Statistics and Simulation, 17, 1135-1153.
28. Snapinn, S., and Knoke, J. (1989). Estimation of error rates in discriminant analysis with selection of variables, Biometrics, 45, 289-299.
29. Toussaint, G. (1974). Bibliography on estimation of misclassification, IEEE Transactions on Information Theory, 20, 472-479.
30. Toussaint, G., and Sharpe, P. (1975). An efficient method for estimating the probability of misclassification applied to a problem in medical diagnosis, Computers in Biology and Medicine, 4, 269-278.
31. Turney, P. (1995). Technical note: bias and the quantification of stability, Machine Learning, 20, 23-33.
32. Weiss, S. (1991). Small sample error rate estimation for k nearest neighbours classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13, 285-289.