Adaptive confidence intervals for the test error in classification

Journal of the American Statistical Association

Volumn 106, Issue 495, 2011, Pages 904-913

  Laber, Eric B ^a   Murphy, Susan A ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Nonregular asymptotics; Pretesting

Indexed keywords

EID: 80054681541     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/jasa.2010.tm10053     Document Type: Article

References (22)

1. Andrews, D. W. K. (1999), "Testing When a Parameter Is on the Boundary of the Maintained Hypothesis, " Discussion Papers 1229, Cowles Foundation for Research in Economics, Yale University. [908]
2. Bartlett, P., Jordan, M., and McAuliffe, J. (2005), "Convexity, Classification, and Risk Bounds, " Journal of the American Statistical Association, 101, 138-156. [907]
3. Bickel, P. J., Klaassen, C. A. J., Ritov, Y., and Wellner, J. A. (1998), Efficient and Adaptive Estimation for Semiparametric Models, Amsterdam: Springer-Verlag. [905]
4. Cheng, X. (2008), "Robust Confidence Intervals in Nonlinear Regression Under Weak Identification, " Job Market paper, Yale. [908]
5. Chernick, M., Murthy, V., and Nealy, C. (1985), "Application of Bootstrap and Other Resampling Techniques: Evaluation of Classifier Performance, " Pattern Recognition Letters, 3, 167-178. [904]
6. Chung, H.-C., and Han, C.-P. (2009), "Conditional Confidence Intervals for Classification Error Rate, " Computational Statistics and Data Analysis, 53, 4358-4369. [904,905]
7. Efron, B. (1983), "Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation, " Journal of the American Statistical Association, 78, 316-331. [904]
8. Efron, B., and Tibshirani, R. (1995), "Cross-Validation and the Bootstrap: Estimating the Error Rate of a Prediction Rule, " Technical Report 172, Stanford. [904]
9. Efron, B., and Tibshirani, R. (1997), "Improvements on Cross-Validation: The.632+ Bootstrap Method, " Journal of the American Statistical Association, 92, 548-560. [904]
10. Haberman, S. (1989), "Concavity and Estimation, " The Annals of Statistics, 17, 1631-1661. [907]
11. Hastie, T., Tibshirani, R., and Friedman, J. (2009), The Elements of Statistical Learning. Springer Series in Statistics, New York: Springer. [904]
12. Isaksson, A., Wallman, M., Göransson, H., and Gustafsson, M. (2008), "Cross-Validation and Bootstrapping Are Unreliable in Small Sample Classification, " Pattern Recognition Letters, 29, 1960-1965. [904]
13. Jiang, W., Varma, S., and Simon, R. (2008), "Calculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling, " Statistical Applications in Genetics and Molecular Biology, 7. [904,910]
14. Kohavi, R. (1995), "A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection, " in IJCAI, Morgan Kaufmann, pp. 1137-1145. [904]
15. Krzanowski, W. J., and Hand, D. J. (1997), "Assessing Error Rate Estimators: The Leave-One-Out Method Reconsidered, " Australian Journal of Statistics, 39, 35-46. [904]
16. Laber, E. B., and Murphy, S. A. (2009), "Adaptive Confidence Intervals for the Test Error in Classification, " Technical Report 497, University of Michigan, Dept. of Statistics. [910,912]
17. Niemiro, W. (1992), "Asymptotics for M-Estimators Defined by Convex Minimization, " The Annals of Statistics, 20, 1514-1533. [907]
18. Schiavo, R. A., and Hand, D. (2000), "TenMore Years of Error Rate Research, " International Statistical Review, 68, 295-310. [904]
19. Shao, J. (1994), "Bootstrap Sample Size in Nonregular Cases, " Proceedings of the American Mathematical Society, 122, 1251-1262. [906]
20. Xie, M., Singh, K., and Zhang, C.-H. (2009), "Confidence Intervals for Population Ranks in the Presence of Ties and Near Ties, " Journal of the American Statistical Association, 104, 775-788. [908]
21. Yang, Y. (2006), "Comparing Learning Methods for Classification, " Statistica Sinica, 16, 635-657. [904,910]
22. Zhang, P. (1995), "Assessing Prediction Error in Non-Parametric Regression, " Scandinavian Journal of Statistics, 22, 83-94. [904]