On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers

Pattern Recognition

Volumn 42, Issue 11, 2009, Pages 2705-2723

  Zollanvari, Amin ^a   Braga Neto, Ulisses M ^a   Dougherty, Edward R ^a,b,c  

^a Texas A and M University   (United States)

^b TRANSLATIONAL GENOMICS RESEARCH INSTITUTE   (United States)

^c UNIVERSITY OF TEXAS MD ANDERSON CANCER CENTER   (United States)

Author keywords

Error estimation; Leave one out; Linear discriminant analysis; Parametric classification; Resubstitution; Sampling distribution

Indexed keywords

ERROR ESTIMATION; LEAVE-ONE-OUT; LINEAR DISCRIMINANT ANALYSIS; PARAMETRIC CLASSIFICATION; RESUBSTITUTION; SAMPLING DISTRIBUTION;

COVARIANCE MATRIX; DISCRIMINANT ANALYSIS; ERROR ANALYSIS; IMAGE RETRIEVAL;

ESTIMATION;

EID: 67649390743     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2009.05.003     Document Type: Article

References (42)

1. Hand D. Classifier technology and the illusion of progress. Stat. Sci. 21 (2006) 1-14
2. Attoor S., and Dougherty E. Classifier performance as a function of distributional complexity. Pattern Recognition 37 8 (2004) 1629-1640
3. Fisher R. The use of multiple measurements in taxonomic problems. Ann. Eugen. 7 (1936) 179-188
4. Wald A. On a statistical problem arising in the classification of an individual into one of two groups. Ann. Math. Stat. 15 (1944) 145-162
5. Anderson T. Classification by multivariate analysis. Psychometrika 16 (1951) 31-50
6. Devroye L., Gyorfi L., and Lugosi G. A Probabilistic Theory of Pattern Recognition (1996), Springer, New York
7. John S. Errors in discrimination. Ann. Math. Stat. 32 (1961) 1125-1144
8. Sitgreaves R. Some results on the distribution of the w-classification. In: Solomon H. (Ed). Studies in Item Analysis and Prediction (1961), Stanford University Press 241-251
9. 2 and Anderson's classification statistic w in terms of simple statistics. In: Solomon H. (Ed). Studies in Item Analysis and Prediction (1961), Stanford University Press 285-292
10. Bowker A., and Sitgreaves R. An asymptotic expansion for the distribution function of the w-classification statistic. In: Solomon H. (Ed). Studies in Item Analysis and Prediction (1961), Stanford University Press 292-310
11. Harter H. On the distribution of Wald's classification statistics. Ann. Math. Stat. 22 (1951) 58-67
12. Sitgreaves R. On the distribution of two random matrices used in classification procedures. Ann. Math. Stat. 23 (1951) 263-270
13. Teichroew D., and Sitgreaves R. Computation of an empirical sampling distribution for the w-classification statistic. In: Solomon H. (Ed). Studies in Item Analysis and Prediction (1961), Stanford University Press 285-292
14. Okamoto M. An asymptotic expansion for the distribution of the linear discriminant function. Ann. Math. Stat. 34 (1963) 1286-1301 (correction: Ann. Math. Stat. 39 (1968) 1358-1359)
15. Kabe D. Some results on the distribution of two random matrices used in classification procedures. Ann. Math. Stat. 34 (1963) 181-185
16. Hills M. Allocation rules and their error rates. J. R. Stat. Soc. Ser. B (Methodol.) 28 1 (1966) 1-31
17. Moran M. On the expectation of errors of allocation associated with a linear discriminant function. Biometrika 62 1 (1975) 141-148
18. McLachlan G. The bias of the apparent error in discriminant analysis. Biometrika 63 2 (1976) 239-244
19. S. Raudys, Comparison of the estimates of the probability of misclassification, in: Proceedings of the 4th International Conference on Pattern Recognition, Kyoto, Japan, 1978, pp. 280-282.
20. Foley D. Considerations of sample and feature size. IEEE Trans. Inf. Theory IT-18 5 (1972) 618-626
21. Genz A. Numerical computation of multivariate normal probabilities. J. Comput. Graphical Stat. 1 (1992) 141-149
22. Genz A., and Bretz F. Methods for the computation of multivariate t-probabilities. J. Stat. Comput. Simulation 11 (2002) 950-971
23. Dougherty E., and Braga-Neto U. Epistemology of computational biology: mathematical models and experimental prediction as the basis of their validity. J. Biol. Syst. 14 1 (2006) 65-90
24. Dougherty E., Hua J., and Bittner M. Validation of computational methods in genomics. Curr. Genomics 8 1 (2007) 1-19
25. Dougherty E. Small sample issues for microarray-based classification. Comp. Funct. Genomics 2 (2001) 28-34
26. T. Nagahata, M. Onda, M. Emi, H. Nagai, K. Tsumagari, T. Fujimoto, et al., Expression profiling to predict postoperative prognosis for estrogen receptor-negative breast cancers by analysis of 25,344 genes on a cdna microarray, Cancer Sci. 95 (3) (2004) 218-225.
27. S. Chiaretti, X. Li, R. Gentleman, A. Vitale, M. Vignetti, F. Mandelli, et al., Gene expression profile of adult t-cell acute lymphocytic leukemia identifies distinct subsets of patients with different response to therapy and survival, Blood 103 (7) (2004) 2771-2778.
28. De Smet F., Pochet N., Engelen K., Van Gorp T., Van Hummelen P., Marchal K., Amant F., Timmerman D., De Moor B., and Vergote I. Predicting the clinical behavior of ovarian cancer from gene expression profiles. Int. J. Gynecol. Cancer 16 Suppl. 1 (2006) 147-151
29. Somura H., Iizuka N., Tamesa T., Sakamoto K., Hamaguchi T., Tsunedomi R., Yamada-Okabe H., Sawamura M., Eramoto M., Miyamoto T., Hamamoto Y., and Oka M. A three-gene predictor for early intrahepatic recurrence of hepatocellular carcinoma after curative hepatectomy. Oncol. Rep. 19 2 (2008) 489-495
30. Shirahata M., Iwao-Koizumi K., Saito S., Ueno N., Oda M., Hashimoto N., Takahashi J., and Kato K. Gene expression-based molecular diagnostic system for malignant gliomas is superior to histological diagnosis. Clin. Cancer Res. 13 24 (2007) 7341-7356
31. Rimkus C., Friederichs J., Boulesteix A., Theisen J., Mages J., Becker K., Nekarda H., Rosenberg R., Janssen K., and Siewert J. Microarray-based prediction of tumor response to neoadjuvant radiochemotherapy of patients with locally advanced rectal cancer. Clin. Gastroenterol. Hepatol. 6 1 (2008) 53-61
32. Dupuy A., and Simon R. Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting. J. Natl. Cancer Inst. 99 2 (2007) 147-157
33. Gevaert O., De Smet F., Gorp T.V., Pochet N., Engelen K., Amant F., De Moor B., Timmerman D., and Vergote I. Expression profiling to predict the clinical behaviour of ovarian cancer fails independent evaluation. BMC Cancer 8 (2008) 18
34. Smith C. Some examples of discrimination. Ann. Eugen. 18 (1947) 272-282
35. Imhof J. Computing the distribution of quadratic forms in normal variables. Biometrika 48 3/4 (1961) 419-426
36. 2. Biometrika 46 (1959) 364
37. Lachenbruch P., and Mickey M. Estimation of error rates in discriminant analysis. Technometrics 10 (1968) 1-11
38. van de Vijver M., He Y., van't Veer L., Dai H., Hart A., Voskuil D., Schreiber G., Peterse J., Roberts C., Marton M., Parrish M., Astma D., Witteveen A., Glas A., Delahaye L., van der Velde T., Bartelink H., Rodenhuis S., Rutgers E., Friend S., and Bernards R. A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 347 25 (2002) 1999-2009
39. Bhattacharjee A., Richards W., Staunton J., Li C., Monti S., Vasa P., Ladd C., Beheshti J., Bueno R., Gillette M., Loda M., Weber E., Lander E., Wong W., Johnson B., Golub T., Sugarbaker D., and Meyerson M. Classification of human lung carcinomas by mrna expression profiling reveals distinct adenocarcinoma subclasses. Proc. Natl. Acad. Sci. USA 98 24 (2001) 13790-13795
40. Provost S., and Rudiuk E. The exact distribution of indefinite quadratic forms in noncentral normal vectors. Ann. Inst. Stat. Math. 48 2 (1996) 381-394
41. Price R. Some non-central f-distributions expressed in closed form. Biometrika 51 (1964) 107-122
42. K. Butler, M. Stephens, The distribution of a sum of binomial random variables, Technical Report ADA266969, Department of Statistics, Stanford University, Stanford, CA, URL: 〈http://handle.dtic.mil/100.2/ADA266969〉, 1993.