Analytic study of performance of error estimators for linear discriminant analysis

IEEE Transactions on Signal Processing

Volumn 59, Issue 9, 2011, Pages 4238-4255

  Zollanvari, Amin ^a,b,c   Braga Neto, Ulisses M ^d   Dougherty, Edward R ^d,e,f  

^a HARVARD MIT DIVISION OF HEALTH SCIENCES AND TECHNOLOGY   (United States)

^b HARVARD MEDICAL SCHOOL   (United States)

^c BRIGHAM AND WOMEN S HOSPITAL   (United States)

^d Department of Electrical and Computer Engineering   (United States)

^e TRANSLATIONAL GENOMICS RESEARCH INSTITUTE   (United States)

^f UNIVERSITY OF TEXAS MD ANDERSON CANCER CENTER   (United States)

Author keywords

Double asymptotics; error estimation; genomic signal processing; leave one out; linear discriminant analysis; resubstitution; root mean square (RMS)

Indexed keywords

ASYMPTOTICS; ERROR ESTIMATIONS; GENOMIC SIGNAL PROCESSING; LEAVE-ONE-OUT; LINEAR DISCRIMINANT ANALYSIS; RESUBSTITUTION; ROOT MEAN SQUARE;

COVARIANCE MATRIX; DISCRIMINANT ANALYSIS; GENE EXPRESSION; SAMPLING; SIGNAL PROCESSING;

ESTIMATION;

EID: 79960987402     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2011.2159210     Document Type: Article

References (51)

1. R. Fisher, "The use of multiple measurements in taxonomic problems," Ann. Eugen., vol. 7, pp. 179-188, 1936.
2. R. Fisher, "The precision of discriminant function," Ann. Eugen., vol. 10, pp. 422-429, 1940.
3. A. Wald, "On a statistical problem arising in the classification of an individual into one of two groups," Ann. Math. Statist., vol. 15, pp. 145-162, 1944.
4. T. Anderson, "Classification by multivariate analysis," Psychometrika, vol. 16, pp. 31-50, 1951.
5. G. T. Toussaint, "Bibliography on estimation of misclassification, " IEEE Trans. Inf. Theory, vol. 20, pp. 472-479, 1974.
6. R. A. Schiavo and D. J. Hand, "Ten more years of error rate research," Int. Statist. Rev., vol. 68, no. 3, pp. 295-310, 2000. (Pubitemid 33207232)
7. M. Hills, "Allocation rules and their error rates," J. Royal Statist. Soc. Ser. B (Methodological), vol. 28, no. 1, pp. 1-31, 1966.
8. G. J. McLachlan, "An asymptotic expansion of the expectation of the estimated error rate in discriminant analysis," Austral. J. Statist., vol. 15, no. 3, pp. 210-214, 1973.
9. N. Glick, "Additive estimators for probabilities of correct classification," Pattern Recogn., vol. 10, pp. 211-222, 1978. (Pubitemid 9029903)
10. K. Fukunaga and R. R. Hayes, "Estimation of classifier performance," IEEE Trans. Pattern Anal. Mach. Intell. vol. 11, pp. 1087-1101, 1989. (Pubitemid 20628923)
11. U. Braga-Neto and E. Dougherty, "Is cross-validation valid for microarray classification?," Bioinformatics, vol. 20, no. 3, pp. 374-380, 2004.
12. S. Raudys, Statistical and Neural Classifiers, An Integrated Approach to Design. London, U.K.: Springer-Verlag, 2001.
13. L. Devroye and T. Wagner, "Distribution-free inequalities for the deleted and hold-out error estimates," IEEE Trans. Inf. Theory, vol. 25, pp. 202-207, 1979. (Pubitemid 9443301)
14. L. Devroye and T.Wagner, "Distribution-free performance bounds for potential function rules," IEEE Trans. Inf. Theory, vol. 25, pp. 601-604, 1979. (Pubitemid 10426456)
15. D. Foley, "Considerations of sample and feature size," IEEE Trans. Inf. Theory, vol. IT-18, pp. 618-626, 1972.
16. A. Zollanvari, U. Braga-Neto, and E. Dougherty, "Joint sampling distribution between actual and estimated classification errors for linear discriminant analysis," IEEE Trans. Inf. Theory, vol. 56, no. 2, pp. 784-804, 2010.
17. V. Serdobolskii, Multivariate Statistical Analysis: A High-Dimensional Approach. New York: Springer, 2000.
18. S. Raudys, "On the amount of a priori information in designing the classification algorithm," Tech. Cybern., vol. 4, pp. 168-174, 1972, in Russian.
19. A. Deev, "Representation of statistics of discriminant analysis and asymptotic expansion when space dimensions are comparable with sample size," Dokl. Akad. Nauk SSSR, vol. 195, pp. 759-762, 1970, in Russian.
20. A. Deev, "Asymptotic expansions for distributions of statistics w, m, and w in discriminant analysis," Statist. Methods of Classific., vol. 31, pp. 6-57, 1972, in Russian.
21. S. Raudys, "Comparison of the estimates of the probability of misclassification," in Proc. Int. Joint Conf. Pattern Recogn., 1978, pp. 280-282.
22. E. Dougherty and U. Braga-Neto, "Epistemology of computational biology: Mathematical models and experimental prediction as the basis of their validity," J. Biolog. Syst., vol. 14, no. 1, pp. 65-90, 2006. (Pubitemid 43289707)
23. I. Shmulevich and E. R. Dougherty, Genomic Signal Processing. Princeton, NJ: Princeton Univ. Press, 2007.
24. K. Umapathy, S. Krishnan, and R. K. Rao, "Audio signal feature extraction and classification using local discriminant bases," IEEE Trans. Audio, Speech, Language Process., vol. 15, pp. 1236-1246, 2007.
25. D. Swets and J.Weng, "Using discriminant eigenfeatures for image retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, pp. 891-896, 1996.
26. W. Zheng, "Heteroscedastic feature extraction for texture classification," IEEE Signal Process. Lett., vol. 16, pp. 766-769, 2009.
27. S. Kim, E. R. Dougherty, I. Shmulevich, K. R. Hess, S. R. Hamilton, J. M. Trent, G. N. Fuller, and W. Zhang, "Identification of combination gene sets for glioma classification," Molecular Cancer Therapeut., vol. 1, pp. 1229-1236, 2002.
28. H. Somura, N. Iizuka, T. Tamesa, K. Sakamoto, T. Hamaguchi, R. Tsunedomi, H. Yamada-Okabe, M. Sawamura, M. Eramoto, T. Miyamoto, Y. Hamamoto, and M. Oka, "A three-gene predictor for early intrahepatic recurrence of hepatocellular carcinoma after curative hepatectomy," Oncol. Rep., vol. 19, pp. 489-495, 2008. (Pubitemid 351176112)
29. T. V. Bandos, L. Bruzzone, and G. Camps-Valls, "Finger-based personal authentication: A comparison of feature-extraction methods based on principal component analysis, most discriminant features and regularized-direct linear discriminant analysis," IEEE Trans. Geosci. Remote Sens., vol. 47, pp. 862-873, 2009.
30. S. John, "Errors in discrimination," Ann. Math. Statist., vol. 32, pp. 1125-1144, 1961.
31. M. J. Sorum, "Estimating the conditional probability of misclassification," Technometr., vol. 13, pp. 333-343, 1971.
32. M. Moran, "On the expectation of errors of allocation associated with a linear discriminant function," Biometrika, vol. 62, no. 1, pp. 141-148, 1975.
33. C. Smith, "Some examples of discrimination," Ann. Eugenics, vol. 18, pp. 272-282, 1947.
34. P. Lachenbruch and M. Mickey, "Estimation of error rates in discriminant analysis," Technometrics, vol. 10, pp. 1-11, 1968.
35. L. Devroye, L. Gyorfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer, 1996.
36. S. Raudys and D. M. Young, "Results in statistical discriminant analysis: Areviewof the former Soviet Union literature," J. Multivar. Anal., vol. 89, pp. 1-35, 2004. (Pubitemid 38284308)
37. L. D. Meshalkin and V. I. Serdobolskii, "Errors in the classification of multi-variate observations," Theory Probabil. Appl., vol. 23, pp. 741-750, 1978.
38. S. Raudys and V. Pikelis, "On dimensionality, sample size, classification error, and complexity of classification algorithm in pattern recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 2, pp. 242-252, 1980. (Pubitemid 10485853)
39. S. Raudys, "On dimensionality, sample size and classification error of nonparametric linear classification algorithms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, pp. 667-671, 1997. (Pubitemid 127762260)
40. H. S. D. Haussler,M.Kearns, and N. Tishby, "Rigorous learning curves from statistical mechanics," in Proc. 7th Ann. ACM Conf. Comput. Learn. Theory, San Mateo, CA, 1994, Morgan Kaufman.
41. F. Wyman, D. Young, and D. Turner, "A comparison of asymptotic error rate expansions for the sample linear discriminant function," Pattern Recogn., vol. 23, no. 7, pp. 775-783, 1990.
42. V. Pikelis, "Comparison of methods of computing the expected classification errors," Automat. Remote Contr., vol. 5, no. 7, pp. 59-63, 1976.
43. S. Raudys, "On determining training sample size of a linear classifier," Comput. Syst., vol. 28, pp. 79-87, 1967, in Russian.
44. A. Jain and W.Waller, "On the optimal number of features in the classification of multivariate Gaussian data," Pattern Recognit., vol. 10, pp. 365-374, 1978. (Pubitemid 9091815)
45. J. Hua, Z. Xiong, J. Lowey, E. Suh, and E. R. Dougherty, "Optimal number of features as a function of sample size for various classification rules," Bioinformat., vol. 21, no. 8, pp. 1509-1515, 2005. (Pubitemid 40552752)
46. P. Billingsley, Probability and Measure, 3rd ed. New York: Wiley, 1995.
47. Y. H. M. van de Vijver et al., "A gene-expression signature as a predictor of survival in breast cancer," New England J. Med., vol. 347, pp. 1999-2009, 2002.
48. O. J. Dunn, "Some expected values for probabilities of correct classification in discriminant analysis," Technometr., vol. 13, pp. 345-353, 1971.
49. R. Kan, "From moments of sum to moments of product," J. Multivar. Anal., vol. 99, pp. 542-554, 2008.
50. P. Sen and J. Singer, Large Sample Methods in Statistics. New York: Chapman and Hall, 1993.
51. T. W. Anderson, Introduction to Multivariate Statistical Analysis. New York: Wiley, 1958.