The illusion of distribution-free small-sample classification in genomics

Current Genomics

Volumn 12, Issue 5, 2011, Pages 333-341

  Dougherty, Edward R ^a,b,c   Zollanvari, Amin ^d   Braga Neto, Ulisses M ^a  

^a Department of Electrical and Computer Engineering   (United States)

^b TRANSLATIONAL GENOMICS RESEARCH INSTITUTE   (United States)

^c UNIVERSITY OF TEXAS MD ANDERSON CANCER CENTER   (United States)

^d HARVARD MEDICAL SCHOOL   (United States)

Author keywords

Classification; Epistemology; Error estimation; Genomics; Validation

Indexed keywords

ACCURACY; ANALYSIS OF VARIANCE; ARTICLE; BAYES THEOREM; BIOINFORMATICS; EPISTEMOLOGY; GENOMICS; HIGH THROUGHPUT SCREENING; PHENOTYPE; ROOT MEAN SQUARE; SAMPLE SIZE; SAMPLING ERROR; STATISTICAL DISTRIBUTION; STATISTICAL PARAMETERS; UNCERTAINTY; VALIDATION PROCESS;

EID: 79961007365     PISSN: 13892029     EISSN: 18755488     Source Type: Journal    
DOI: 10.2174/138920211796429763     Document Type: Article

References (25)

1. Dougherty, E. R.; Braga-Neto, U. M. Epistemology of computational biology: mathematical models and experimental prediction as the basis of their validity. J. Biol. Syst., 2006, 14, 65-90.
2. Devroye, L.; Wagner, T. Distribution-free performance bounds for the deleted and hold-out error estimates. IEEE Trans. Inform. Theory, 1979, 25, 202-207.
3. Devroye, L.; Gyorfi, L.; Lugosi, G. A probabilistic theory of pattern recognition. Springer, New York, 1996.
4. Zollanvari, A.; Braga-Neto, U. M.; Dougherty, E. R. On the joint sampling distribution between the actual classification error and the resubstitution and leave-one-out error estimators for linear classifiers. IEEE Trans. Inform. Theory, 2010, 56, 784-804.
5. Zollanvari, A.; Braga-Neto, U. M.; Dougherty, E. R. Analytic study of performance of error estimators for linear discriminant analysis. IEEE Trans. Sig. Proc., 2011, doi 0.1109/TSP.2011.2159210.
6. Wyman, F.; Young, D.; Turner, D. A comparison of asymptotic error rate expansions for the sample linear discriminant function. Pattern Recognit., 1990, 23, 775-783.
7. Pikelis, V. Comparison of methods of computing the expected classification errors. Automat. Remote Cont., 1976, 5, 59-63.
8. Zipf, G. K. Psycho-Biology of Languages, Houghton-Mifflin, Boston, 1935.
9. Braga-Neto, U. M.; Dougherty, E. R. Exact performance of error estimators for discrete classifiers. Pattern Recognit., 2005, 38, 1799-1814.
10. Xu, Q.; Hua, J.; Braga-Neto, U. M.; Xiong, Z.; Suh, E.; Dougherty, E. R. Confidence intervals for the true classification error conditioned on the estimated error. Tech. Cancer Res. Treat., 2006, 5, 579-590.
11. Braga-Neto, U. M.; Dougherty, E. R. Exact correlation between actual and estimated errors in discrete classification. Pattern Recognit. Lett., 2010, 31, 407-413.
12. Glick, N. Additive estimators for probabilities of correct classification. Pattern Recognit., 1978, 10, 211-222.
13. Braga-Neto, U. M.; Dougherty, E. R. Is cross-validation valid for small-sample microarray classification. Bioinformatics, 2004, 20, 374-380.
14. Hanczar, B.; Hua, J.; Dougherty, E. R. Decorrelation of the true and estimated classifier errors in high-dimensional settings. EURASIP J. Bioinf. Syst. Biol., 2007, Article ID 38473, 12 pages.
15. Hanczar, B.; Dougherty, E. R. On the comparison of classification algorithms for microarray data. Curr. Bioinf., 2010, 5, 29-39.
16. Rocke, D. M.; Idecker, T.; Troyanskaya, O.; Quackenbush, J.; Dopazo, J. Papers on normalization, variable selection, classification or clustering of microarray data. Bioinformatics, 2009, 25, 701-702.
17. Mehta, T.; Murat, T.; Allison, D. B. Towards sound epistemological foundations of statistical methods for high-dimensional biology. Nat. Genet., 2004, 36, 943-947.
18. Dougherty, E. R. On the epistemological crisis in genomics. Curr. Genomics, 2008, 9, 69-79.
19. Boulesteix, A-L. Over-optimism in bioinformatics research. Bioin-formatics, 2010, 26, 437-439.
20. Jelizarow, M.; Guillemot, V.; Tenenhaus, A.; Strimmer, K.; Boul-esteix, A-L. Over-optimism in bioinformatics: an illustration. Bio-informatics, 2010, 26, 1990-1998.
21. Braga-Neto, U. M. Fads and fallacies in the name of small-sample microarray classification. IEEE Sig. Proc. Mag., 2007, 24, 91-99.
22. Boulesteix, A-L.; Strobl, C. Optimal classifier selection and negative bias in error rate estimation: an empirical study on high-dimensional prediction. BMC Med. Res. Meth., 2009, 9, 85.
23. Yousefi, M. R.; Hua, J.; Sima, C.; Dougherty, E. R. Reporting bias when using real data sets to analyze classification performance. Bioinformatics, 2010, 26, 68-76.
24. Yousefi, M. R.; Hua, J.; Dougherty, E. R. Multiple-rule bias in the comparison of classification rules. Bioinformatics, 2011, 27, 1675-1683.
25. Fisher, R. A. Statistical Methods for Research Workers, Oliver and Boyd, Edinburgh 1925.