Random KNN feature selection - a fast and stable alternative to Random Forests

BMC Bioinformatics

Volumn 12, Issue 1, 2011, Pages

  Li, Shengqiao ^a,b   Harner, E James ^a   Adjeroh, Donald A ^c  

^a WEST VIRGINIA UNIVERSITY   (United States)

^b NATIONAL INSTITUTE FOR OCCUPATIONAL SAFETY AND HEALTH   (United States)

^c West Virginia University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION ACCURACY; CLASSIFICATION PERFORMANCE; FEATURE SELECTION METHODS; GENE EXPRESSION PROFILING; HIGH DIMENSIONAL DATA; K-NEAREST NEIGHBORS; MICROARRAY DATA SETS; SELECTION PROCEDURES;

CLASSIFICATION (OF INFORMATION); GENE EXPRESSION;

DECISION TREES;

ARTICLE; BIOLOGICAL MODEL; CLUSTER ANALYSIS; DNA MICROARRAY; GENE EXPRESSION PROFILING; GENETICS; HUMAN; NEOPLASM;

CLUSTER ANALYSIS; GENE EXPRESSION PROFILING; HUMANS; MODELS, GENETIC; NEOPLASMS; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS;

EID: 81255178681     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/1471-2105-12-450     Document Type: Article

References (36)

1. Theodoridis S, Koutroumbas K. Pattern recognition. 2003, Academic Press.
2. Duda RO, Hart PE, Stork DG. Pattern Classification. 2000, New York: John Wiley & Sons.
3. Saeys Y, Inza I, Larrañaga P. A review of feature selection techniques in bioinformatics. Bioinformatics 2007, 23(19).
4. Breiman L, Friedman J, Stone CJ, Olshen R. Classification and Regression Trees. 1984, Chapman & Hall/CRC.
5. Breiman L. Random Forests. Machine Learning 2001, 45:5-32.
6. Al-Ani A, Deriche M, Chebil J. A new mutual information based measure for feature selection. Intelligent Data Analysis 2003, 7:43-57.
7. Last M, Kandel A, Maimon O. Information-theoretic algorithm for feature selection. Pattern Recognition Letters 2001, 22(6-7):799-811.
8. Song GJ, Tang SW, Yang DQ, Wang TJ. A Spatial Feature Selection Method Based on Maximum Entropy Theory. Journal of Software 2003, 14(9):1544-1550.
9. Mitra P, Murthy C, Pal S. Unsupervised feature selection using feature similarity. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2002, 24(3):301-312.
10. Kira K, Rendell LA. A practical approach to feature selection. ML92: Proceedings of the ninth international workshop on Machine learning 1992, 249-256. San Francisco, CA: Morgan Kaufmann Publishers Inc.
11. Kononenko I, Šimec E, Robnik-Šikonja M. Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF. Applied Intelligence 1997, 7:39-55.
12. Hall MA, Smith LA. Feature Selection for Machine Learning: Comparing a Correlation-based Filter Approach to the Wrapper. Proceedings of the Twelfth Florida International Artificial Intelligence Research Symposium Conference 1999, 235-239. Menlo Park, CA: The AAAI Press.
13. Whitley D, Ford M, Livingstone D. Unsupervised Forward Selection: A Method for Eliminating Redundant Variables. Journal of Chemical Information and Computer Science 2000, 40(5):1160-1168.
14. Yu L, Liu H. Efficient Feature Selection via Analysis of Relevance and Redundancy. The Journal of Machine Learning Research 2004, 5:1205-1224.
15. Kohavi R, John GH. Wrappers for feature selection. Artificial Intelligence 1997, 97(1-2):273-324.
16. Blum A, Langley P. Selection of relevant features and examples in machine learning. Artificial Intelligence 1997, 97(1-2):245-271.
17. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 1974, 19(6):716-723.
18. Schwarz G. Estimating the dimension of a model. The Annals of Statistics 1978, 6(2):461-4643.
19. Zhai HL, Chen XG, Hu ZD. A new approach for the identification of important variables. Chemometrics and Intelligent Laboratory Systems 2006, 80:130-135.
20. Li S, Fedorowicz A, Singh H, Soderholm SC. Application of the Random Forest Method in Studies of Local Lymph Node Assay Based Skin Sensitization Data. Journal of Chemical Information and Modeling 2005, 45(4):952-964. 10.1021/ci050049u, 16045289.
21. Díaz-Uriarte R, de Andrés SA. Gene selection and classification of microarray data using random forest. BMC Bioinformatics 2006, 7(3).
22. Strobl C, Boulesteix AL, Zeileis A, Hothorn T. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics 2007, 8(25).
23. Calle ML, Urrea V. Letter to the Editor: Stability of Random Forest importance measures. Briefings in Bioinformatics 2011, 12:86-89. 10.1093/bib/bbq011, 20360022.
24. Nicodemus KK. Letter to the Editor: On the stability and ranking of predictors from random forest variable importance measures. Briefings in Bioinformatics 12(4):369-373.
25. Li S. Random KNN Modeling and Variable Selection for High Dimensional Data. PhD thesis 2009, West Virginia University.
26. Fix E, Hodges J. Discriminatory Analysis-Nonparametric Discrimination: Consistency Properties. 1951, Tech. Rep. 21-49-004, 4, US Air Force, School of Avaiation Medicine.
27. Cover T, Hart P. Nearest Nieghbor Pattern Classification. IEEE Transaction on Information Theory 1967, IT-13:21-27.
28. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning - Data Mining, Inference, and Prediction. 2001, New York: Springer, chap. 9, section 2.
29. Crookston NL, Finley AO. yaImpute: An R Package for kNN Imputation. Journal of Statistical Software 2007, 23(10):1-16.
30. Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, Botstein D, Altman RB. Missing value estimation methods for DNA microarrays. Bioinformatics 2001, 17(6):520-525. 10.1093/bioinformatics/17.6.520, 11395428.
31. Ho TK. The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998, 20(8):832-844.
32. Dietterich TG. Machine-Learning Research: Four Current Directions. The AI Magazine 1998, 18(4):97-136.
33. Breiman L. Heuristics of instability and stabilization in model selection. The Annals of Statistics 1996, 24(6):2350-2383.
34. McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 1947, 12:153-157. 10.1007/BF02295996, 20254758.
35. Svetnik V, Liaw A, Tong C, Culberson J, Sheridan R, Feuston B. Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. Journal of Chemical Information and Computer Science 2003, 43(6):1947-1958.
36. Lin Y, Jeon Y. Random Forests and Adaptive Nearest Neighbors. Journal of the American Statistical Association 2006, 101(474):578-590.