Stratified sampling for feature subspace selection in random forests for high dimensional data

Pattern Recognition

Volumn 46, Issue 3, 2013, Pages 769-787

  Ye, Yunming ^a,e   Wu, Qingyao ^a,e   Zhexue Huang, Joshua ^b,d   Ng, Michael K ^c   Li, Xutao ^a,e  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

^c HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

^d Shenzhen Key Laboratory of High Performance Data Mining   (China)

^e Shenzhen Key Laboratory of Internet Information Collaboration   (China)

Author keywords

Classification; Decision trees; Ensemble classifier; High dimensional data; Random forests; Stratified sampling

Indexed keywords

DATA SETS; ENSEMBLE CLASSIFIERS; FEATURE SUBSPACE; GENE CLASSIFICATION; HIGH DIMENSIONAL DATA; IMAGE CATEGORIZATION; IN-BUILDINGS; NEAREST NEIGHBOR ALGORITHM; RANDOM FOREST ALGORITHM; RANDOM FORESTS; REAL DATA SETS; SIMPLE RANDOM SAMPLING; STATE-OF-THE-ART ALGORITHMS; STRATIFIED SAMPLING; SYNTHETIC DATA;

CLASSIFICATION (OF INFORMATION); CLUSTERING ALGORITHMS; DECISION TREES; FACE RECOGNITION; FORESTRY;

FEATURE EXTRACTION;

CLASSIFICATION; DATA PROCESSING; DECISION THEORY; INFORMATION RETRIEVAL; PERFORMANCE; RANDOM PROCESSES; SAMPLING;

EID: 84870244637     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2012.09.005     Document Type: Article

References (45)

1. T.K. Ho The random subspace method for constructing decision forests IEEE Transactions on Pattern Analysis and Machine Intelligence 20 8 1998 832 844
2. L. Breiman Random forests Machine Learning 45 1 2001 5 32
3. P. Geurts, D. Ernst, and L. Wehenkel Extremely randomized trees Machine Learning 63 1 2006 3 42
4. T. Dietterich An experimental comparison of three methods for constructing ensembles of decision trees bagging, boosting, and randomization Machine Learning 40 2 2000 139 157
5. Y. Amit, and D. Geman Shape quantization and recognition with randomized trees Neural Computation 9 7 1997 1545 1588
6. R.E. Banfield, L.O. Hall, K.W. Bowyer, and W.P. Kegelmeyer A comparison of decision tree ensemble creation techniques IEEE Transactions on Pattern Analysis and Machine Intelligence 29 1 2007 173 180
7. G. Biau, L. Devroye, and G. Lugosi Consistency of random forests and other averaging classifiers The Journal of Machine Learning Research 9 2008 2015 2033
8. Y. Lin, and Y. Jeon Random forests and adaptive nearest neighbors Journal of the American Statistical Association 101 474 2006 578 590
9. N. Meinshausen Quantile regression forests The Journal of Machine Learning Research 7 2006 999
10. R. Díaz-Uriarte, and A. de Andrés Gene selection and classification of microarray data using random forest BMC Bioinformatics 7 1 2006 3
11. A. Bureau, J. Dupuis, K. Falls, K. Lunetta, B. Hayward, T. Keith, and P. Van Eerdewegh Identifying SNPs predictive of phenotype using random forests Genetic Epidemiology 28 2 2005 171 182
12. B. Goldstein, A. Hubbard, A. Cutler, and L. Barcellos An application of random forests to a genome-wide association dataset: methodological considerations and new findings BMC Genetics 11 1 2010 49
13. Y. Meng, Y. Yu, L. Cupples, L. Farrer, and K. Lunetta Performance of random forest when SNPs are in linkage disequilibrium BMC Bioinformatics 10 1 2009 78
14. D. Schwarz, I. König, and A. Ziegler On safari to random jungle a fast implementation of random forests for high-dimensional data Bioinformatics 26 14 2010 1752
15. J. Shotton, M. Johnson, R. Cipolla, Semantic texton forests for image categorization and segmentation, in: Proceedings of the IEEE CVPR Conference on Computer Vision and Pattern Recognition, 2008, pp. 1-8.
16. F. Moosmann, E. Nowak, and F. Jurie Randomized clustering forests for image classification IEEE Transactions on Pattern Analysis and Machine Intelligence 30 9 2008 1632 1646
17. V. Lepetit, P. Lagger, P. Fua, Randomized trees for real-time keypoint recognition, in: Proceedings of the IEEE CVPR Conference on Computer Vision and Pattern Recognition, 2005, pp. 775-781.
18. A. Bosch, A. Zisserman, X. Muoz, Image classification using random forests and ferns, in: Proceedings of the IEEE ICCV International Conference on Computer Vision, 2007, pp. 1-8.
19. G. Rogez, J. Rihan, S. Ramalingam, C. Orrite, P. Torr, Randomized trees for human pose detection, in: Proceedings of the IEEE CVPR Conference on Computer Vision and Pattern Recognition, 2008, pp. 1-8.
20. G.J. Williams, Combining decision trees: initial results from the MIL algorithm, in: J.S. Gero, R.B. Stanton (Eds.), Artificial Intelligence Developments and Applications: Selected Papers from the First Australian Joint Artificial Intelligence Conference, Sydney, Australia, 2-4 November, 1987, Elsevier Science Publishers, North-Holland, 1988, pp. 273-289.
21. T.K. Ho, Random decision forests, in: Proceedings of the 3rd International Conference on Document Analysis and Recognition, IEEE Computer Society, 1995, pp. 278-282.
22. L. Breiman Bagging predictors Machine Learning 24 2 1996 123 140
23. L. Breiman, J. Friedman, R. Olsen, C. Stone, Classification and Regression Trees, 1984.
24. T. Dietterich Machine learning research four current directions AIM Magazine 18 1997 97 136
25. E. Bauer, and R. Kohavi An empirical comparison of voting classification algorithms bagging, boosting, and variants Machine Learning 36 1-2 1999 105 139
26. D. Amaratunga, J. Cabrera, Y.S. Lee, Enriched random forests, Bioinformatics 24 (18) 2010-2014
27. T. Liu, F. Wang, G. Agrawal, Stratified sampling for data mining on the deep web, in: Proceedings of the IEEE ICDM International Conference on Data Mining, 2010, pp. 324-333.
28. S. Chaudhuri, G. Das, and V. Narasayya Optimized stratified sampling for approximate query processing ACM Transactions on Database Systems (TODS) 32 2 2007 9
29. S. Fernandes, C. Kamienski, J. Kelner, D. Mariz, and D. Sadok A stratified traffic sampling methodology for seeing the big picture Computer Networks 52 2008 2677 2689
30. J. Weston, A. Elisseeff, B. Schölkopf, and M. Tipping Use of the zero norm with linear models and kernel methods The Journal of Machine Learning Research 3 2003 1439 1461
31. D. Cai, X. He, and J. Han Srda an efficient algorithm for large-scale discriminant analysis IEEE Transactions on Knowledge and Data Engineering 20 1 2007 1 12
32. R. Caruana, N. Karampatziakis, A. Yessenalina, An empirical evaluation of supervised learning in high dimensions, in: Proceedings of the IEEE ICML International Conference on Machine Learning, 2008, pp. 96-103.
33. D. Hand, and R. Till A simple generalisation of the area under the roc curve for multiple class classification problems Machine Learning 45 2 2001 171 186
34. B. Menze, B. Kelm, D. Splitthoff, U. Koethe, and F. Hamprecht On oblique random forests Machine Learning and Knowledge Discovery in Databases 2011 453 469
35. M. Varma, and A. Zisserman A statistical approach to texture classification from single images International Journal of Computer Vision 62 1-2 2005 61 81
36. J. Zhang, M. Marszalek, S. Lazebnik, C. Schmid, Local features and kernels for classification of texture and object categories: a comprehensive study, in: Proceedings of the IEEE CVPR Conference on Computer Vision and Pattern Recognition, 2007, pp. 13-13.
37. S. Zhu Statistical modeling and conceptualization of visual patterns IEEE Transactions on Pattern Analysis and Machine Intelligence 25 6 2003 691 712
38. D. Lowe Distinctive image features from scale-invariant keypoints International Journal of Computer Vision 60 2 2004 91 110
39. E. Nowak, F. Jurie, B. Triggs, Sampling strategies for bag-of-features image classification, in: Proceedings of the ECCV European Conference on Computer Vision, 2006, pp. 490-503.
40. V. Lepetit, and P. Fua Keypoint recognition using randomized trees IEEE Transactions on Pattern Analysis and Machine Intelligence 28 9 2006 1465 1479
41. G. Csurka, C. Dance, L. Fan, J. Willamowski, C. Bray, Visual categorization with bags of keypoints, in: Proceedings of the ECCV European Conference on Computer Vision, vol. 1, 2004, p. 22.
42. K. Chatfield, V. Lempitsky, A. Vedaldi, A. Zisserman, The devil is in the details: an evaluation of recent feature encoding methods, URL: http://www.robots.ox.ac.uk/∼vgg/research/encoding-eval/.
43. M. Turk, and A. Pentland Eigenfaces for recognition Journal of Cognitive Neuroscience 3 1 1991 71 86
44. A. Georghiades, P. Belhumeur, and D. Kriegman From few to many illumination cone models for face recognition under variable lighting and pose IEEE Transactions on Pattern Analysis and Machine Intelligence 23 6 2001 643 660
45. F. Samaria, A. Harter, Parameterisation of a stochastic model for human face identification, in: Proceedings of the IEEE Conference on Applications of Computer Vision, 1994, pp. 138-142.