Booster in High Dimensional Data Classification

IEEE Transactions on Knowledge and Data Engineering

Volumn 28, Issue 1, 2016, Pages 29-40

  Kim, Hyunji ^a   Choi, Byong Su ^b   Huh, Moon Yul ^c  

^a Korea Fair Trade Mediation Agency   (South Korea)

^b Hansung University   (South Korea)

^c Sungkyunkwan University   (South Korea)

Author keywords

Booster; feature selection; High dimensional data classification; Q statistic; stability

Indexed keywords

ALGORITHMS; BOOSTERS (ROCKET); CLUSTERING ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; DATA MINING; FEATURE EXTRACTION; FORECASTING;

CLASSIFICATION MODELS; EMPIRICAL STUDIES; EVALUATION MEASURES; HIGH DIMENSIONAL DATA; MICROARRAY DATA; MICROARRAY DATA SETS; PREDICTION ACCURACY; Q STATISTICS;

CLASSIFICATION (OF INFORMATION);

EID: 84961644945     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2015.2458867     Document Type: Conference Paper

References (77)

1. T. Abeel, T. Helleputte, Y. V. de Peer, P. Dupont, and Y. Saeys, "Robust biomarker identification for cancer diagnosis with ensemble feature selection methods," Bioinformatics, vol. 26, no. 3, pp. 392-398, 2010.
2. D. Aha and D. Kibler, "Instance-based learning algorithms," Mach. Learn., vol. 6, no. 1, pp. 37-66, 1991.
3. S. Alelyan, "On feature selection stability: A data perspective," PhD dissertation, Arizona State Univ., Tempe, AZ, USA, 2013.
4. A. A. Alizadeh, M. B. Eisen, R. E. Davis, C. M. Izidore, S. Lossos, A. Rosenwald, J. C. Boldrick, H. Sabet, T. Tran, X. Yu, J. I. Powell, L. Yang, G. E. Marti, T. Moore, J. H. Jr, L. Lu, D. B. Lewis, R. Tibshirani, G. Sherlock, W. C. Chan, T. C. Greiner, D. D. Weisenburger, J. O. Armitage, R. Warnke, R. Levy, W. Wilson, M. R. Grever, J. C. Byrd, D. Botstein, P. O. Brown, and L. M. Staudt, "Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling," Nature, vol. 403, no. 6769, pp. 503-511, 2000.
5. U. Alon, N. Barkai, D. A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine, "Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays," Proc. Nat. Acad. Sci., vol. 96, no. 12, pp. 6745-6750, 1999.
6. F. Alonso-Atienza, J. L. Rojo-Alvare, A. Rosado-Muñoz, J. J. Vinagre, A. Garcia-Alberola, and G. Camps-Valls, "Feature selection using support vector machines and bootstrap methods for ventricular fibrillation detection," Expert Syst. Appl., vol. 39, no. 2, pp. 1956-1967, 2012.
7. P. J. Bickel and E. Levina, "Some theory for Fisher's linear discriminant function, naive Bayes, and some alternatives when there are many more variables than observations," Bernoulli, vol. 10, no. 6, pp. 989-1010, 2004.
8. Z. I. Botev, J. F. Grotowski, and D. P. Kroese, "Kernel density estimation via diffusion," The Ann. Statist., vol. 38, no. 5, pp. 2916-2957, 2010.
9. G. Brown, A. Pocock, M. J. Zhao, and M. Lujan, "Conditional likelihood maximization: A unifying framework for information theoretic feature selection," J. Mach. Learn. Res., vol. 13, no. 1, pp. 27-66, 2012.
10. C. Kamath, Scientific data mining: a practical perspective, Siam, 2009.
11. B. C. Christensen, E. A. Houseman, C. J. Marsit, S. Zheng, M. R. Wrensch, H. H. Nelson, M. R. Karagas, J. F. Padbury, R. Bueno, D. J. Sugarbaker, R.-F. Yeh, J. K. Wiencke, and K. T. Kelsey, "Aging and environmental exposures alter tissue-specific DNA methylation dependent upon CpG island context," PLOS Genetics, vol. 5, no. 8, e1000602, 2009.
12. C. Corinna and V. Vapnik, "Support-vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
13. T. M. Cover and J. A. Thomas, Elements of Information Theory (Series in Telecommunications and Signal Processing), 2nd ed. Hoboken, NJ, USA: Wiley, 2002.
14. D. Dembele, "A flexible microarray data simulataion model," Microarrays, vol. 2, no. 2, pp. 115-130, 2013.
15. D. Dernoncourt, B. Hanczar, and J. D. Zucker, "Analysis of feature selection stability on high dimension and small sample data," Comput. Statist. Data Anal., vol. 71, pp. 681-693, 2014.
16. J. Fan and Y. Fan, "High dimensional classification using features annealed independence rules," Ann. Statist., vol. 36, no. 6, pp. 2605-2637, 2008.
17. J. Fan, P. Hall, and Q. Yao, "To how many simultaneous hypothesis tests can normal, Student's t or bootstrap calibration be applied?," J. Am. Statist. Assoc., vol. 102, no. 480, pp. 1282-1288, 2007.
18. U. M. Fayyad and K. B. Irani, "Multi-interval discretization of continuous-valued attributes for classification learning," Artif. Intell., vol. 13, no. 2, pp. 1022-1027, 1993.
19. A. J. Ferreira and M. A. T. Figueiredo, "Efficient feature selection filters for high dimensional data," Pattern Recog. Lett., vol. 33, no. 13, pp. 1794-1804, 2012.
20. B. Franay, G. Doquire, and M. Verleysen, "Theoretical and empirical study on the potential inadequacy of mutual information for feature selection in classification," Neurocomputing, vol. 112, pp. 64-78, 2013
21. W. A. Freije, F. E. Castro-Vargas, Z. Fang, S. Horvath, T. Cloughesy, and L. M. Liau, "Gene expression profiling of gliomas strongly predicts survival," Cancer Res., vol. 64, no. 18, pp. 6503-6510, 2004.
22. T. R. Golub, D. K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, M. L. Loh, J. R. Downing, M. A. Caligiuri, C. D. Bloomfield, and E. S. Lander, "Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring," Am. Assoc. Advancement Sci., vol. 286, no. 5439, pp. 531-537, 1999.
23. G. J. Gordon, R. V. Jensen, L. L. Hsiao, S. R. Gullans, J. E. Blumenstock, S. Ramaswamy, W. G. Richards, D. J. Sugarbaker, and R. Bueno, "Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma," Cancer Res., vol. 62, no. 17, pp. 4963-4967, 2002.
24. G. Gulgezen, Z. Cataltepe, and L. Yu, "Stable and accurate feature selection," in Proc. Mach. Learn. Knowl. Discovery Databases, pp. 455-468, 2009.
25. G. Guoan, G. G. Tarek, H. Chiang-Ching, G. T. Dafydd, A. S. Kerby, M. G. T. Jeremy, L. R. K. Sharon, E. M. David, J. G. Thomas, D. I. Mark, B. O. Mark, M. H. Samir, and G. B. David, "Proteomic analysis of lung adenocarcinoma: Identification of a highly expressed set of proteins in tumors," Clinical Cancer Res., vol. 8, no. 7, pp. 2298-2305, 2002.
26. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," The J. Mach. Learn. Res., vol. 3, pp. 1157-1182, 2003.
27. M. A. Hall, "Correlation-based feature selection for machine learning," Department of Computer Science, PhD dissertation, The Univ. of Waikato, Hamilton, New Zealand, 1999.
28. Y. Han and L. Yu, "A variance reduction framework for stable feature selection," Statist. Anal. Data Mining, vol. 5, no. 5, pp. 428-445, 2012.
29. T. Hastie, The Elements of Statistical Learning, New York, NY, USA: Springer, 2009.
30. Z. He and W. Yu, "Stable feature selection for biomarker discovery," Comput. Biol. Chem., vol. 34, no. 4, pp. 215-225, 2010.
31. M. Hilario and A. Kalousis, "Approaches to dimensionality reduction in proteomic biomarker studies," Briefings Bioinf., vol. 9, no. 2, pp. 102-118, 2008.
32. Q. Hu, L. Zhang, D. Zhang, W. Pan, S. An, and W. Pedrycz, "Measuring relevance between discrete and continuous features based on neighborhood mutual information," Expert Syst. with Appl., vol. 38, no. 9, pp. 10737-10750, 2011.
33. J. Hua, W. D. Tembe, and E. R. Dougherty, "Performance of feature-selection methods in the classification of high-dimension data," Pattern Recognit., vol. 42, no. 3, pp. 409-424, 2009.
34. W. L. Hung, M. S. Yang, and D. H. Chen, "Bootstrapping approach to feature-weight selection in fuzzy c-means algorithms with an application in color image segmentation," Pattern Recognit. Lett., vol. 29, no. 9, pp. 1317-1325, 2008.
35. P. Jafari and F. Azuaje, "An assessment of recently published gene expression data analyses: reporting experimental design and statistical factors," BMC Med. Inf. Decision Making, vol. 6, no. 27, 2006.
36. N. D. Jay, S. Papillon-Cavanagh, C. Olsen, G, Bontempi, and B. Haibe-Kains, "mRMRe: An R package for parallelized mRMR ensemble feature selection," Bioinformatics, vol. 29, no. 18, pp. 2365-2368, 2012.
37. G. H. John, R. Kohavi, and K. Pfleger," Irrelevant features and the subset selection problem," in Proc. 11th Int. Conf. Mach. Learn., vol. 94, pp. 121-129, 1994.
38. G. H. John and P. Langley, "Estimating continuous distributions in Bayesian classifiers," in Proc. 11th Conf. Uncertainty Artif. Intell., pp. 338-345, 1995.
39. R. V. Jorge and A. E. Pablo, "A review of feature selection methods based on mutual information," Neural Comput. Appl., vol. 24, no. 1, pp. 175-186, 2014.
40. A. Kalousis, J. Prados, and M. Hilario, "Stability of feature selection algorithms: A study on high-dimensional spaces," Knowl. Inf. Syst., vol. 12, no. 1, pp. 95-116, 2007.
41. I. Kojadinovic, "Relevance measures for subset variable selection in regression problems based on k-additive mutual information," Comput. Statist. Data Anal., vol. 49, no. 4, pp. 1205-1227, 2005.
42. D. Koller and M. Sahami, "Toward optimal feature selection," in Proc. 13th Int. Conf. Mach. Learn., pp. 284-292, 1996.
43. L. I. Kuncheva, "A stability index for feature selection," in Proc. Artif. Intell. Appl., pp. 421-427, 2007.
44. H. Liu, J. Li, and L.Wong, "A comparative study on feature selection and classification methods using gene expression profiles and proteomic patterns," Genome Informatics Series, vol. 13, pp. 51-60, 2002.
45. H. Liu, M. Xu, H. Gu, A. Gupta, J. Lafferty, and L. Wasserman, "Forest density estimation," The J. Mach. Learn. Res., vol. 12, pp. 907-951, 2011.
46. R. S. Marko, and I. Kononenko, "Theoretical and empirical analysis of ReliefF and RReliefF," Machine learning, vol. 53, no. 1-2, pp. 23-69, 2003.
47. N.Meinshausen and P. Buhlmann, "Stability selection," J. Roy. Statist. Soc.: Series B (Statist.Methodol.), vol. 72, no. 4, pp. 417-473, 2010.
48. D. Meyer, E. Dimitriadou, K. Hornik, A. Weingessel, and F. Leisch, "e1071: Misc functions of the department of statistics (e1071), TU Wien," 2012.
49. P. E. Meyer, C. Schretter, and G. Bontempi, "Information-theoretic feature selection in microarray data using variable complementarity," IEEE J. Sel. Topics Signal Process., vol. 2, no. 3, pp. 261-274, 2008.
50. H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
51. L. P. Scott, P. Tamayo, M. Gaasenbeek, L. M. Sturla, M. Angelo, M. E. McLaughlin, J. Y. H. Kim, L. C. Goumnerova, P. M. Black, C. Lau, J. C. Allen, D. Zagzag, J. M. Olson, T. Curran, C. Wetmore, J. A. Biegel, T. Poggio, S. Mukherjee, R. Rifkin, A. Califano, G. Stolovitzky, D. N. Louis, J. P. Mesirov, E. S. Lander, and T. R. Golub, "Prediction of central nervous system embryonal tumour outcome based on gene expression," Nature, vol. 415, no. 6870, pp. 436-442, 2002.
52. R Core Team. (2014). R: A Language and environment for statistical computing, r foundation for statistical computing, Vienna, Austria [Online]. Available: http://www.R-project.org/
53. Y. Saeys, I. Inza, and P. Larranaga, "A review of feature selection techniques in bioinformatics," Bioinformatics, vol. 23, no. 19, pp. 2507-2517, 2007.
54. S. A. Sajan, J. L. Rubenstein, M. E. Warchol, and M. Lovett, "Identification of direct downstream targets of Dlx5 during early inner ear development," Human Molecular Genetics, vol. 20, no. 7, pp. 1262-1273, 2011.
55. D. W. Scott, Multivariate Density Estimation: Theory, Practice, and Visualization, Hoboken, NJ, USA: Wiley, 2009.
56. H. Silva and A. Fred, "Feature subspace ensembles: A parallel classifier combination scheme using feature selection," Multiple Classifier Syst., vol. 4472, pp. 261-270, 2007.
57. B. W. Silverman, Density Estimation for Statistics and Data Analysis, Boca Raton, FL, USA: CRC Press, 1986.
58. C. Sima, S. Attoor, U. Braga-Neto, J. Lowey, E. Suh, and E. R. Dougherty, "Impact of error estimation on feature selection," Pattern Recognit., vol. 38, no. 12, pp. 2472-2482, 2005.
59. S. Singhal, C. G. Kyvernitis, S. W. Johnson, L. R. Kaiser, M. N. Liebman, and S. M. Albelda, "Microarray data simulator for improved selection of differentially expressed genes," Cancer Biology and therapy, vol. 2, no. 4, pp. 383-391, 2003.
60. D. Singh, P. G. Febbo, K. Ross, D. G. Jackson, J. Manola, C. Ladd, P. Tamayo, A. A. Renshaw, A. V. D'Amico, J. P. Richie, E. S. Lander, M. Loda, P. W. Kantoff, T. R. Golub, and W. R. Sellers, "Gene expression correlates of clinical prostate cancer behavior," Cancer Cell, vol. 1, no. 2, pp. 203-209, 2002.
61. P. Somol and J. Novovicova, "Evaluating stability and comparing output of feature selectors that optimize feature subset cardinality," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 1921-1939, Nov. 2010.
62. Q. Song, J. Ni, and G. Wang, "A fast clustering-based feature subset selection algorithm for high-dimensional data," IEEE Trans. Knowl. Data Eng., vol. 25, no. 1, pp. 1-14, Jan. 2013.
63. T. Sørlie, C. M. Perou, R. Tibshirani, T. Aas, S. Geisler, H. Johnsen, T. Hastie, M. B. Eisen, M. van de Rijn, S. S. Jeffrey, T. Thorsen, H. Quist, J. C. Matese, P. O. Brown, D. Botstein, P. E. Lønning, and A. L. Børresen-Dale, "Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications," Proc. Nat. Acad. Sci., vol. 98, no. 19, pp. 10869-10874, 2001.
64. A. Spira, J. E. Beane, V. Shah, K. Steiling, G. Liu, F. Schembri, S. Gilman, Y. M. Dumas, P. Calner, P. Sebastiani, S. Sridhar, J. Beamis, C. Lamb, T. Anderson, N. Gerry, J. Keane, M. E. Lenburg, and J. S. Brody, "Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer," Nature Med., vol. 13, no. 3, pp. 361-366, 2007.
65. J. Stefanowski, "An experimental study of methods combining multiple classifiers-diversified both by feature selection and bootstrap sampling," Issues Representation Process. Uncertain Imprecise Inf., Akademicka OficynaWydawnicza, Warszawa, pp. 337-354, 2005.
66. A. I. Su, M. P. Cooke, K. A. Ching, Y. Hakak, J. R. Walker, T. Wiltshire, A. P. Orth, R. G. Vega, L. M. Sapinoso, A. Moqrich, A. Patapoutian, G. M. Hampton, P. G. Schultz, and J. B. Hogenesch, "Large-scale analysis of the human and mouse transcriptomes," Proc. Nat. Acad. Sci. USA, vol. 99, no. 7, pp. 4465-4470, 2002.
67. Y. Sun, S. Todorovic, and S. Goodison, "Local-learning-based feature selection for high-dimensional data analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1610-1626, Sep. 2010.
68. K. M. Ting, J. R. Wells, S. C. Tan, S. W. Teng, and G. I. Webb, "Feature-subspace aggregating: Ensembles for stable and unstable learners," Mach. Learn., vol. 82, no. 3, pp. 375-397, 2011.
69. N. Tishby, F. Pereira, and W. Bialek, "The information bottleneck method," in Proc. 37th Annu. Allerton Conf. Commun., Control Comput., pp. 368-377, 1999.
70. L. J. van 't Veer, H. Dai, M. J. van de Vijver, Y. D. He, A. A. Hart, H. L. Peterse, K. van der Kooy, M. J. Marton, A. T. Witteveen, G. J. Schreiber, R. M. Kerkhoven, C. Roberts, R. S. Linsley, R. Bernards, and S. H. Friend, "Gene expression profiling predicts clinical outcome of breast cancer," Nature, vol. 415, no. 6871, pp. 530-536, 2002.
71. W. N. Venables and B. D. Ripley, Modern Applied Statistics with S, 4th ed. New York, NY, USA: Springer, 2003.
72. I. Witten and E. Frank, Data Mining-Practical Machine Learning Tools and Techniques with JAVA Implementations, San Mateo, CA, USA: Morgan Kaufmann, 2000.
73. T. Windeatt, M. Prior, N. Effron, and N. Intrator, "Ensemblebased feature selection criteria," in Proc. MLDM Posters, pp. 168-182, 2007.
74. B. Wu, T. Abbott, D. Fishman, W. McMurray, G. Mor, K. Stone, D. Ward, K. Williams, and H. Zhao, "Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data," Bioinformatics, vol. 19, no. 13, pp. 1636-1643, 2003.
75. W. Yao and Q. Wang, "Robust variable selection through MAVE," Comput. Statist. Data Anal., vol. 63, pp. 42-49, 2013.
76. E. J. Yeoh, M. E. Ross, S. A. Shurtleff, W. K. Williams, D. Patel, R. Mahfouz, F. G. Behm, S. C. Raimondi, M. V. Relling, A. Patel, C. Cheng, D. Campana, D. Wilkins, X. Zhou, J. Li, H. Liu, C.-H. Pui, W. E. Evans, C. Naeve, and L. Wong JR, "Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling," Cancer Cell, vol. 1, no. 2, pp. 133-143, 2002.
77. L. Yu and H. Liu, "Efficient feature selection via analysis of relevance and redundancy," The J. Mach. Learn. Res., vol. 5, no. 2, pp. 1205-1224, 2004.