Feature selection based on class-dependent densities for high-dimensional binary data

IEEE Transactions on Knowledge and Data Engineering

Volumn 24, Issue 3, 2012, Pages 465-477

  Javed, Kashif ^a   Babri, Haroon A ^b   Saeed, Mehreen ^a  

^a UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Pakistan)

^b NATIONAL UNIVERSITY OF COMPUTER AND EMERGING SCIENCES   (Pakistan)

Author keywords

binary data; classification; Feature ranking; feature subset selection; two stage feature selection

Indexed keywords

AGNOSTIC LEARNING; BINARY DATA; CLASSIFICATION ACCURACY; COMPUTATIONALLY EFFICIENT; DATA SETS; DENSITY-BASED; FEATURE RANKING; FEATURE SELECTION ALGORITHM; FEATURE SETS; FEATURE SUBSET; FEATURE SUBSET SELECTION; HIGH DIMENSIONAL DATA; HIGH-DIMENSIONAL; KNOWLEDGE MANAGEMENT SYSTEM; MUTUAL INFORMATIONS; NAIVE BAYES; PRE-PROCESSING STEP; PRIOR KNOWLEDGE; REAL LIFE DATASETS; REDUNDANT INFORMATIONS; RIDGE REGRESSION; TWO STAGE;

ALGORITHMS; CLASSIFICATION (OF INFORMATION); KNOWLEDGE MANAGEMENT; REGRESSION ANALYSIS; SET THEORY;

FEATURE EXTRACTION;

EID: 84856505051     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2010.263     Document Type: Article

References (46)

1. I. Guyon, A. Saffari, G. Dror, and G. Cawley, "Agnostic Learning vs. Prior Knowledge Challenge" Proc. Int'l Joint Conf. Neural Networks (IJCNN), http://www.agnostic.inf.ethz.ch, 2007.
2. "Feature Selection Challenge by Neural Information Processing Systems Conference (NIPS)" http://www.nipsfsc.ecs.soton. ac.uk, 2003.
3. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. Wiley, 2001.
4. D. Koller and M. Sahami, "Toward Optimal Feature Selection" Proc. 13th Int'l Conf. Machine Learning, pp. 284-292, 1996.
5. L. Yu and H. Liu, "Efficient Feature Selection via Analysis of Relevance and Redundancy" J. Machine Learning Research, vol. 5, pp. 1205-1224, 2004.
6. M. Dash and H. Liu, "Feature Selection for Classification" Intelligent Data Analysis, Elsevier Science B.V., vol. 1, no. 3, pp. 131-156, 1997.
7. I. Guyon and A. Elisseeff, "An Introduction to Variable and Feature Selection" J. Machine Learning Research, vol. 3, pp. 1157-1182, 2003.
8. L. Jimenez and D. Landgrebe, "Supervised Classification in High Dimensional Space: Geometrical, Statistical and Asymptotical Properties of Multivariate Data" IEEE Trans. Systems, Man and Cybernetics-Part C: Applications and Rev., vol. 28, no. 1, pp. 39-54, Feb. 1998. (Pubitemid 128748606)
9. D. Scott and J. Thompson, "Probability Density Estimation in Higher Dimensions" Proc. 15th Symp. Interface, Elsevier Science Publishers, pp. 173-179, 1983. (Pubitemid 14496542)
10. R. Bellman, Adaptive Control Processes: A Guided Tour. Princeton Univ. Press, 1961.
11. S. Ruadys and A. Jain, "Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners" IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 3, pp. 252-264, Mar. 1991.
12. K. Kira and L.A. Rendell, "A Practical Approach to Feature Selection" Proc. Ninth Int'l Conf. Machine Learning, pp. 249-256, 1992.
13. I. Guyon, J. Watson, S. Barnhill, and V. Vapnik, "Gene Selection for Cancer Classification Using Support Vector Machines" Machine Learning, vol. 46, pp. 389-422, 2002. (Pubitemid 34129977)
14. J.B. Tenenbaum, V. de Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction" Science, vol. 290, pp. 2319-2323, 2000. (Pubitemid 32041577)
15. L.K. Saul and S.T. Roweis, "Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds" J. Machine Learning Research, vol. 4, pp. 119-155, 2003.
16. L. van der Maaten, E. Postma, and H. van den Herik, "Dimensionality Reduction: A Comparative Review" Technical Report TiCC-TR 2009-005, Tilburg Univ., 2009.
17. R. Kohavi and G. John, "Wrappers for Feature Subset Selection" Artificial Intelligence, vol. 97, pp. 273-324, Dec. 1997. (Pubitemid 127401107)
18. H. Liu and L. Yu, "Toward Integrating Feature Selection Algorithms for Classification and Clustering" IEEE Trans. Knowledge and Data Eng., vol. 17, no. 4, pp. 491-502, Apr. 2005. (Pubitemid 40495592)
19. A.L. Blum and P. Langley, "Selection of Relevant Features and Examples in Machine Learning" Artificial Intelligence, Elsevier B.V., vol. 97, pp. 245-271, 1997. (Pubitemid 127401106)
20. I. Guyon, S. Gunn, M. Nikravesh, and L.A. Zadeh, Feature Extraction Foundations and Applications. Springer, 2006.
21. M. Hall, "Correlation-Based Feature Selection for Discrete and Numeric Class Machine Learning" Proc. 17th Int'l Conf. Machine Learning, 2000.
22. R. Ruiz and J.S. Aguilar-Ruiz, "Analysis of Feature Rankings for Classification" Proc. Int'l Symp. Intelligent Data Analysis (IDA), pp. 362-372, 2005.
23. L. Yu and H. Liu, "Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution" Proc. 20th Int'l Conf. Machine Learning, 2003.
24. F. Fleuret, "Fast Binary Feature Selection with Conditional Mutual Information" J. Machine Learning Research, vol. 5, pp. 1531-1555, 2004.
25. H. Peng, F. Long, and C. Ding, "Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy" IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1226-1238, Aug. 2005. (Pubitemid 41245053)
26. G. Qu, S. Hariri, and M. Yousaf, "A New Dependency and Correlation Analysis for Features" IEEE Trans. Knowledge and Data Eng., vol. 17, no. 9, pp. 1199-1207, Sept. 2005.
27. J. Pearl, Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
28. A. Freno, "Selecting Features by Learning Markov Blankets" Proc. 11th Int'l Conf., KES 2007 and XVII Italian Workshop Neural Networks Conf. Knowledge-Based Intelligent Information and Eng. Systems: Part I (KES/WIRN), pp. 69-76, 2007.
29. M. Saeed, "Bernoulli Mixture Models for Markov Blanket Filtering and Classification" J. Machine Learning Research, vol. 3, pp. 77-91, 2008.
30. A. Juan and E. Vidal, "On the Use of Bernoulli Mixture Models for Text Classification" Pattern Recognition, vol. 35, pp. 2705-2710, 2002.
31. A. Juan and E. Vidal, "Bernoulli Mixture Models for Binary Images" Proc. 17th Int'l Conf. Pattern Recognition, (ICPR '04), 2004.
32. "Annual KDD Cup 2001" http://www.sigkdd.org/kddcup/, 2001.
33. R. Agrawal and R. Srikant, "Fast Algorithms for Mining Association Rules" Proc. 20th Int'l Conf. Very Large Databases (VLDB '94), 1994.
34. J. Wilbur, J. Ghosh, C. Nakatsu, S. Brouder, and R. Doerge, "Variable Selection in High-Dimensional Multivariate Binary Data with Application to the Analysis of Microbial Community DNA Fingerprints" Biometrics, vol. 58, pp. 378-386, 2002. (Pubitemid 34619789)
35. I. Guyon et al., "CLOP" http://ymer.org/research/files/clop/ clop.zip, 2011.
36. M. Saeed, "Hybrid Learning Using Mixture Models and Artificial Neural Networks" Hands-on Pattern Recognition Challenges in Data Representation, Model Selection, and Performance Prediction, http://www.clopinet.com/ChallengeBook.html, Microtome, 2008.
37. M. Saeed and H. Babri, "Classifiers Based on Bernoulli Mixture Models for Text Mining and Handwriting Recognition" Proc. IEEE Int'l Joint Conf. Neural Networks, 2008.
38. T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley and Sons, 1991.
39. L. Jimenez and D.A. Landgrebe, "Projection Pursuit in High Dimensional Data Reduction: Initial Conditions, Feature Selection and the Assumption of Normality" Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, 1995.
40. C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
41. R.W. Lutz, "Doubleboost" Fact Sheet http://clopinet.com/ isabelle/Projects/agnostic/, 2007.
42. V. Nikulin, "Classification with Random Sets, Boosting and Distance-Based Clustering" Fact Sheet http://clopinet.com/isabelle/ Projects/agnostic/, 2007.
43. V. Franc, "Modified Multi-Class SVM Formulation; Efficient LOO Computation" Fact Sheet http://clopinet.com/isabelle/Projects/agnostic/, 2007.
44. H.J. Escalante, "Particle Swarm Optimization for Neural Networks" Fact Sheet http://clopinet.com/isabelle/Projects/agnostic/, 2007.
45. J. Reunanen, "Cross-Indexing" Fact Sheet http://clopinet.com/ isabelle/Projects/agnostic/, 2007.
46. I.C. ASML team, "Feature Selection with Redundancy Elimination + Gradient Boosted Trees" Fact Sheet http://clopinet.com/isabelle/Projects/ agnostic/, 2007.