A two-stage Markov blanket based feature selection algorithm for text classification

Neurocomputing

Volumn 157, Issue , 2015, Pages 91-104

  Javed, Kashif ^a   Maruf, Sameen ^a   Babri, Haroon A ^a  

^a UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Pakistan)

Author keywords

Markov blanket discovery; Text classification; Two stage feature selection

Indexed keywords

BISMUTH COMPOUNDS; CLASSIFIERS; FEATURE EXTRACTION; SUPPORT VECTOR MACHINES; TEXT PROCESSING;

CLASSIFIER'S PERFORMANCE; EFFECTIVENESS AND EFFICIENCIES; FEATURE SELECTION ALGORITHM; FEATURE SUBSET SELECTION; FEATURE TRANSFORMATIONS; MARKOV BLANKETS; STATISTICAL SIGNIFICANCE; TEXT CLASSIFICATION;

CLASSIFICATION (OF INFORMATION);

ARTICLE; AUTOMATION; BAYES THEOREM; CLASSIFICATION ALGORITHM; INFORMATION PROCESSING; K NEAREST NEIGHBOR; MARKOV BLANKET FILTER; MATHEMATICAL MODEL; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; SUPPORT VECTOR MACHINE; SYSTEMATIC ERROR; TWO STAGE FEATURE SELECTION;

EID: 84924067797     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2015.01.031     Document Type: Article

References (47)

1. Forman G. An extensive empirical study of feature selection metrics for text classification. J. Mach. Learn. Res. 2003, 3:1289-1305.
2. Y. Yang, J. Pedersen, A comparative study on feature selection in text categorization, in: Proceedings of the 14th International Conference on Machine Learning (ICML), 1997, pp. 412-420.
3. D. Koller, M. Sahami, Toward Optimal Feature Selection, Technical Report 1996-77, Stanford InfoLab, 1996.
4. Uysal A.K., Gunal S. A novel probabilistic feature selection method for text classification. Knowl. Based Syst. 2012, 36(2):226-235.
5. Uguz H. A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm. Knowl. Based Syst. 2011, 24(7):1024-1032.
6. Aggarwal C.C., Zhai C. Mining Text Data 2012, Springer.
7. Joachims T. Learning to Classify Text using Support Vector Machines 2002, Kluwer Academic Publishers.
8. Montanes E., Diaz I., Ranilla J., Combarro E.F., Fernandez J. Scoring and selecting terms for text categorization. IEEE Intell. Syst. 2005, 20(3):40-47.
9. Sebastiani F. Machine learning in automated text categorization. ACM Comput. Surv. 2002, 34(1):1-47.
10. T. Joachims, Text categorization with support vector machines: learning with many relevant features, in: Proceedings of 10th European Conference on Machine Learning (ECML), 1998, pp. 137-142.
11. D. Mladenic, M. Grobelnik, Feature selection for unbalanced class distribution and naive bayes, in: Proceedings of the Sixteenth International Conference on Machine Learning (ICML), 1999, pp. 258-267.
12. T. Basu, C.A. Murthy, Effective text classification by a supervised feature selection approach, in: Proceedings of 2012 International Conference on Data Mining, 2012, pp. 918-925.
13. Salton G., Buckley C. Term-weighting approaches in automatic text retrieval. Inf. Process. Manag. 1988, 24(5):513-523.
14. Quan X., Liu W., Qiu B. Term weighting schemes for question categorization. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33(5):1009-1021.
15. Zhang W., Yoshida T., Tang X. A comparative study of tf*idf, lsi and multi-words for text classification. Expert Syst. Appl. 2011, 38:2758-2765.
16. K. Javed, Development of feature selection algorithms for high-dimensional binary data (Ph.D. thesis), Dept. of Electrical Engineering, University of Engineering and Technology, Lahore, Pakistan, 2012.
17. Yu L., Liu H. Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 2004, 5:1205-1224.
18. Kohavi R., John G. Wrappers for feature subset selection. Artif. Intell. 1997, 97:273-324.
19. Guyon I., Gunn S., Nikravesh M., Zadeh L. Feature Extraction: Foundations and Applications 2006, Springer.
20. L. Yu, H. Liu, Feature selection for high-dimensional data: a fast correlation based filter solution, in: Proceedings of 20th International Conference on Machine Learning (ICML), 2003, pp. 856-863.
21. Guyon I., Elisseeff A. An introduction to variable and feature selection. J. Mach. Learn. Res. 2003, 3:1157-1182.
22. Dash M., Liu H. Feature selection for classification. Intell. Data Anal. 1997, 1(3):131-156.
23. Javed K., Saeed M., Babri H.A. The correctness problem: evaluating the ordering of binary features in rankings. Knowl. Inf. Syst. 2014, 39(3):543-563.
24. Chen J., Huang H., Tian S., Qu Y. Feature selection for text classification with naive bayes. Expert Syst. Appl. 2009, 36(3):5432-5435.
25. Javed K., Babri H., Saeed M. Feature selection based on class-dependent densities for high-dimensional binary data. IEEE Trans. Knowl. Data Eng. 2012, 24(3):465-477.
26. S. Maruf, Feature selection based on Markov blankets for the text classification problem (M.Sc. thesis), Dept. of Electrical Engineering, University of Engineering and Technology, Lahore, Pakistan, 2014.
27. Duch W. Filter methods. Feature Extraction: Foundations and Applications 2006, 89-117. Springer. I. Guyon, M. Nikravesh, S. Gunn, L. Zadeh (Eds.).
28. Arauzo-Azofra A., Aznarte J., Benitez J. Empirical study of feature selection methods based on individual feature evaluation for classification problems. Expert Syst. Appl. 2011, 38(7):8170-8177.
29. M. Makrehchi, Feature ranking for text classifiers (Ph.D thesis), Dept. of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada, 2007.
30. Javed K., Babri H.A., Saeed M. Impact of a metric of association between two variables on performance of filters for binary data. Neurocomputing 2014, 143:248-260.
31. Peng H., Long F., Ding C. Feature selection based on mutual information: criteria of max-dependency max-relevance and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27(8):1226-1238.
32. S. Das, Filters, wrappers, and a boosting based hybrid for feature selection, in: Proceedings of the 18th International Conference on Machine Learning (ICML), 2001, pp. 74-81.
33. Ogura H., Amano H., Kondo M. Feature selection with a measure of deviations from poisson in text categorization. Decis. Support Syst. 2009, 36(3):6826-6832.
34. E. Montanes, J.R. Quevedo, I. Diaz, A wrapper approach with support vector machines for text categorization, in: Computational Methods in Neural Modeling, Lecture Notes in Computer Science, vol. 2686, Springer, 2003, pp. 230-237.
35. Meng J., Lin H., Yu Y. A two-stage feature selection method for text categorization. Comput. Math. Appl. 2011, 62(7):2793-2800.
36. Uysal A.K., Gunal S. Text classification using genetic algorithm oriented latent semantic features. Expert Syst. Appl. 2014, 41(13):5938-5947.
37. Mitchell T. Machine Learning 1997, McGraw-Hill.
38. M. Saeed, Bernoulli mixture models for Markov blanket filtering and classification, in: JMLR Workshop and Conference Proceedings: Causation and Prediction Challenge (WCCI 2008), vol. 3, 2008, pp. 77-91.
39. Pearl J. Probabilistic Reasoning in Intelligent Systems 1988, Morgan Kaufmann.
40. C. Aliferis, I. Tsamardinos, A. Statnikov, HITON, a novel Markov blanket algorithm for optimal variable selection, in: Proceedings of Annual Symposium of American Medical Informatics Association (AMIA), 2003, pp. 21-25.
41. MathWorks, MATLAB: The Language of Technical Computing, 2010. http://www.mathworks.com/products/matlab/index.html.
42. G. Forman, Forman's version of Reuters-21578, TREC, and OHSUMED data sets, http://www.hpl.hp.com/techreports/2002/HPL-2002-147R1.html http://www.hpl.hp.com/techreports/2002/HPL-2002-147R1.html, 2003.
43. E.H. Han, G. Karypis, Centroid-based document classification: analysis and experimental results, in: Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery, 2000, pp. 424-431.
44. Torkkola K. Discriminative features for text document classification. Pattern Anal. Appl. 2003, 301-9692.
45. A. Saffari, I. Guyon, Quick Start Guide for Challenge Learning Object Package (CLOP), Tech. rep., Graz University of Technology and Clopinet, http://clopinet.com/clop/http://clopinet.com/clop/, 2006.
46. I. Guyon, A. Saffari, G. Dror, G. Cawley, Agnostic learning vs. prior knowledge challenge, in: Proceedings of International Joint Conference on Neural Networks (IJCNN), 2007, pp. 829-834.
47. Hall M., Holmes G. Benchmarking attribute selection techniques for discrete class data mining. IEEE Trans. Knowl. Data Eng. 2003, 15(6):1437-1447.