Feature selection via maximizing global information gain for text classification

Knowledge-Based Systems

Volumn 54, Issue , 2013, Pages 298-309

  Shang, Changxing ^a,b,c   Li, Min ^a,b   Feng, Shengzhong ^a   Jiang, Qingshan ^a   Fan, Jianping ^a  

^a SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c ZHENGZHOU INFORMATION SCIENCE AND TECHNOLOGY INSTITUTE   (China)

Author keywords

Distributional clustering; Feature selection; High dimensionality; Information bottleneck; Text classification

Indexed keywords

FEATURE EXTRACTION; REDUNDANCY; TEXT PROCESSING;

CURSE OF DIMENSIONALITY; DISCRIMINATIVE POWER; DISTRIBUTIONAL CLUSTERING; EFFICIENT FEATURE SELECTIONS; HIGH DIMENSIONALITY; INFORMATION BOTTLENECK; PRE-PROCESSING STEP; TEXT CLASSIFICATION;

CLASSIFICATION (OF INFORMATION);

EID: 84901812079     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2013.09.019     Document Type: Article

References (30)

1. A. McCallum, K. Nigam, A comparison of event models for naive bayes text classification, in: AAAI-98 Workshop on Learning for Text Categorization, Vol. 752, Citeseer, 1998, pp. 41-48.
2. D. Lewis, Naive (bayes) at forty: the independence assumption in information retrieval, Machine Learning ECML-98 (1998) 4-15.
3. T. Joachims, Text categorization with support vector machines: learning with many relevant features, Machine learning ECML-98 (1998) 137-142.
4. Y. Yang, X. Liu, A re-examination of text categorization methods, in: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, 1999, pp. 42-49.
5. Y. Yang, J. Pedersen, A comparative study on feature selection in text categorization, in: Machine Learning-Internatinal Workshop then Conference, Morgan Kaufmann Publishers, Inc., 1997, pp. 412-420.
6. J. Quinlan, Induction of decision trees, Machine Learning 1 (1) (1986) 81-106.
7. D. Mladenic, M. Grobelnik, Feature selection for unbalanced class distribution and Naive Bayes, in: Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999), Morgan Kaufmann Publishers, Inc., 1999, pp. 258-267.
8. H. Peng, F. Long, C. Ding, Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy, IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (2005) 1226-1238.
9. H.H. Yang, J. Moody, Data visualization and feature selection: new algorithms for nongaussian data, in: Advances in Neural Information Processing Systems, MIT Press, 1999, pp. 687-693.
10. P. Meyer, G. Bontempi, On the use of variable complementarity for feature selection in cancer classification, Applications of Evolutionary Computing (2006) 91-102.
11. P. Meyer, G. Bontempi, Information-theoretic feature selection in micoarray data using variable complementarity, IEEE Journal of Selected Topics in Singal Processing 2 (2008) 261-274.
12. F. Pereira, N. Tishby, L. Lee, Distributional clustering of english words, in: Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, 1993, pp. 183-190.
13. N. Tishby, F.C. Pereira, W. Bialek, The information bottleneck method, in: Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, 1999, pp. 368-377.
14. N. Slonim, N. Tishby, Agglomerative information bottleneck, in: Advances in Neural Information Processing Systems, MIT Press, 1999, pp. 617-623.
15. C. Ding, X. He, Cluster merging and splitting in hierarchical clustering algorithms, in: Proceedings of the 2002 IEEE International Conference on Data Mining, 2002, ICDM 2003, IEEE, 2002, pp. 139-146.
16. R. Bekkerman, R. El-Yaniv, N. Tishby, Y. Winter, Distributional word clusters vs. words for text categorization, The Journal of Machine Learning Research 3 (2003)1183-1208.
17. N. Slonim, N. Tishby, Unsupervised document classification using sequential information maximization, in: ACM SIGIR 2002, ACM Press, 2002, pp. 129-136.
18. I. Dhillon, S. Mallela, R. Kumar, A divisive information theoretic feature clustering algorithm for text classification, The Journal of Machine Learning Research 3 (2003) 1265-1287.
19. T.M. Cover, J.A. Thomas, Elements of Information Theory, Wiley-Interscience, 1991.
20. G. Brown, A new perspective for information theoretic feature selection, in: Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS), 2009, pp. 49-56.
21. G. Brown, A. Pocock, M. Zhao, M. Luján, Conditional likelihood maximisation: a unifying framework for information theoretic feature selection, The Journal of Machine Learning Research 13 (2012) 27-66.
22. S. Kullback, R. Leibler, On information and sufficiency, Annals of Mathematical Statistics 22 (1951) 79-86.
23. J. Lin, Divergence measures based on the shannon entropy, IEEE Transactions on Information Theory 37 (1) (1991) 145-151.
24. A.K. McCallum, Bow: A Toolkit for Statistical Language Modeling, Text Retrieval, Classification and Clustering , 1996.
25. S. Bird, E. Loper, E. Klein, Natural Language Processing with Python, OReilly, 2009.
26. P. Domingos, M. Pazzani, On the optimality of the simple bayesian classifier under zero-one loss, Machine Learning 29 (2) (1997) 103-130.
27. J. Friedman, On bias, variance, 0/1-loss, and the curse-of- dimensionality, Data Mining and Knowledge Discovery 1 (1) (1997) 55-77.
28. V. Vapnik, The Nature of Statistical Learning Theory, Springer, 1999.
29. I. Mierswa, M. Wurst, R. Klinkenberg, M. Scholz, T. Euler, Yale: rapid prototyping for complex data mining tasks, in: KDD 06: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006, pp. 935-940.
30. K. Schneider, Techniques for improving the performance of naive bayes for text classification, Computational Linguistics and Intelligent Text Processing (2005) 682-693.