T-Test feature selection approach based on term frequency for text categorization

Pattern Recognition Letters

Volumn 45, Issue 1, 2014, Pages 1-10

  Wang, Deqing ^a   Zhang, Hui ^a   Liu, Rui ^a   Lv, Weifeng ^a   Wang, Datao ^b  

^a BEIHANG UNIVERSITY   (China)

^b Jinan Resident Office of China National Audit Office   (China)

Author keywords

Feature selection; Student t test; Term frequency; Text classification

Indexed keywords

CLASSIFICATION (OF INFORMATION); FEATURE EXTRACTION; INFORMATION FILTERING;

CHI-SQUARE STATISTICS; COMPARATIVE EXPERIMENTS; FEATURE SELECTION METHODS; SELECTION TECHNIQUES; STUDENT T TESTS; TERM FREQUENCY; TEXT CATEGORIZATION; TEXT CLASSIFICATION;

TEXT PROCESSING;

EID: 84896987950     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2014.02.013     Document Type: Article

References (40)

1. F. Sebastiani Machine learning in automated text categorization ACM Comput. Surv. 34 1 2002 1 47
2. K.H.Y. Lin, C. Yang, H.H. Chen, Emotion classification of online news articles from the reader's perspective, in: Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, 2008.
3. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin Liblinear: a library for large linear classification J. Mach. Learn. Res. 9 2008 1871 1874
4. Deqing Wang, Hui Zhang, Rui Liu, Mengxiang Lin, and Wenjun Wu Predicting bugs components via mining bug reports J. Softw. 7 5 2012 1149 1154
5. R. Quinlan Induction of decision trees Mach. Learn. 1 1 1986 81 106
6. YiMing Yang, Jan O. Pedersen, A comparative study on feature selection in text categorization, in: Proceedings of ICML, 1997, pp. 412-420.
7. E-H. Han, G. Karypis, Centroid-based document classification: analysis & experimental results, in: Proceedings of PKDD, 2000.
8. Deqing Wang, Junjie Wu, Ke Xu, and Mengxiang Lin Towards enhancing centroid classifier for text classification - a border-instance approach Neurocomputing 101 2013 299 308
9. C. Cortes, and V. Vapnik Support-vector networks Mach. Learn. 20 1995 273 297
10. Isabelle Guyon, and Andre Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
11. F. Fazayeli, L.P. Wang, and J. Mandziuk Feature selection based on the rough set theory and expectation-maximization clustering algorithm Rough Sets Current Trends Comput. 5306 2005 272 282
12. Su. Chao-Ton, and Yu-Hsiang Hsiao Multiclass MTS for simultaneous feature selection and classification IEEE Trans. Knowl. Data Eng. 21 2009 192 205
13. Deqing Wang, Hui Zhang, Rui Liu, Weifeng Lv, Feature selection based on term frequency and T-test for text categorization, in: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, 2012, pp. 1482-1486.
14. D. Koller, M. Sahami, Hierarchically classifying documents using very few words, in: Proceedings of ICML, 1997, pp. 170-178.
15. D. Mladenic, M. Grobelnik, Feature selection for unbalanced class distribution and Naive Bayes, in: Proceedings of ICML, 1999.
16. Karl-Michael Schneider, Weighted average pointwise mutual information for feature selection in text categorization, in: Proceedings of The PKDD05, 2005, pp. 252-263.
17. D.M. Blei, A.Y. Ng, and M.I. Jordan Latent dirichlet allocation J. Mach. Learn. Res. 3 2003 993 1022
18. L.P. Wang, Feng Chu, and Wei Xie Accurate cancer classification using expressions of very few genes IEEE/ACM Trans. Bioinf. Comput. Biol. 4 2007 40 53 (Pubitemid 46375815)
19. Nina Zhou, and Liping Wang A modified t-test feature selection method and its application on the hapmap genotype data Geno. Prot. Bioinfo. 5 3-4 2007 242 249 (Pubitemid 351215123)
20. Bing Liu, Chunru Wan, and L.P. Wang An efficient semi-unsupervised gene selection method via spectral biclustering IEEE Trans. Nano-Biosci. 5 2 2006 110 114 (Pubitemid 44144067)
21. I.-S. Oh, J.-S. Lee, and B.-R. Moon Hybrid genetic algorithms for feature selection IEEE Trans. Pattern Anal. Mach. Intell. 26 2004 1424 1437 (Pubitemid 40168020)
22. R. Tibshirani, T. Hastie, B. Narasimhan, and G. Chu Diagnosis of multiple cancer types by shrunken centroids of gene expression Proc. Natl. Acad. Sci. 99 2002 6567 6572 (Pubitemid 34526173)
23. Stewart M. Yang, Xiaobin Wu, Zhihong Deng, Ming Zhang, Dongqing Yang, Relative term-frequency based feature selection for text categorization, in: Proceedings of ICMLC, 2002.
24. Karl-Michael Schneider, A new feature selection score for multinomial Naive Bayes text classification based on KL-divergence, 2004.
25. G. Forman An extensive empirical study of feature selection metrics for text classification J. Mach. Learn. Res. 3 2003 1289 1305
26. T. Dunning Accurate methods for the statistics of surprise and coincidence Comput. Linguist. 19 1 1993 61 74
27. Andrew McCallum, Kamal Nigam, A comparison of event models for Naive Bayes text classification, in: Proceedings of the AAAI-98 Workshop, 1998.
28. R. Fano Transmission of Information 1961 MIT Press
29. Harun Uguz A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm Knowl.-Based Syst. 24 7 2011 1024 1032
30. Alper Unler, Alper Murat, and Ratna Babu Chinnam mr2pso: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification Inf. Sci. 181 20 2011 4625 4641
31. J. Devore, and R. Peck Statistics: The Exploration and Analysis of Data third ed. 1997 Duxbury Press Pacific Grove, USA
32. J.W. Lindeberg Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung Math. Z. 15 1922 211 225
33. W. Feller Ueber den zentralen Grenzwertsatz der Wahrscheinlichkeitsrechnung Math. Z. 40 1935 521 559
34. P. Billingsley Probability and Measure Third ed. 1995 John Wiley & sons pp. 357-363
35. S. William The probable error of a mean Biometrika 6 1 1908 1 25
36. G. Salton, and C. Buckley Term-weighting approaches in automatic text retrieval Inf. Process. Manage. 24 5 1988 513 523
37. L.P. Wang Support Vector Machines: Theory and Application 2005 Springer
38. A. Sloin, and D. Burshtein Support vector machine training for improved hidden markov modeling IEEE Trans. Signal Process. 56 2008 172 188
39. C. Chang, C. Lin, LibSVM: a library for support vector machines, 2001.
40. D. Lewis, Evaluating and optimizing autonomous text classification systems, in: Proceedings of The 18th ACM International Conference on Research and Development in Information Retrieval, 1995, pp. 246-254.