Feature selection based on term frequency and t-test for text categorization

ACM International Conference Proceeding Series

Volumn , Issue , 2012, Pages 1482-1486

  Wang, Deqing ^a   Zhang, Hui ^a   Liu, Rui ^a   Lv, Weifeng ^a  

^a BEIHANG UNIVERSITY   (China)

Author keywords

feature selection; T test; term frequency; text classification

Indexed keywords

CHI-SQUARE STATISTICS; COMPARATIVE EXPERIMENTS; DOCUMENT FREQUENCY; FEATURE SELECTION METHODS; HIGH FREQUENCY HF; INFORMATION GAIN; SELECTION FUNCTION; T-TEST; TERM FREQUENCY; TEXT CATEGORIZATION; TEXT CLASSIFICATION; TEXT CORPORA;

CLASSIFICATION (OF INFORMATION); FEATURE EXTRACTION; TEXT PROCESSING;

KNOWLEDGE MANAGEMENT;

EID: 84871054166     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2396761.2398457     Document Type: Conference Paper

References (21)

1. P. Billingsley. Probability and Measure (Third ed.). John Wiley & sons, 1995, 357-363.
2. C. Chang and C. Lin. Libsvm: a library for support vector machines. 2001.
3. C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 1995, (20), 273-297.
4. T. Dunning. Accurate methods for the statistics of surprise and coincidence. Comput. Linguist., 1993, 19(1), 61-74.
5. G. Forman. An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research, 2003, 3, 1289-1305.
6. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. Journal of Machine Learning Research, 2003, 3, 1157-1182.
7. E.-H. Han and G. Karypis. Centroid-based document classification: Analysis & experimental results. In: Proceedings of PKDD, 2000.
8. D. Koller and M. Sahami. Hierarchically classifying documents using very few words. In: Proceedings of ICML, 1997, 170-178.
9. S. Li, R. Xia, C. Zong, and C. Huang. A framework of feature selection methods for text categorization. In: Proceedings of 47th ACL and the 4th AFNLP, 2009.
10. A. McCallum and K. Nigam. A comparison of event models for naive bayes text classification. In Proceedings of the AAAI-98 Workshop, 1998.
11. D. Mladenic and M. Grobelnik. Feature selection for unbalanced class distribution and naive bayes. In: Proceedings of ICML, 1999.
12. G. Salton and C. Buckley. Term-weighting approaches in automatic text retrieval. Information Processing & Management, 1988, 24(5), 513-523.
13. F. Sebastiani. Machine learning in automated text categorization. ACM Comput Surv, 2002, 34(1), 1-47.
14. R. Tibshirani, T. Hastie, B. Narasimhan, and G. Chu. Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proc. Natl. Acad. Sci., 2002, 99: 6567-6572.
15. H. Uguz. A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm. Knowl.-Based Syst., 2011, 24(7): 1024-1032.
16. A. Unler, A. Murat, and R. B. Chinnam. mr2pso: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inf. Sci., 2011, 181(20):4625-4641.
17. Y.-Q. Wei, P.-Y. Liu, and Z.-F. Zhu. A feature selection method based on improved tfidf. In: Proceedings of the ICPCA, 2008, 94-97.
18. S. William. The probable error of a mean. Biometrika, 1908, 6(1), 1-25.
19. S.-M. Yang, X. Wu, and Z. Deng. Relative term-frequency based feature selection for text categorization. In: Proceedings of ICMLC, 2002.
20. Y.-M. Yang and J.-P. Pedersen. A comparative study on feature selection in text categorization. In: Proceedings of ICML, 1997, 412-420.
21. N.-N. Zhou and L.-P. Wang. A modified t-test feature selection method and its application on the hapmap genotype data. Geno. Prot. Bioinfo., 2007, 5(3-4), 242-249.