Text clustering with feature selection by using statistical data

IEEE Transactions on Knowledge and Data Engineering

Volumn 20, Issue 5, 2008, Pages 641-652

  Li, Yanjun ^a   Luo, Congnan ^b   Chung, Soon M ^c  

^a FORDHAM UNIVERSITY   (United States)

^b TERADATA CORPORATION   (United States)

^c Wright State University   (United States)

Author keywords

Chi2 statistic; Feature selection; Performance analysis; Text clustering; Text mining

Indexed keywords

CHI2 STATISTIC; FEATURE SELECTION; PERFORMANCE ANALYSIS; TEXT CLUSTERING; TEXT MINING;

CLUSTER ANALYSIS; TEXT PROCESSING;

CLUSTERING ALGORITHMS;

EID: 60249094938     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2007.190740     Document Type: Article

References (24)

1. C. C. Aggrawal and P. S. Yu, "Finding Generalized Projected Clusters in High Dimensional Spaces," Proc. of ACM SIGMOD Int'l Conf. on Management of Data, pp. 70-81, 2000.
2. L. Bottou and Y. Bengio, "Convergence Properties of the K-means Algorithms," Advances in Neural Information Processing Systems 7, pp. 585-592, 1994.
3. C. Buckley and A. F. Lewit, "Optimizations of Inverted Vector Searches," Proc. of Annual ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 97-110, 1985.
4. Classic data set, available at ftp://ftp.cs.cornell.edu/pub/smart/.
5. M. Dash and H. Liu, "Feature Selection for Classification," Intelligent Data Analysis, vol.1, no.3, pp. 131-156, 1997.
6. M. Dash and H. Liu, "Feature Selection for Clustering," Proc. of Paci.c-Asia Conf. on Knowledge Discovery and Data Mining (PAKDD), pp. 110-121, 2000.
7. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," Journal of the Royal Statistical Society, vol.39, no.1, pp. 1-38, 1977.
8. G. Forman, "Feature Selection: We've Barely Scratched the Surface," IEEE Intelligent Systems, November, 2005.
9. L. Galavotti, F. Sebastiani, and M. Simi, "Experiments on the Use of Feature Selection and Negative Evidence in Automated Text Categorization," Proc. of the 4th European Conf. on Research and Advanced Technology for Digital Libraries, pp. 59-68, 2000.
10. M. R. Garey, D. S. Johnson, and H. S. Witsenhausen, "Complexity of the Generalized Lloyd-Max Problem," IEEE Trans. on Information Theory, vol.28, no.2, pp. 255-256, 1982.
11. J. A. Hartigan, Clustering Algorithms, John Wiley & Sons, 1975.
12. T. Liu, S. Liu, Z. Chen, and W. Ma, "An Evaluation on Feature Selection for Text Clustering," Proc. of Int'l Conf. on Machine Learning, 2003.
13. C. Manning and H. Schutze, Foundations of Statistical Natural Language Processing, MIT Press, 1999.
14. H. T. Ng, W. B. Goh, and K. L. Low, "Feature Selection, Perception Learning, and a Usability Case Study for Text Categorization," Proc. of the 20th ACM Int'l Conf. on Research and Development in Information Retrieval, pp. 67-73, 1997.
15. G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant Features and the Subset Selection Problem," Proc. of Int'l Conf. on Machine Learning, pp. 121-129, 1994.
16. J. R. Quinlan, "Induction of Decision Trees," Machine Learning, vol.1, pp. 81-106, 1986.
17. Reuters-21578 Distribution 1.0, available at http://www.daviddlewis.com/ resources/testcollections/reuters21578.
18. C. J. van Rijsbergen, Information Retrieval, 2nd edition, Butterworth, London, 1979.
19. F. Sebastiani, "Machine Learning in Automated Text Categorization," ACM Computing Surveys, vol.34, no.1, pp. 1-47, 2002.
20. M. Steinbach, G. Karypis, and V. Kumar, "A Comparison of Document Clustering Techniques," Proc. KDD Workshop on Text Mining, 2000.
21. W. J. Wilbur and K. Sirotkin, "The Automatic Identification of Stop Words," Journal of Information Science, vol.18, no.1, pp. 45-55, 1992.
22. Y. Yang, "Noise Reduction in a Statistical Approach to Text Categorization," Proc. of Annual ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 256-263, 1995.
23. Y. Yang and J. O. Pedersen, "A Comparative Study on Feature Selection in Text Categorization," Proc. of Int'l Conf. on Machine Learning, pp. 412-420, 1997.
24. Y. Zhao and G. Karypis, "Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering," Machine Learning, vol.55. no.3, pp. 311-331, 2004.