Concept decompositions for large sparse text data using clustering

Machine Learning

Volumn 42, Issue 1-2, 2001, Pages 143-175

  Dhillon, Inderjit S ^a   Modha, Dharmendra S ^b  

^a University of Texas at Austin   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DATA MINING; ERROR ANALYSIS; FRACTALS; INFORMATION RETRIEVAL; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; STATISTICAL METHODS; VECTORS;

PRINCIPAL COMPONENT ANALYSIS; SINGLE VALUE DECOMPOSITION; SPARSITY;

TEXT PROCESSING;

EID: 0034824884     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1007612920971     Document Type: Article

References (40)

1. Berry, M. W., Dumais, S. T., & O'Brien, G. W. (1995). Using linear algebra for intelligent information retrieval. SIAM Review 37(4), 573-595.
2. Björck, A. & Golub, G. (1973). Numerical methods for computing angles between linear subspaces. Mathematics of Computation 27(123).
3. Boley, D., Gini, M., Gross, R., Han, E.-H., Hastings, K., Karypis, G., Kumar, V., Mobasher, B., & Moore, J. (1999). Document categorization and query generation on the World Wide Web using WebACE. AI Review 13(5-6), 365-391.
4. Broder, A. Z., Glassman, S. C., Manasse, M. S., & Zweig, G. (1997). Syntactic clustering of the web. Technical Report 1997-015, Digital Systems Research Center.
5. Caid, W R. & Oing, P. (1997). System and method of context vector generation and retrieval. US Patent No. 5619709.
6. Cutting, D. R., Karger, D. R., Pedersen, J. O., & Tukey, J. W. (1992). Scatter/Gather: A cluster-based approach to browsing large document collections. In: Proc. ACM SIGIR.
7. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science 41 (6), 391-407.
8. Dhillon, I. S. & Modha, D. S. (1999). Concept decompositions for large sparse text data using clustering. Technical Report RJ 10147 (95022), IBM Almaden Research Center.
9. Dhillon, I. S. & Modha, D. S. (2000). A parallel data-clustering algorithm for distributed memory multiprocessors. In: M. J. Zaki and C. T. Ho (eds.): Large-Scale Parallel Data Mining, Lecture Notes in Artificial Intelligence, Volume 1759. Springer-Verlag, New York, pp. 245-260. Presented at the 1999 Large-Scale Parallel KDD Systems Workshop, San Diego, CA.
10. Dhillon, I. S., Modha, D. S., & Spangler, W. S. (1998). Visualizing Class Structure of Multidimensional Data. In: S. Weisberg (ed.): Proceedings of the 30th Symposium on the Interface: Computing Science and Statistics, Vol. 30. Minneapolis, MN, pp. 488-493.
11. Duda, R. O. & Hart, P. E. (1973). Pattern Classification and Scene Analysis. New York: Wiley.
12. Frakes, W. B. & Baeza-Yates, R. (1992). Information Retrieval: Data Structures and Algorithms. Englewood Cliffs, New Jersey Prentice Hall.
13. Gallant, S. I. (1994). Methods for generating or revising context vectors for a plurality of word stems. US Patent No. 5325298.
14. Garey, M. R., Johnson, D. S., & Witsenhausen, H. S. (1982). The complexity of the generalized Lloyd-Max problem. IEEE Trans. Inform. Theory 28(2), 255/256.
15. Golub, G. H. and Van Loan, C. F. (1996). Matrix computations. Baltimore, MD, USA: The Johns Hopkins University Press.
16. Hartigan, J. A. (1975). Clustering Algorithms. New York: Wiley.
17. Hearst, M. A. & Pedersen, J. O. (1996). Reexamining the cluster hypothesis: Scatter/Gather on retrieval results. In: Proc. ACM SIGIR.
18. Hofmann, T. (1999). Probabilistic latent semantic indexing. In: Proc. ACM SIGIR.
19. Isbell, C. L. & Viola, P. (1998). Restructuring sparse high dimensional data for effective retrieval. In: Advances in neural information processing (Vol. 11).
20. Kleinberg, J., Papadimitriou, C. H., & Raghavan, P. (1998). A microeconomic view of data mining. Data Mining and Knowledge Discovery 2(4), 311-324.
21. Kolda, T. G. (1997). Limited-Memory Matrix Methods with Applications. Ph.D. Thesis, The Applied Mathematics Program, University of Maryland, College Park, Mayland.
22. Leland, W. E., Taqqu, M. S., Willinger, W., & Wilson, D. V. (1994). On the self-similar nature of ethernet traffic. IEEE/ACM Transactions on Networking 2(1), 1-15.
23. Mandelbrot, B. B. (1988). Fractal geometry of nature. W. H. Freeman & Company.
24. O'Leary, D. P. & Peleg, S. (1983). Digital image compression by outer product expansion. IEEE Trans. Communications 31, 441-444.
25. Papadimitriou, C. H., Raghavan, P., Tamaki, H., & Vempala, S. (1998). Latent semantic indexing: A probabilistic analysis. In: Proc. Seventeenth ACM-SIGACT-SIGMOD-SIGART Symp. Principles of Database Systems, Seattle, Washington. pp. 159-168.
26. Pollard, D. (1982). Quantization and the method of k-means. IEEE Trans. Inform. Theory 28, 199-205.
27. Rasmussen, E. (1992). Clustering Algorithms. In: W. B. Frakes & R. Baeza-Yates (eds.): Information retrieval: Data structures and algorithms, pp. 419-442, Prentice-Hall.
28. Rissanen, J., Speed, T., & Yu, B. (1992). Density estimation by stochastic complexity. IEEE Trans. Inform. Theory 38, 315-323.
29. Sabin, M. J. & Gray, R. M. (1986). Global convergence and empirical consistency of the generalized Lloyd algorithm. IEEE Trans. Inform. Theory 32(2), 148-155.
30. Sahami, M., Yusufali, S., & Baldonado, M., (1999). SONIA: A Service for Organizing Networked Information Autonomously. In: Proc. ACM Digital Libraries.
31. Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Inform. proc. & management. pp. 513-523.
32. Salton, G. & McGill, M. J. (1983). Introduction to modern retrieval. New York: McGraw-Hill Book Company.
33. Saul, L. & Pereira, F. (1997). Aggregate and mixed-order Markov models for statistical language processing. In: Proc. 2nd Int. Conf. Empirical Methods in Natural Language Processing.
34. Schütze, H. & Silverstein, C. (1997). Projections for efficient document clustering. In: Proc. ACM SIGIR.
35. Silverstein, C. & Pedersen, J. O. (1997). Almost-constant-time clustering of arbitrary corpus subsets. In: Proc. ACM SIGIR.
36. Singhal, A., Buckley, C., Mitra, M., & Salton, G. (1996). Pivoted document length normalization. In: Proc. ACM SIGIR.
37. Vaithyanathan, S. & Dom, B. (1999). Model selection in unsupervised learning with applications to document clustering. In: Proc. 16th Int. Machine Learning Conf., Bled, Slovenia.
38. Willet, P. (1988). Recent trends in hierarchic document clustering: a critical review. Inform. Proc. & Management pp. 577-597.
39. Zamir, O. & Etzioni, O. (1998). Web document clustering: A feasibility demonstration. In: Proc. ACM SIGIR.
40. Zipf, G. K. (1949). Human behavior and the principle of least effort. Reading, MA : Addison Wesley.