Frequency-sensitive competitive learning for scalable balanced clustering on high-dimensional hyperspheres

IEEE Transactions on Neural Networks

Volumn 15, Issue 3, 2004, Pages 702-718

  Banerjee, Arindam ^a   Ghosh, Joydeep ^a  

^a The University of Texas at Austin   (United States)

Author keywords

Balanced clustering; Expectation maximization (EM); Frequency sensitive competitive learning (FSCL); High dimensional clustering; Kmeans; Normalized data; Scalable clustering; Streaming data; Text clustering

Indexed keywords

LEARNING ALGORITHMS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; PROBABILITY DISTRIBUTIONS; VECTORS;

BALANCED CLUSTERING; COMPETITIVE LEARNING; EXPECTATION MAXIMIZATION; NORMALIZED DATA; SCALABLE CLUSTERING; STREAMING DATA; TEXT CLUSTERING;

LEARNING SYSTEMS;

ARTICLE; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS;

ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS;

EID: 2542583128     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.824416     Document Type: Article

References (58)

1. A. K. Jain and R. C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, NJ: Prentice-Hall, 1988.
2. J. Ghosh, "Scalable clustering," in The Handbook of Data Mining, N. Ye, Ed. Mahwah, NJ: Lawrence Erlbaum Assoc., 2003, pp. 247-277.
3. P. Smyth, "Clustering sequences with Hidden Markov Models," in Advances in Neural Information Processing Systems-9, M. J. M. C. Mozer and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, vol. 9, pp. 648-654.
4. J. A. Blimes, "A gentle tutorial of the em algorithm and its application to par estimation for Gaussian mixture and hidden markov models,", Univ. California, Berkeley, CA Tech. Rep., 1998.
5. K. Rose, "Deterministic annealing for clustering, compression, classification, regression, and related optimization problems," Proc. IEEE, vol. 86, pp. 2210-39, Nov. 1998.
6. T. S. Jaakkola and D. Haussler, "Exploiting generative models in discriminative classifiers," in Advances in Neural Information Processing Systems-11, M. S. Kearns, S. A. Solla, and D. D. Cohn, Eds. Cambridge, MA: MIT Press, 1999, vol. 11, pp. 487-493.
7. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
8. B. Scholkopf and A. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2001.
9. P. Indyk, "A sublinear-time approximation scheme for clustering in metric spaces," in Proc. 40th Symp. Foundations of Computer Science, 1999, pp. 154-159.
10. R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval. New York: Addison-Wesley, 1999.
11. A. Strehl and J. Ghosh, "Relationship-based clustering and visualization for high-dimensional data mining," INFORMS J. Computing, vol. 15, no. 2, Spring 2003.
12. J. H. Friedman, "An overview of predictive learning and function approximation," in Proc. From Statistics to Neural Networks, NATO/ASI Workshop, V. Cherkassky, J. Friedman, and H. Wechsler, Eds., 1994, pp. 1-61.
13. K. Chang and J. Ghosh, "A unified model for probabilistic principal surfaces," IEEE Trans. Pattern Anal. Machine Intell., vol. 23, pp. 22-41, Jan. 2001.
14. G. Hinton, P. Dayan, and M. Revow, "Modeling the manifolds of images of handwritten digits," IEEE Trans. Neural Networks, vol. 8, pp. 65-74, Jan. 1997.
15. K. V. Mardia, "Statistics of directional data," J. Roy. Statist. Soc. Ser. B (Methodological), vol. 37, no. 3, pp. 349-393, 1975.
16. I. S. Dhillon and D. S. Modha, "Concept decompositions for large sparse text data using clustering," Mach. Learn., vol. 42, no. 1, pp. 143-175, Jan. 2001.
17. I. S. Dhillon, J. Fan, and Y. Guan, "Efficient clustering of very large document collections," in Data Mining for Scientific and Engineering Applications, V. K. R. Grossman, C. Kamath, and R. Namburu, Eds. Norwell, MA: Kluwer, 2001.
18. A. Banerjee and J. Ghosh, "On scaling up balanced clustering algorithms," in Proc. 2nd SIAM Int. Conf. Data Mining, Apr. 2002, pp. 333-349.
19. P. S. Bradley, K. P. Bennett, and A. Demiriz, "Constrained k-means Clustering Microsoft Research, Tech. Rep.,", 2000.
20. S. Grossberg, "Adaptive pattern classification and universal recoding: 1. Parallel development and coding of neural feature detectors," Biol. Cybern., vol. 23, pp. 121-134, 1976.
21. D. E. Rumelhart and D. Zipser, "Feature discovery by competitive learning," Cogn. Sci., vol. 9, pp. 75-112, 1985.
22. S. Grossberg, "Cortical dynamics of three-dimensional form, color and brightness perception, I: Monocular theory," Percep. Psychophys., vol. 41, pp. 87-116, 1987.
23. Y. J. Zhang and Z. Q. Liu, "Self-splitting competitive learning: A new on-line clustering paradigm," IEEE Trans. Neural Networks, vol. 13, pp. 369-380, Mar. 2002.
24. J. C. Bezdek and S. F. Pal, Fuzzy Models for Pattern Recognition. Piscataway, NJ: IEEE Press, 1992.
25. D. DeSieno, "Adding conscience to competitive learning," in IEEE Annu. Int. Conf. Neural Networks, July 1988, pp. 1117-1124.
26. S. C. Ahalt, A. K. Krishnamurthy, P. Chen, and D. E. Melton, "Competitive learning algorithms for vector quantization," Neural Networks, vol. 3, no. 3, pp. 277-290, 1990.
27. T. M. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.
28. M. Kearns, Y. Mansour, and A. Y. Ng, "An information-theoretic analysis of hard and soft assignment methods for clustering," in Proc. Uncertainty Artificial Intelligence, 1997, pp. 282-293.
29. S. Basu, A. Banerjee, and R. Mooney, "Semi-supervised clustering by seeding," in Proc. 19th Int. Conf. Machine Learning, 2002, pp. 19-26.
30. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc. Ser. B (Methodological), vol. 39, no. 1, pp. 1-38, 1977.
31. J. Sinkkonen and S. Kaski, "Clustering based on conditional distributions in an auxiliary space," Neural Computat., vol. 14, pp. 217-239, 2002.
32. A. Banerjee, I. S. Dhillon, J. Ghosh, and S. Sra, Clustering on Hyperspheres Using Expectation Maximization Department of Computer Sciences: Univ. of Texas at Austin, 2003.
33. J. N. Kapur and H. K. Kesavan, Entropy Optimization Principles With Applications. New York: Academic, 1992.
34. N. W. McLachlan, Bessel Functions for Engineers. New York: Oxford Univ. Press, 1955.
35. A. Banerjee and J. Ghosh, "Frequency sensitive competitive learning for clustering on high-dimensional hyperspheres," in Proc. Int. Joint Conf. Neural Networks (JCNN'02), Honolulu, HI, May 2002, pp. 1590-1595.
36. A. S. Galanopoulos and S. C. Ahalt, "Codeword distribution for frequency sensitive competitive learning with one dimensional input data," IEEE Trans. Neural Networks, vol. 7, pp. 752-756, May 1996.
37. A. S. Galanopoulos, R. L. Moses, and S. C. Ahalt, "Diffusion approximation of frequency sensitive competitive learning," IEEE Trans. Neural Networks, vol. 8, pp. 1026-1030, Sept. 1997.
38. V. Ramamurti and J. Ghosh, "On the use of localized gating in mixtures of experts networks," in Proc. SPIE Conf. Applications Science Computational Intelligence, vol. 3390, Orlando, FL, Apr. 1998, pp. 24-35.
39. N. Gershenfeld and A. Weigend, "The future of time series: Learning and understanding," in Time Series Prediction, A. S. Weigend, and N. Gershenfeld, Eds. Reading, MA: Addison-Wesley, 1993, pp. 243-264.
40. B. Widrow and M. A. Lehr, "30 years of adaptive neural networks: Perceptron, madaline and backpropagation," Proc. IEEE, vol. 78, pp. 1415-1442, Sept. 1990.
41. M. C. Mozer, "Neural network architectures for temporal sequence processing," in Time Series Prediction, A. S. Weigend and N. Gershenfeld, Eds. Reading, MA: Addison-Wesley, 1993, pp. 243-264.
42. J.-P.C.S. Dehaene and J.-P. Nadal, "Neural networks that learn temporal sequences by selection," in Proc. Nat. Academy Sciences, vol. 84, May 1987, pp. 2727-2731.
43. B. W. Stiles and J. Ghosh, "A habituation based neural network for spatio-temporal classification," Neurocomputing, vol. 15, pp. 273-303, July 1997.
44. P. Turney, The management of context-sensitive features: A review of strategies. presented at International Conference on Machine Learning (ICML'96) Workshop Learning in Context-Sensitive Domains. [Online]. Available: http://ai.iit.nrc.ca/bibliographies/context-sensitive.html
45. G. Hulten, L. Spencer, and P. Domingos, "Mining time-changing data streams," in Proc. 7th ACM SIGKDD Int. Conf. Knowledge Discovery Data Mining, San Francisco, CA, 2001, pp. 97-106.
46. S. Guha, N. Mishra, R. Motwani, and L. O'Callaghan, "Clustering data streams," in Proc. Annu. Symp. Foundations Computer Science (FOCS 2000), Nov. 2000, pp. 359-367.
47. G. K. Gupta and J. Ghosh, "Detecting seasonal trends and cluster motion visualization for very high dimensional transactional data," in Proc. 1st Siam Conf. Data Mining, (SDM2001), 2001, pp. 115-129.
48. H. G. C. Tråvén, "A neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions," IEEE Trans. Neural Networks, vol. 2, pp. 366-377, May 1991.
49. V. Ramamurti and J. Ghosh, "Structurally adaptive modular networks for nonstationary environments," IEEE Trans. Neural Networks, vol. 10, pp. 152-60, 1999.
50. R. M. Neal and G. E. Hinton, "A view of the EM algorithm that justifies incremental, sparse, and other variants," in Learning in Graphical Models, M. I. Jordan, Ed. Cambridge, MA: MIT Press, 1998, pp. 355-368.
51. J. C. Principe, J.-M. Kuo, and S. Celebi, "An analysis of the gamma memory in dynamic neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 331-337, Mar. 1994.
52. A. Strehl, J. Ghosh, and R. Mooney, "Impact of similarity measures on web-page clustering," in Proc. 17th Nat. Conf. Artificial Intelligence: Workshop Artificial Intelligence Web Search (AAAI 2000), Austin, TX, July 2000, AAAI, pp. 58-64.
53. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
54. B. Widrow and S. Stearns, Adaptive Signal Processing Englewood Cliffs. Englewood Cliffs, NJ: Prentice-Hall, 1985.
55. M. Meila and D. Heckerman, "An Experimental Comparison of Several Clustering and Initialization Methods," Microsoft Research Tech. Rep. MSR-TR-98-06, 1998.
56. H. Akaike, "A new look at statistical model identification," IEEE Trans. Automat. Contr., vol. AU-19, pp. 716-722, Dec. 1974.
57. G. Schwatz, "Estimating the dimension of a model," Ann. Statistics, vol. 6, no. 2, 1978.
58. P. J. Green, "Reversible jump markov chain monte carlo computation and bayesian model determination," Biometrika, vol. 82, pp. 711-732, 1995.