Information cut for clustering using a gradient descent approach

Pattern Recognition

Volumn 40, Issue 3, 2007, Pages 796-806

  Jenssen, Robert ^a   Erdogmus, Deniz ^b   Hild II, Kenneth E ^c   Principe, Jose C ^d   Eltoft, Torbjørn ^a  

^a UNIVERSITY OF TROMSØ   (Norway)

^b OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF FLORIDA   (United States)

Author keywords

Annealing; Clustering; Gradient descent optimization; Graph theoretic cut; Information theory; Parzen window density estimation

Indexed keywords

ALGORITHMS; ANNEALING; COMPUTATIONAL COMPLEXITY; DISTANCE MEASUREMENT; GRADIENT METHODS; IMAGE SEGMENTATION; INFORMATION THEORY; PROBABILITY DENSITY FUNCTION;

CLUSTERING; GRADIENT DESCENT OPTIMIZATION; GRAPH THEORETIC CUT; INFORMATION CUT; PARZEN WINDOW DENSITY ESTIMATION;

GRAPH THEORY;

EID: 33750527085     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2006.06.028     Document Type: Article

References (35)

1. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification and Scene Analysis. second ed. (2001), Wiley, New York
2. Jain A.K., and Dubes R.C. Algorithms for Clustering Data (1988), Prentice-Hall, Englewood Cliffs, NJ
3. Theodoridis S., and Koutroumbas K. Pattern Recognition (1999), Academic Press, San Diego
4. Wu Z., and Leahy R. An optimal graph theoretic approach to data clustering: theory and its applications to image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 15 11 (1993) 1101-1113
5. Shi J., and Malik J. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22 8 (2000) 888-905
6. Ding C.H.Q., He X., Zha H., Gu M., and Simon H.D. A min-max cut algorithm for graph partitioning and data clustering. Proceedings of IEEE International Conference on Data Mining. San Jose, USA (November 29-December 2, 2001) 107-114
7. Gdalyahu Y., Weinshall D., and Werman M. Self-organization in vision: stochastic clustering for image segmentation, perceptual grouping, and image database organization. IEEE Trans. Pattern Anal. Mach. Intell. 23 10 (2001) 1053-1074
8. Scanlon J., and Deo N. Graph-theoretic algorithms for image segmentation. IEEE International Symposium on Circuits and Systems. Orlando, Florida (1999) VI141-VI144
9. Ng A.Y., Jordan M., and Weiss Y. On spectral clustering: analysis and an algorithm. Advances in Neural Information Processing Systems vol. 14 (2002), MIT Press, Cambridge 849-856
10. Weiss Y. Segmentation using eigenvectors. a unifying view. Proceedings of IEEE International Conference on Computer Vision. Corfu, Greece (September 20-25, 1999) 975-982
11. Kannan R., Vempala S., and Vetta A. On clusterings. good, bad and spectral. Proceedings of IEEE Symposium on Foundations of Computer Science. Redondo Beach, USA (November 12-14, 2000) 367-377
12. Alpert C., and Yao S. Spectral partitioning. the more eigenvectors the better. Proceedings of ACM/IEEE Design Automation Conference. San Francisco, USA (June 12-16, 1995)
13. Azar Y., Fiat A., Karlin A., McSherry F., and Saia J. Spectral analysis of data. Proceedings of ACM Symposium on Theory of Computing. Heraklion, Greece (June 6-8, 2001) 619-626
14. Scott G., and Longuet-Higgins H. Feature grouping by relocalisation of eigenvectors of the proximity matrix. Proceedings of British Machine Vision Conference. Oxford, UK (September 24-27, 1990) 103-108
15. D.J. Higham, M. Kibble, A unified view of spectral clustering, Technical Report 2, Department of Mathematics, University of Strathclyde, January 2004.
16. Jenssen R., Eltoft T., and Principe J.C. Information theoretic spectral clustering. Proceedings of International Joint Conference on Neural Networks. Budapest, Hungary (July 25-29, 2004), Avon Books, New York 111-116
17. Gokcay E., and Principe J. Information theoretic clustering. IEEE Trans. Pattern Anal. Mach. Intell. 24 2 (2002) 158-170
18. Watanabe S. Pattern Recognition: Human and Mechanical (1985), Wiley, New York
19. Rose K., Gurewitz E., and Fox G.C. Vector quantization by deterministic annealing. IEEE Trans. Information Theory 38 4 (1992) 1249-1257
20. Hofmann T., and Buhmann J.M. Pairwise data clustering by deterministic annealing. IEEE Trans. Pattern Anal. Mach. Intell. 19 1 (1997) 1-14
21. Roberts S.J., Everson R., and Rezek I. Maximum certainty data partitioning. Pattern Recognition 33 (2000) 833-839
22. Tishby N., and Slonim N. Data clustering by Markovian relaxation and the information bottleneck method. Advances in Neural Information Processing Systems vol. 13 (2001), MIT Press, Cambridge 640-646
23. Jenssen R., Principe J.C., and Eltoft T. Information cut and information forces for clustering. Proceedings of IEEE International Workshop on Neural Networks for Signal Processing. Toulouse, France, September 17-19 (2003) 459-468
24. R. Jenssen, D. Erdogmus, K.E. Hild, J.C. Principe, T. Eltoft, Optimizing the Cauchy-Schwarz PDF distance for information theoretic, non-parametric clustering, in: Proceedings of International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, St. Augustine, USA, November 9-11, 2005, pp. 34-45.
25. Principe J., Xu D., and Fisher J. Information theoretic learning. In: Haykin S. (Ed). Unsupervised Adaptive Filtering vol. I (2000), Wiley, New York (Chapter 7)
26. Parzen E. On the estimation of a probability density function and the mode. Ann. Math. Statist. 32 (1962) 1065-1076
27. Silverman B.W. Density Estimation for Statistics and Data Analysis (1986), Chapman & Hall, London
28. Scott D.W. Multivariate Density Estimation (1992), Wiley, New York
29. Wand M.P., and Jones M.C. Kernel Smooting (1995), Chapman & Hall, London
30. Rao S.S. Engineering Optimization: Theory and Practice (1996), Wiley, New York
31. Bezdek J.C. A convergence theorem for the fuzzy isodata clustering algorithms. IEEE Trans. Pattern Anal. Mach. Learn. 2 1 (1980) 1-8
32. MacQueen J. Some methods for classification and analysis of multivariate observations. Proceedings of Berkeley Symposium on Mathematical Statistics and Probability (1967), University of California Press, Berkeley 281-297
33. R. Murphy, D. Ada, UCI repository of machine learning databases, Technical Report, Department of Computer Science, University of California, Irvine, 1994.
34. Jain A.K., and Farrokhnia F. Unsupervised texture segmentation using Gabor filters. Pattern Recognition 24 12 (1991) 1167-1186
35. Elgammal A., Duraiswami R., and Davis L. The fast Gauss transform for efficient kernel density evaluation with applications in computer vision. IEEE Trans. Pattern Anal. Mach. Intell. 25 (2003) 1499-1504