Pairwise data clustering by deterministic annealing

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 19, Issue 1, 1997, Pages 1-14

  Hofmann, Thomas ^a   Buhmann, Joachim M ^a  

^a UNIVERSITY OF BONN   (Germany)

Author keywords

Please leave no spaces around the dash and delete this text

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMBINATORIAL MATHEMATICS; DATA PROCESSING; ESTIMATION; HEURISTIC METHODS; IMAGE PROCESSING; INFERENCE ENGINES; OPTIMIZATION; PROBABILITY; VARIATIONAL TECHNIQUES; VECTORS;

COMBINATORIAL OPTIMIZATION; DETERMINISTIC ANNEALING; MAXIMUM ENTROPY; PAIRWISE DATA CLUSTERING;

PATTERN RECOGNITION;

EID: 0030734483     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/34.566806     Document Type: Article

References (47)

1. E.T. Jaynes, "Information Theory and Statistical Mechanics," Physical Review, vol. 106, pp. 620-630,1957.
2. E.T. Jaynes, "Information Theory and Statistical Mechanics II," Physical Review, vol. 108, pp. 171-190,1957.
3. E.T. Jaynes, "On the Rationale of Maximum-Entropy Methods," Proc IEEE, vol. 70, pp. 939-952,1982.
4. A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, NJ: Prentice Hall, 1988.
5. G.J. McLachlan and K.E. Basford, Mixture Models. New York, Basel: Marcel Dekker, Inc., 1988.
6. R.M. Gray, "Vector Quantization," JEEE Acoustics, Speech and Signal Processing, pp. 4-29, Apr. 1984.
7. A. Gersho and R.M. Gray, Vector Quantization and Signal Processing. Boston: Kluwer Academic Publisher, 1992.
8. R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
9. P. Simic, "Statistical Mechanics as the Underlying Theory of "Elastic" and "Neural" Optimizations," Network, vol. 1, pp. 89-103,1990.
10. P. Simic, "Constrained Nets for Graph Matching and Other Quadratic Assignment Problems," Neural Computation, vol. 3, pp. 268-281,1991.
11. A. Yuille, P. Stolorz, and J. Utans, "Statistical Physics, Mixtures of Distributions and the EM Algorithm," Neural Computation, vol. 6, pp. 334-340,1994.
12. S. Gold and A. Rangarajan, "A Graduated Assignment Algorithm for Graph Matching," 7ÉEE Trans Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 377-388,1996.
13. D. Geiger and F. Girosi, "Parallel and Deterministic Algorithms from MRFs: Surface Reconstruction," IEEE Trans Pattern Analysis and Machine Intelligence, vol. 13, pp. 401-412, May 1991.
14. A.L. Yuille, "Generalized Deformable Models, Statistical Physics and Matching Problems," Neural Computation, vol. 2, no. 1, pp. 124,1990.
15. C. Bregler and S. Omohundro, "Surface Learning with Applications to Lipreading," Advances in Neural Information Processing Systems vol. 6, J. Cowan, G. Tesauro, and J. Alspector, eds., 1994.
16. K. Rose, E. Gurewitz, and G. Fox, "Statistical Mechanics and Phase Transitions in Clustering," Physical Review Letters, vol. 65, no. 8, pp. 945-948,1990.
17. K. Rose, E. Gurewitz, and G. Fox, "A Deterministic Annealing Approach to Clustering," Pattern Recognition Letters, vol. 11, no. 11, pp. 589-594,1990.
18. K. Rose, E. Gurewitz, and G. Fox, "Vector Quantization by Deterministic Annealing,"7EEE Trans Information Theory, vol. 38, no. 4, pp. 1249-1257,1992.
19. K. Rose, E. Gurewitz, and G. Fox, "Constrained Clustering as an Optimization Method," IEEE Trans Pattern Analysis and Machine Intelligence, vol. 15, no. 8, pp. 785-794,1993.
20. J.M. Buhmann and H. Kühnel, "Complexity Optimized Data Clustering by Competitive Neural Networks," Neural Computation, vol. 5, pp. 75-88,1993.
21. J.M. Buhmann and H. Kühnel, "Vector Quantization with Complexity Costs," IEEE Trans Information Theory, vol. 39, pp. 1,1331,145, July 1993.
22. P.A. Chou, T. Lookabaugh, and R.M. Gray, "Entropy-Constrained Vector Quantization," IEEE Trans Acoustics, Speech and Signal Processing, vol. 37, pp. 31 -42,1989.
23. S. Kirkpatrick, C. Gelatt, and M. Vecchi, "Optimization by Simulated Annealing," Science, vol. 220, pp. 671-680,1983.
24. V. Cerny, "Thermodynamical Approach to the Traveling Salesman Problem: an Efficient Simulation Algorithm," J. Optimization Theory and Applications, vol. 45, pp. 41-51,1985.
25. C.W. Gardiner, Handbook of Stochastic Methods. Berlin:Springer, 1983.
26. Y. Tikochinsky, N. Tishby, and R.D. Levine, "Alternative Approach to Maximum-Entropy Inference," Physical Review A, vol. 30, pp. 2638-2644,1984.
27. I. Csiszár, "I-Divergence, Geometry of Probability Distributions and Minimization Problems," Annals Of Probability, vol. 3, pp. 146-158,1975.
28. T. Kohonen, Self-Organization and Associative Memory. Berlin:Springer, 1984.
29. H. Ritter, T. Martinetz, and K. Schulten, Neural Computation and Self-Organizing Maps. New York: Addison Wesley, 1992.
30. T.M. Cover and J.A. Thomas, Elements of Information Theory. New York: John Wiley and Sons, 1991.
31. A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Society Ser. B (methodological), vol. 39, pp. 1-38,1977.
32. T. Hofmann, J. Puzicha, and J. M. Buhmann, "Unsupervised Segmentation of Textured Images by Pairwise Data Clustering," Technical Report IAI-TR-96-2, Rheinische Friedrich-WilhelmsUniversität Bonn, Institut für Informatik III, Feb. 1996.
33. M. Mezard, G. Parisi, and M.A. Virasoro, Spin Glass Theory and Beyond. Singapore: World Scientific, 1987.
34. R.E. Peierls, "On a Minimum Property of the Free Energy," Physical Review, vol. 54, p. 918,1938.
35. D.J. Thouless, P.W. Anderson, and R.G. Palmer, "A Solution to a "Solvable" Model of a Spin Glass," Philosophical Magazine, vol. 35, p. 593,1977.
36. J.W. Sammon Jr, "A Non-Linear Mapping for Data Structure Analysis," IEEE Trans. Computers, vol. 18, pp. 401-409,1969.
37. J.B. Kruskal, "Nonmetric Multidimensional Scaling: a Numerical Method," Psychometrika, vol. 29, pp. 115-129,1964.
38. P. Huber, "Projection Pursuit," Annals of Statistics, vol. 13, pp. 435475,1985.
39. T. Hastie and W. Stuetzle, "Principal Curves," J. American Statistical Ass'n, vol. 84, pp. 502-516,1989.
40. D. Bavelier and M. Jordan, "Representing Words in Connectionist Models." 34th Ann. Meeting Psychonomics Society, Washington, DC., 1993.
41. D. Geman, S. Geman, C. Graffigne, and P. Dong, "Boundary Detection by Constrained Optimization," /EEE Trans Pattern Analysis and Machine Intelligence, vol. 12, pp. 609-628, July 1990.
42. I. Fogel and D. Sagi, "Gabor Filters as Texture Discriminators," Biological Cybernetics, vol. 61, pp. 103-113,1989.
43. J. Daugman, "Uncertainty Relation for Resolution in Space, Spatial Frequency, and Orientation Optimized by Two-Dimensional Visual Cortical Filters," J. Optical Society of America A, vol. 2, no. 7, pp. 1,160-1,169,1985.
44. B. Julesz, "Visual Pattern Discrimination," IRE Transactions on Information Theory, pp. 84-92, Feb. 1961.
45. T. Hofmann, J. Puzicha, and J.M. Buhmann, "Unsupervised Segmentation of Textured Images by Pairwise Data Clustering," Proc. Int'l Conf. Image Processing Lausanne, 1996.
46. T. Hofmann and J.M. Buhmann, "Infering Hierarchical Clustering Structures by Deterministic Annealing," Proc. Knowledge Discovery and Data Mining Conf., Portland, 1996.
47. G. Parisi, Statistical Field Theory. Redwood City, Calif.: Addison Wesley, 1988.