The information bottleneck revisited or how to choose a good distortion measure

IEEE International Symposium on Information Theory - Proceedings

Volumn , Issue , 2007, Pages 566-570

  Harremoës, Peter ^a   Tishby, Naftali ^b  

^a CWI   (Netherlands)

^b HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION BOTTLENECKS;

CYBERNETICS; ENCODING (SYMBOLS); IMAGE CODING; INFORMATION THEORY; SIGNAL DISTORTION; TECHNICAL PRESENTATIONS;

ELECTRIC DISTORTION;

EID: 51649124977     PISSN: 21578101     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISIT.2007.4557285     Document Type: Conference Paper

References (16)

1. N. Tishby, F. Pereira, and W. Bialek, "The information bottleneck method," in Proceedings of the 37-th Annual Allerton Conference on Communication, Control and Computing, pp. 368-377, 1999.
2. N. Slonim and N. Tishby, "Document clustering using word clusters via the information bottleneck method," in Proceedings of the ACM SIGIR conference, (New York, NY, USA), pp. 208-215, ACM Press, 2000.
3. G. Chechik, A. Globerson, N. Tishby, and Y. Weiss, "Information bottleneck for Gaussian variables," in Advances in Neural Information Processing Systems 16 (S. Thrun, L. Saul, and B. Schölkopf, eds.), Cambridge, MA: MIT Press, 2004.
4. R. Gilad-Bachrach, A. Navot, and N. Tishby, "A study of the information bottleneck for method and ist relationship to classical problems," tech. rep., HUH Leibniz, 2006.
5. C. R. Shalizi and J. P. Crutchfield, "Information bottlenecks, causal states, and statistical relevance bases: How to represent relevant information in memoryless transduction," tech. rep., Santa Fe Institute, 2002.
6. N. Slonim and Y. Weiss, "Maximum likelihood and the information bottleneck," in Advances in Neural Information Processing Systems 15, Vancouver, Canada, 2002 (S. Becker, S. Thrun, andK. Obermayer, eds.), pp. 351-358, 2003.
7. G. Elidan and N. Friedman, "The information bottleneck EM algorithm," in Proceedings of the 19th Annual Conference on Uncertainty in Artificial Intelligence (UAI-03), (San Francisco, CA), pp. 200-208, Morgan Kaufmann. 2003.
8. D. Gondek and T. Hofmann, "Conditional information bottleneck clustering," in Workshop on Clustering Large Data Sets, IEEE International Conference on Data Mining, Melbourne, USA, November 19-22, 2003., 2003.
9. T. Cover and J. A. Thomas, Elements of Information Theory. Wiley, 1991.
10. P. D. Grünwald and A. P. Dawid, "Game theory, maximum entropy, minimum discrepancy, and robust Bayesian decision theory," Annals of Mathematical Statistics, vol. 32, no. 4, pp. 1367-1433, 2004.
11. I. Csiszár and P. Shields, Information Theory and Statistics: A Tutorial. Foundations and Trends in Communications and Information Theory, Now Publishers Inc., 2004.
12. F. Liese and I. Vajda, "On divergence and informations in statistics and information theory," IEEE Tranns. Inform. Theory, vol. 52, pp. 4394 - 4412, Oct. 2006.
13. M. C. Pardo and I. Vajda, "About distances of discrete distributions satisfying the data processing theorem of information theory," IEEE Trans. Inform. Theory, vol. 43, pp. 1288-1293, 1997.
14. L. M. Bregman, "The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming," USSR Comput. Math. and Math. Phys., vol. 7, pp. 200-217, 1967. Translated from Russian.
15. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman divergences," Journal of Machine Learning Research, vol. 6, pp. 1705-1749, 2005.
16. F. Topsøe, "An information theoretical identity and a problem involving capacity," Studia Scientiarum Mathematicarum Hungarica, vol. 2, pp. 291-292, 1967.