Deformed statistics formulation of the information bottleneck method

IEEE International Symposium on Information Theory - Proceedings

Volumn , Issue , 2009, Pages 1323-1327

  Venkatesan, R C ^a   Plastina, A ^b  

^a Systems Research Corporation   (India)

^b UNIVERSIDAD NACIONAL DE LA PLATA   (Argentina)

Author keywords

[No Author keywords available]

Indexed keywords

APRIORI; DISTORTION MEASURES; GENERALIZED KULLBACK-LEIBLER DIVERGENCE; INFORMATION BOTTLENECK; LAGRANGIAN; NON-ADDITIVE; NONADDITIVITY; SOURCE ALPHABET; THEORETICAL BASIS; TSALLIS ENTROPIES; TSALLIS STATISTICS; VARIATIONAL PRINCIPLES;

ENCODING (SYMBOLS); VARIATIONAL TECHNIQUES;

INFORMATION THEORY;

EID: 70449517110     PISSN: 21578102     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISIT.2009.5205932     Document Type: Conference Paper

References (25)

1. T. Cover and J. Thomas Elements of Information Theory, John Wiley & Sons, New York, NY, 1991.
2. T. Berger T, Rate Distortion Theory, Prentice-Hall, Englewood Cliffs, 1971.
3. N. Tishby N, F. C. Pereira and W. Bialek, "The information bottleneck method", Proceedings of the 37th Annual Allerton Conference on Communication Control and Computing Eds B Hajek and R S Sreenivas 368, University of Illinois, Urbana, IL, 1999.
4. N. Slonim The Information Bottleneck: Theory and Applications PhD thesis, Hebrew University, Jerusalem, 2003.
5. C. Tsallis "Possible generalizations of Boltzmann-Gibbs statistics" J. Stat. Phys. 542 479, 1988.
6. M. Gell-Mann and C. Tsallis, Eds. Nonextensive Entropy- Interdisciplinary Applications, Oxford University Press, Oxford, 2004.
7. R. C. Venkatesan and A. Plastino "Generalized statistics framework for rate distortion theory", Physica A,388, 12, 2337, 2009.
8. P.T. Landsberg and V. Vedral, "Distributions and channel capacities in generalized statistical mechanics", Phys. Lett. A, 247, 211, 1998.
9. T. Yamano, "Information theory based on nonadditive information content", Phys. Rev. E, 63, 046105, 2001.
10. S. Furuichi, "Information theoretical properties on Tsallis entropies", J. Math. Phys., 47, 023302, 2006.
11. H. Suyari, "Source coding theorem based on a nonadditive information content", IEEE Trans. on Inform. Theory, 50,8, 1783, 2004.
12. T. Yamano, "Source coding theorem based on a nonadditive information content", Physica A, 305, 190 (2002).
13. G. L. Ferri, S. Martinez S and A. Plastino A "Equivalence of the four versions of Tsallis statistics" J. Stat. Mech.: Theory and Experiment 2005(04) P04009, 2005.
14. E. M. F. Curado and C. Tsallis "Generalized statistical mechanics: connection with thermodynamics" J. Phys. A: Math Gen. 24 L69, 1991.
15. C. Tsallis, R. S. Mendes and A. R. Plastino "The role of constraints within generalized nonextensive statistics" Physica A 261 534, 1998.
16. S. Martinez, F. Nicolás, F. Pennini and A. Plastino "Generalized statistical mechanics: connection with thermodynamics" Physica A 286 489, 2000.
17. J. Naudts "Deformed exponentials and logarithms in generalized thermostatistics" Physica A 340 32, 2004.
18. W. Bialek , I. Nemenman I and N. Tishby, "Predictability, Complexity, and Learning Predictability, complexity, and learning", Neural Comp. 13 2409, 2001.
19. C. Tsallis "Generalized entropy-based criterion for consistent testing" Phys. Rev. E 58 1442, 1998.
20. L. Borland, A. Plastino and C. Tsallis "Information gain within nonextensive thermostatistics" J. Math. Phys. 39 6490, 1998.
21. E. Borges "A possible deformed algebra and calculus inspired in nonextensive thermostatistics" Physica A 340 95, 2004.
22. G. J. McLachlan and T. Krishnan The EM Algorithm and Extensions, John Wiley & Sons, New York, NY, 1996.
23. I. Csiszár and G. Tusnády "Information geometry and alternating minimization procedures" Statistics and Decisions 1 205, 1984.
24. S. Furuichi "On uniqueness theorems for Tsallis entropy and Tsallis relative entropy" IEEE Trans Inform. Theory 51 3638, 2005.
25. P. Harremöes and N. Tishby "The information bottleneck revisited or how to choose a good distortion measure" Proceedings of the IEEE Int. Symp. on Information Theory 2007 566, 2007.