Divergence from factorizable distributions and matroid representations by partitions

IEEE Transactions on Information Theory

Volumn 55, Issue 12, 2009, Pages 5375-5381

  Matúš, František ^a  

^a INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

Author keywords

Almost affine code; Contingency table; Exponential family; Gibbs distribution; Hierarchical model; Information divergence; Log linear model; Matroid representation; Maximum likelihood; Partition; Relative entropy; Secret sharing scheme; Shannon entropy

Indexed keywords

ALMOST AFFINE CODE; CONTINGENCY TABLE; EXPONENTIAL FAMILY; GIBBS DISTRIBUTION; HIERARCHICAL MODEL; INFORMATION DIVERGENCE; LOGLINEAR MODEL; MATROID REPRESENTATION; PARTITION; RELATIVE ENTROPY; SHANNON ENTROPY; SHARING SCHEMES;

ENTROPY; HIERARCHICAL SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION; REGRESSION ANALYSIS;

MAXIMUM LIKELIHOOD;

EID: 77955627631     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2032806     Document Type: Article

References (20)

1. N. Ay, "An information-geometric approach to a theory of pragmatic structuring," Ann. Probab., vol.30, pp. 416-436, 2002.
2. N. Ay and A. Knauf, "Maximizing multi-information," Kybernetika, vol.45, pp. 517-538, 2006.
3. O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory. New York: Wiley, 1978.
4. L. D. Brown, Fundamentals of Statistical Exponential Families, ser. Lecture Notes-Monograph Series. Beechwood, OH: Inst. Math. Statist., 1986, vol.9.
5. N. N. Chentsov, Statistical Decision Rules and Optimal Inference, ser. Translations of Mathematical Monographs. Providence, RI: Amer. Math. Soc., 1982, Russian original published by: Moscow, U.S.S.R.: Nauka, 1972.
6. I. Csiszár and F. Matúš, "Information projections revisited," IEEE Trans. Inf. Theory, vol.49, no.6, pp. 1474-1490, Jun. 2003.
7. I. Csiszár and F. Matúš, "Generalized maximum likelihood estimates for exponential families," Probab. Theory Related Fields, vol.141, pp. 213-246, 2008.
8. I. Csiszár and P. C. Shields, "Information theory and statistics: A tutorial," Foundations and Trends in Commun. Inf. Theory, vol.1, no.4, pp. 417-528, 2004.
9. D. Geiger, C. Meek, and B. Sturmfels, "On the toric algebra of graphical models," Ann. Statist., vol.34, pp. 1463-1492, 2006.
10. S. J. Haberman, The Analysis of Frequency Data. Chicago, IL: Univ. Chicago Press, 1974.
11. S. Della Pietra, V. Della Pietra, and J. Lafferty, "Inducing features of random fields," IEEE Trans. Pattern Anal. Machine Intell., vol.19, no.4, pp. 380-393, Apr. 1997.
12. F. Matúš, "Probabilistic conditional independence structures and matroid theory: Background," Int. J. Gen. Syst., vol.22, pp. 185-196, 1994.
13. F. Matúš, "Matroid representations by partitions," Discr. Math., vol.203, pp. 169-194, 1999.
14. F. Matúš and N. Ay, "On maximization of the information divergence from an exponential family," in Proc. WUPES'03, J. Vejnarová, Ed., Univ. Economics, Prague, Czech Republic, 2003, pp. 199-204, .
15. F. Matúš, "Maximization of information divergences from binary i.i.d. sequences," in Proc. IPMU 2004, Perugia, Italy, 2004, vol.2, pp. 1303-1306.
16. F. Matúš, "Two constructions on limits of entropy functions," IEEE Trans. Inf. Theory, vol.53, no.1, pp. 320-330, jan. 2007.
17. F. Matúš, "Optimality conditions for maximizers of the information divergence from an exponential family," Kybernetika, vol.43, pp. 731-746, 2007.
18. D. J. A. Welsh, Matroid Theory. London, U.K.: Academic, 1976.
19. J. G. Oxley, Matroid Theory. Oxford, New York, Tokyo: Oxford Univ. Press, 1992.
20. R. W. Yeung, A First Course in Information Theory. New York: Kluwer Academic/Plenum, 2002.