On characterization of entropy function via information inequalities

IEEE Transactions on Information Theory

Volumn 44, Issue 4, 1998, Pages 1440-1452

  Zhang, Zhen ^a   Yeung, Raymond W ^b  

^a University of Southern California ^*  (United States)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Entropy; Inequality; Information measure; Mutual information

Indexed keywords

ENTROPY; FUNCTIONS; PROBABILITY; RANDOM PROCESSES; SET THEORY; VECTORS;

DISCRETE RANDOM VARIABLES; INFORMATION THEORETIC INEQUALITY; SHANNON'S INFORMATION MEASURES;

INFORMATION THEORY;

EID: 0032124111     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.681320     Document Type: Article

References (39)

1. N. M. Abramson, Information Theory and Coding. New York: McGraw-Hill, 1963.
2. L. L. Campbell, "Entropy as a measure," IEEE Trans. Inform. Theory, vol. IT-11, pp. 112-114, Jan. 1965.
3. I. Csiszâr and J. Körner, Information Theory: Coding Theorem for Discrete Memoryless Systems. New York: Academic, and Budapest, Hungary: Akademiai Kiado, 1981.
4. A. P. Dawid, "Conditional independence in statistical theory (with discussion)," J. Roy. Statist. Soc., Ser. B, vol. 41, pp. 1-31.
5. S. Fujishige, "Polymatroidal dependence structure of a set of random variables," Inform. Contr.. vol. 39, pp. 55-72, 1978.
6. T. Kawabata and R. W. Yeung, "The structure of the /-measure of a Markov chain," IEEE Trans. Inform. Theory, vol. 38, pp. 1146-1149, 1992.
7. T. S. Han, "Linear dependence structure of the entropy space," Inform. Contr., vol. 29, pp. 337-368.
8. _, "Nonnegative entropy measures of multivariate symmetric correlations," Inform. Contr., vol. 36, pp. 133-156, 1978.
9. _, "A uniqueness of Shannon's information distance and related nonnegativity problems," J. Comb., Inform. Syst. Sei., vol. 6, pp. 320-321, 1981.
10. G.-d. Hu, "On the amount of information," Tear. Veroyatnost. i Prime-nen., vol. 4, pp. 447-455, 1962, in Russian.
11. F. Mattis, private communication.
12. F. J. MacWilliams and N. J. A. Sloane, The Theory of Error Correcting Codes. Amsterdam, The Netherlands: North-Holland, Elsevier Science B.V., 1977.
13. M. Malus, "Abstract functional dependency structures," Theor. Comput. Sci., vol. 81, pp. 117-126, 1991.
14. _, "On equivalence of Markov properties over undirected graphs," J. Appl. Probab., vol. 29, pp. 745-749, 1992.
15. _, "Ascending and descending conditional independence relations," in Trans, llth Prague Conf. Information Theory, Statistical Decision Functions and Random Processes, vol. B. Prague, Czechoslovakia: Academia, pp. 181-200, 1992.
16. _, "Probabilistic conditional independence structures and matroid theory: Background," Int. J. Gen. Syst., vol. 22, pp. 185-196.
17. _, "Extreme convex set functions with many nonnegative differences," Discr. Math., vol. 135, pp. 177-191, 1994.
18. F. Mattis, "Conditional independences among four random variables II," Combin., Prob. Comput., vol. 4, pp. 407-417, 1995.
19. _, "Conditional independence structures examined via minors," Ann. Math., Artificial Intell, vol. 21, pp. 99-128, 1997.
20. F. Mattis and M. Studeny, "Conditional independences among four random variables I," Combin., Prob. Comput., vol. 4, pp. 269-278, 1995.
21. W. J. McGill, "Multivariate information transmission," in Trans. Prof. Group Inform. Theory, 1954 Symp. Information Theory, vol. PGIT-4, 1955, pp. 93-111.
22. A. Papoulis, Probability, Random Variables and Stochastic Processes, 2nd ed. New York: McGraw-Hill, 1984.
23. J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, CA: Morgan Kaufman, 1988.
24. An Introduction to Information Theory. New York: McGraw-Hill, 1961.
25. M. Studeny, "Attempts at axiomatic description of conditional independence," in Proc. Workshop on Uncertainty Processing in Expert Systems, supplement to Kybernetika, vol. 25, nos. 1-3, pp. 65-72, 1989.
26. _, "Multiinformation and the problem of characterization of conditional independence relations," Probl. Contr. Inform. Theory, vol. 18, pp. 3-16, 1989.
27. _, "Conditional independence relations have no finite complete characterization," in Trans, llth Prague Conf. Information Theory, Statistical Decision Functions and Random Processes, vol. B. Prague, Czechoslovaka: Academia, pp. 377-396, 1992.
28. _, "Structural semigraphoids," Int. J. Gen. Syst., vol. 22, no. 2, pp. 207-217, 1994.
29. _, "Descriptions of structures of stochastic independence by means of faces and imsets (in three parts)," Int. J. Gen. Syst., vol. 23, pp. 123-137, pp. 201-219, pp. 323-341, 1994/1995.
30. T. Tsujishita, "On triple mutual information," Adv. Appl. Math., vol. 16, pp. 269-274, 1995.
31. S. Watanabe, "A study of ergodicity and redundancy on intersymbol correlation of finite range," in Trans. 1954 Symp. Inform. Theory (Cambridge, MA, Sept. 15-17, 1954), p. 85.
32. _, "Information theoretical analysis of multivariate correlation," IBM J., pp. 66-81, 1960.
33. D. J. A. Welsh, Matroid Theory. New York: Academic 1976.
34. R. W. Yeung, "A new outlook on Shannon's information measures," IEEE Trans. Inform. Theory, vol. 37, pp. 466-474, 1991.
35. _, "A framework for linear information inequalities," IEEE Trans. Inform. Theory, vol. 43, pp. 1924-1934, Nov. 1997.
36. R. W. Yeung, T. T. Lee, and Z. Ye, "An information-theoretic characterization of Markov random fields and its applications," IEEE Trans. Inform. Theory, submitted for publication.
37. R. W. Yeung and Z. Zhang, "Distributed source coding for satellite communication," IEEE Trans. Inform. Theory, submitted for publication.
38. R. W. Yeung and Y.-O. Yan, "Information theoretic inequality prover." [Online] Available: http://www.ie.cuhk.edu.hk/ ITIP or http://it.ucsd. eduAverb+~+whyeung(mirrorsite)..
39. Z. Zhang and R. W. Yeung, "A non-Shannon type conditional inequality of information quantities," IEEE Trans. Inform. Theory, vol. 43, pp. 1982-1985, Nov. 1997.