Normalized information-based divergences

Problems of Information Transmission

Volumn 43, Issue 3, 2007, Pages 167-189

  Coeurjolly, Jean Francois ^a   Drouilhet, Remy ^a   Robineau, J F ^a  

^a UNIV GRENOBLE ALPES   (France)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; INFORMATION THEORY; MATHEMATICAL TECHNIQUES; RANDOM PROCESSES; VECTORS;

CATEGORICAL RANDOM VECTORS; INFORMATION-BASED DIVERGENCE; MATHEMATICAL PROPERTIES; MUTUAL INFORMATION;

DATA PROCESSING;

EID: 35848970975     PISSN: 00329460     EISSN: 16083253     Source Type: Journal    
DOI: 10.1134/S0032946007030015     Document Type: Article

References (21)

1. Shannon, C.E., A Mathematical Theory of Communication (continued), Bell Syst. Tech. J., 1948, vol. 27, no. 4, pp. 623-656.
2. Crutchfield J.P., Information and Its Metric, Nonlinear Structures in Physical Systems: Pattern Formation, Chaos, and Waves (Proc. 2nd Woodward Conf., San Jose State Univ., 1989), Lam, L. and Morris, H.C., Eds., New York: Springer, 1990, pp. 119-130.
3. Hillman, C., A Formal Theory of Information: I. Statics, Preprint, 1998. Available from http://citeseer.ist.psu.edu/89520.html.
4. Bennett, C.H., Gács, P., Li, M., Vitányi, P.M.B., and Zurek, W.H., Information Distance, IEEE Trans. Inform. Theory, 1998, vol. 44, no. 4, pp. 1407-1423.
5. Li, M., Chen, X., Li, X., Ma, B., and Vitányi, P.M.B., The Similarity Metric, IEEE Trans. Inform. Theory, 2004, vol. 50, no. 12, pp. 3250-3264.
6. Cilibrasi, R. and Vitányi, P.M.B., Clustering by Compression, IEEE Trans. Inform. Theory, 2005, vol. 51, no. 4, pp. 1523-1545.
7. Li, M., Badger, J.H., Chen, X., Kwong, S., Kearney, P., and Zhang, H., An Information-Based Sequence Distance and Its Application to Whole Mitochondrial Genome Phylogeny, Bioinformatics, 2001, vol. 17, no. 1, pp. 149-154.
8. Cilibrasi, R., Vitányi, P., and de Wolf, R., Algorithmic Clustering of Music, 2003, e-print arxiv.org/cs.SD/0303025.
9. Cilibrasi, R. and Vitányi, P., Automatic Meaning Discovery Using Google, 2005, e-print arxiv.org/cs.CL/0412098.
10. Grünwald, P. and Vitányi, P., Shannon Information and Kolmogorov Complexity, 2004, e-print arxiv.org/cs.IT/0410002.
11. Leung-Yan-Cheong, S.K. and Cover, T.M., Some Equivalences between Shannon Entropy and Kolmogorov Complexity, IEEE Trans. on Inf. Theory, 1978, vol. 24, no. 3, pp. 331-338.
12. Hammer, D., Romashchenko, A., Shen, A., and Vereshchagin, N., Inequalities for Shannon Entropy and Kolmogorov Complexity, J. Comput. Syst. Sci., 2000, vol. 60, no. 2, pp. 442-464.
13. Kraskov, A., Stögbauer, H., Andrzejak, R.G., and Grassberger, P., Hierarchical Clustering Based on Mutual Information, 2003, e-print arxiv.org/q-bio/0311039.
14. Liu, H. and Motoda, H., Feature Selection for Knowledge Discovery and Data Mining, Boston: Kluwer, 1998.
15. Robineau, J.-F., Méthodes de sélection de variables (parmi un grand nombre) dans un cadre de discrimination, PhD Thesis, Grenoble, France: Univ. Joseph Fourier, 2004.
16. Granger, C.W., Maasoumi, E., and Racine, J., A Dependence Metric for Possibly Nonlinear Processes, J. Time Ser. Anal., 2004, vol. 25, no. 5, pp. 649-669.
17. Cover, T.M. and Thomas, J.A., Elements of Information Theory, New York: Wiley, 1991.
18. Kaltchenko, A., Algorithms for Estimation of Information Distance with Application to Bioinformatics and Linguistics, in Proc. 2004 Canadian Conf. on Electrical and Computer Engineering, 2004, vol. 4, p. 2255. Available from arxiv.org/cs.CC/0404039.
19. Li, M. and Vitányi, P., An Introduction to Kolmogorov Complexity and Its Applications, New York: Springer, 1997, 2nd ed.
20. Li, M, Chen, X., Li, X., Ma, B., and Vitányi, P., The Similarity Metric, in Proc. 14th Annual ACMSIAM Symp. on Discrete Algorithms (SODA-03), New York, 2003, pp. 863-872.
21. Ullah, A., Entropy, Divergence and Distance Measures with Econometric Applications, J. Stat. Plan. Inference, 1996, vol. 49, no. 1, pp. 137-162.