Some properties of Rényi entropy and Rényi entropy rate

Information Sciences

Volumn 179, Issue 14, 2009, Pages 2426-2433

  Golshani, Leila ^a   Pasha, Einollah ^b   Yari, Gholamhossein ^c  

^a ISLAMIC AZAD UNIVERSITY   (Iran)

^b KHARAZMI UNIVERSITY   (Iran)

^c IRAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Iran)

Author keywords

Quasi linear mean; R nyi entropy; R nyi entropy rate; Shannon entropy; Shannon entropy rate; Stationary process

Indexed keywords

CHAIN RULES; MARKOV CHAINS; QUASI-LINEAR MEAN; SHANNON ENTROPY; SHANNON ENTROPY RATE; STATIONARY PROCESS;

MARKOV PROCESSES; SIGNAL THEORY;

ENTROPY;

EID: 67349225775     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2009.03.002     Document Type: Article

References (34)

1. Andai A. On the geometry of generalized Gaussian distributions. Journal of Multivariate Analysis 100 (2009) 777-793
2. Bentes S.R., Menezes R., and Mendes D.A. Long memory and volatility clustering: is the empirical evidence consistent across stock markets?. Physica A 387 (2008) 3826-3830
3. Bercher J.F. On some entropy functionals derived from Rényi information divergence. Information Sciences 178 (2008) 2489-2506
4. C. Cachin, Entropy measures and unconditional security in cryptography, PhD Thesis, Swiss Federal Institute of Technology Zurich, 1997.
5. Csiszár I. Generalized cutoff rates and Rényi's information measures. IEEE Transactions on Information Theory 41 (1995) 26-34
6. Cover T.M., and Thomas J. The Elements of Information Theory (1991), John Wiley and Sons
7. De Gregorio A., and lacus S.M. On Rényi information for ergodic diffusion processes. Information Sciences (2009) 279-291
8. Dukkipati A., Bhatnagar S., and Murty M.N. Gelfand-Yaglom-Perez theorem for generalized relative entropy functionals. Information Sciences 177 (2007) 5707-5714
9. Farhadi A., and Charalambous C.D. Robust codind for a class of sources: applications in control and reliable communication over limited capacity channels. Systems and Control Letters 57 (2008) 1005-1012
10. Guiasu S. Information Theory with Applications (1977), McGraw-Hill, Inc.
11. N.J.A. Harvey, K. Onak, J. Nelson, Streaming algorithms for estimating entropy, in: IEEE Information Theory Workshop, 2008, pp. 227-231.
12. Jacquet P., Seroussi G., and Szpankowski W. On the entropy of hidden Markov process. Theoretical Computer Science 395 (2008) 203-219
13. Jenssen R., and Eltoft T. A new information theoretic analysis of sum-of-squared-error kernel clustering. Neurocomputing 72 (2008) 23-31
14. Jizba P., and Arimitsu T. The world according to Rényi: thermodynamics of multifractal systems. Annals of Physics 312 (2004) 17-59
15. Kanaya F., and Han T.S. The asymptotics of posterior entropy and error probability for Bayesian estimation. IEEE Transactions on Information Theory 41 (1995) 1988-1992
16. Khinchin A.I. Mathematical Foundations of Information Theory (1957), Dover Publications, New York
17. Kirchanov V.S. Using the Rényi entropy to describe quantum dissipative systems in statistical mechanics. Theoretical and Mathematical Physics 156 (2008) 1347-1355
18. Klimko E.M., and Sucheston L. On convergence of information in spaces with infinite invariant measure. Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete 10 (1968) 226-235
19. Kolmogorov A.N. Sur la notion de la moyenne. Atti della Accademia Nazionale dei Lincei Rend 12 6 (1930) 388-391
20. Lake D.E. Rényi entropy measures of heart rate gaussianity. IEEE Transactions on Biomedical Engineering 53 (2006) 21-27
21. Nagumo M. Uber eine klasse der mittelwerte. Japanese Journal of Mathematics 7 (1930) 71-79
22. Nilsson M., and Kleijn W.B. On the estimation of differential entropy from data located on embedded manifolds. IEEE Transactions on Information Theory 53 (2007) 2330-2341
23. Paris J.B., and Rad S.R. Inference processes for quantified predicate knowledge. Logic, Language, Information and Computation 5110 (2008) 249-259
24. E. Pasha, L. Golshani, The Rényi entropy rate for Markov chains with countable state space, Bulletin of the Iranian Mathematical Society (submitted for publication).
25. Z. Rached, A. Fady, L.L. Campbell, Rényi's entropy rate for discrete Markov sources, in: Proceedings of the CISS'99, Baltimore, MD, March 1999, pp. 17-19.
26. Rached Z., Fady A., and Campbell L.L. Rényi's divergence and entropy rate for finite alphabet Markov sources, sources. IEEE Transactions on Information Theory 47 (2001) 1553-1560
27. Rényi A. On measures of entropy and information. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics Probability vol. 1 (1961), Univ. Calif. Press, Berkeley, CA 547-561
28. Rényi A. Probability Theory (1970), North-Holland, Amsterdam
29. Seneta E. Nonnegative Matrix and Markov Chains (1981), Springer-Verlag, New York
30. Shannon C.E. A mathematical theory of communication. Bell System Technical Journal 27 (1948) 379-423 623-656
31. H. Shimokawa, Rényi's entropy and error exponent of source coding with countably infinite alphabet, in: IEEE International Symposium on Information Theory, vol. 1- 6, 2006, pp. 1831-1835.
32. Vollbrecht K.G.H., and Wolf M.M. Conditional entropies and their relation to entanglement criteria. Journal of Mathematical Physics 43 (2002) 4299-4306
33. Wimmer H.K. Spectral radius and radius convergence. American Mathematical Monthly 81 (1974) 625-627
34. K. Zografos, Entropy and Divergence Measures for Mixed Variables, Statistical Models and Methods for Biomedical and Technical Systems, Statistics for Industry and Technology, Birkhauser, 2008, pp. 519-534.