The Kullback-Leibler divergence rate between Markov sources

IEEE Transactions on Information Theory

Volumn 50, Issue 5, 2004, Pages 917-921

  Rached, Ziad ^a   Alajaji, Fady ^a   Campbell, L Lorne ^a  

^a QUEEN S UNIVERSITY   (Canada)

Author keywords

Classification; Decision theory; Kullback Leibler divergence rate; Nonnegative matrices; Pattern recognition; Perron Frobenius theory; Rate of convergence; Shannon entropy rate; Time invariant Markov sources

Indexed keywords

DECISION THEORY; MATRIX ALGEBRA; PATTERN RECOGNITION; PROBABILITY DISTRIBUTIONS; SIGNAL PROCESSING; VECTORS;

KULLBACK-LEIBLER DIVERGENCE RATE; PERRON -FROBENIUS THEORY; RATE OF CONVERGENCE; SHANNON ENTROPY RATE; TIME-INVARIANT MARKOV SOURCES;

MARKOV PROCESSES;

EID: 2442538938     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.826687     Document Type: Letter

References (18)

1. M. B. Bassat, "f-entropies, probability of error, and feature selection," Inform. Contr., vol. 39, pp. 227-242, 1978.
2. C. H. Chen, Statistical Pattern Recognition. Rochelle Park, NJ: Hayden, 1973, ch. 4.
3. C. K. Chow and C. N. Liu, "Approximating discrete probability distributions with dependence trees," IEEE Trans. Inform. Theory, vol. IT-14, pp. 462-467, May 1968.
4. D. R. Cox and H. D. Miller, The Theory of Stochastic Processes. London, U.K.: Methuen and Co. Ltd, 1965.
5. M. N. Do, "Fast approximation of Kullback-Leibler distance for dependence trees and hidden Markov models," IEEE Signal Processing Lett., vol. 10, pp. 115-118, Apr. 2003.
6. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
7. ____, Discrete Stochastic Processes. Boston, MA: Kluwer, 1996.
8. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
9. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985.
10. T. T. Kadota and L. A. Shepp, "On the best finite set of linear observables for discriminating two Gaussian signals," IEEE Trans. Inform. Theory, vol. IT-13, pp. 278-284, Apr. 1967.
11. T. Kailath, "The divergence and Bhattacharyya distance measured in signal selection," IEEE Trans. Commun. Technol., vol. COM-15, pp. 52-60, Feb. 1967.
12. D. Kazakos and T. Cotsidas, "A decision theory approach to the approximation of discrete probability densities," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, pp. 61-67, Jan. 1980.
13. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, pp. 79-86, 1951.
14. K. Marton and P. C. Shields, "The positive-divergence and blowing-up properties," Israel J. Math., vol. 86, pp. 331-348, 1994.
15. Z. Rached, F. Alajaji, and L. L. Campbell, "Rényi's divergence and entropy rates for finite alphabet Markov sources," IEEE Trans. Inform. Theory, vol. 47, pp. 1553-1561, May 2001.
16. E. Seneta, Non-Negative Matrices and Markov Chains. New York: Springer-Verlag, 1981.
17. P. C. Shields, "Two divergence-rate counterexamples," J. Theor. Probab., vol. 6, pp. 521-545, 1993.
18. Z. Ye and T. Berger, Information Measures for Discrete Random Fields. Beijing, New York: Science, 1998.