On the entropy of a hidden Markov process

Theoretical Computer Science

Volumn 395, Issue 2-3, 2008, Pages 203-219

  Jacquet, Philippe ^a   Seroussi, Gadiel ^b   Szpankowski, Wojciech ^c  

^a INRIA ROCQUENCOURT   (France)

^b HEWLETT PACKARD LABORATORIES   (United States)

^c PURDUE UNIVERSITY   (United States)

Author keywords

Hidden Markov process; Product of random matrices; R nyi entropy; Shannon entropy; Spectral representation of matrices; Top Lyapunov exponent

Indexed keywords

APPROXIMATION THEORY; DIFFERENTIAL EQUATIONS; ENTROPY; LYAPUNOV METHODS; MATRIX ALGEBRA; POLYNOMIAL APPROXIMATION; RANDOM PROCESSES; SIGNAL THEORY; TELECOMMUNICATION NETWORKS;

(001) PARAMETER; (E ,3E) PROCESS; ASYMPTOTIC ESTIMATES; BINARY SYMMETRIC CHANNEL (BSC); ELSEVIER (CO); ENTROPY RATES; EXPLICIT EXPRESSIONS; FIRST ORDERS; HIDDEN MARKOV PROCESS (HMP); HIGHER ORDER; LONG STANDING; LYAPUNOV EXPONENT (LE); OPEN PROBLEMS; PRODUCT OF RANDOM MATRICES; SIMULATION RESULTS; TAYLOR APPROXIMATIONS;

MARKOV PROCESSES;

EID: 45749088482     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2008.01.012     Document Type: Article

References (38)

1. Bahl L.R., Cocke J., Jelinek F., and Raviv J. Optimal decoding of linear codes for minimizing symbol error rate. IEEE Trans. Inform. Theory IT-20 (1974) 284-287
2. Baum L.E., and Petrie T. Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Statist. 37 (1966) 1554-1563
3. Blackwell D. The entropy of functions of finite-state Markov chains. Trans. First Prague Conf. Information Theory, Statistical Decision Functions, Random Processes (Prague, Czechoslovakia) (1957), Pub. House Chechoslovak Acad. Sci. 13-20
4. Churchill G.A. Stochastic models for heterogeneous DNA sequences. Bull. Math. Biol. 51 1 (1989) 79-94
5. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), John Wiley & Sons, Inc., New York
6. A.W. Drake, Observation of a Markov process through a noisy channel, Ph.D. Thesis, Massachusetts Institute of Technology, 1962
7. A.W. Drake, Observation of a Markov source through a noisy channel, in: Proc. IEEE Symp. Signal Transmission and Processing, Columbia Univ., New York, 1965, pp. 12-18
8. Weissman T., Ordentlich E., Seroussi G., Verdú S., and Weinberger M. Universal discrete denoising: Known channel. IEEE Trans. Inform. Theory 52 (2005) 5-28
9. Ephraim Y., and Merhav N. Hidden Markov processes. IEEE Trans. Inform. Theory IT-48 (2002) 1518-1569
10. Fan J., Poo T.L., and Marcus B. Constraint gain. IEEE Trans. Inform. Theory 50 (2001) 1989-1999
11. In: Ferguson J.D. (Ed). Application of Hidden Markov Models to Text and Speech (1980), IDA-CRD, Princeton, NJ
12. Furstenberg H., and Kesten H. Products of random matrices. Ann. Math. Statist. (1960) 457-469
13. R. Gharavi, V. Ananththaram, An upper bound for the largest Lyapunov exponent of a markovian product of nonnegative matrices, preprint, 2004
14. Holliday T., Goldsmith A., and Glynn P. Capacity of finite state channels based on Lyapunov exponents of random matrices. IEEE Trans. Inform. Theory 52 (2006) 3509-3532
15. Han G., and Marcus B. Analyticity of entropy rate of hidden Markov chains. IEEE Trans. Inform. Theory 52 (2006) 5251-5266
16. G. Han, B. Marcus, Analyticity of entropy rate in families of hidden Markov chains (II), in: Proc. ISIT 2006, Seattle, 2006, pp. 103-107
17. G. Han, B. Marcus, Capacity of noisy constrained channels, in: Proc. ISIT 2007, Nice, France, 2007
18. Jelinek F., Bahl L.R., and Mercer R.L. Design of a linguistic statistical decoder for recognition of continuous speech. IEEE Trans. Inform. Theory IT-21 (1975) 250-256
19. Jacquet P., and Szpankowski W. Entropy computations via analytic depoissonization. IEEE Trans. Inform. Theory IT-45 (1999) 1072-1081
20. P. Jacquet, G. Seroussi, W. Szpankowski, On the entropy of a hidden Markov process (extended abstract), in: Data Compression Conference, Snowbird, 2004, pp. 362-371
21. P. Jacquet, G. Seroussi, W. Szpankowski, Noisy constrained capacity, in: Proc. ISIT 2007, Nice, France, 2007
22. Khasminskii R., and Zeitouni O. Asymptotic filtering for finite state Markov chains. Stochastic Process. Appl. 63 (1996) 1-10
23. Karlin S., and Taylor H. A First Course in Stochastic Processes (1975), Academic Press, New York
24. Leroux B.G. Maximum-likelihood estimation for hidden Markov models. Stochastic Process. Appl. 40 (1992) 127-143
25. Lukashin A.V., and Borodovsky M. GeneMark.hmm: New solutions for gene finding. Nucleic Acids Res. 26 4 (1998) 1107-1115
26. Noble B., and Daniel J. Applied Linear Algebra (1988), Prentice Hall, Englewood Cliffs, NJ
27. Oseledec V. A multiplicative ergodic theorem. Trudy Moskov Mat. Obsc (1968)
28. E. Ordentlich, T. Weissman, New bounds on th entropy rate of hidden Markov process, in: Information Theory Workshop, San Antonio, 2004, pp. 117-122
29. Ordentlich E., and Weissman T. On the optimality of symbol by symbol filtering and denoising. IEEE Trans. Inform. Theory 52 (2006) 19-40
30. Petrie T. Probabilistic functions of finite state Markov chains. Ann. Math. Statist. 40 1 (1969) 97-115
31. Rabiner L.R. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77 (1989) 257-286
32. Raviv J. Decision making in Markov chains applied to the problem of pattern recognition. IEEE Trans. Inform. Theory IT-3 (1967) 536-551
33. A. Rényi, On measures of entropy and information, in: Proceedings of 4th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, 1960, pp. 547-561
34. Seneta E. Non-Negative Matrices (1973), John Wiley & Sons, New York
35. Szpankowski W. A generalized suffix tree and its (un)expected asymptotic behaviors. SIAM J. Comput. 22 (1993) 1176-1198
36. Szpankowski W. Average Case Analysis of Algorithms on Sequences (2001), John Wiley & Sons, Inc., New York
37. Tsitsiklis J., and Blondel V. The Lyapunov exponent and joint spectral radius of pairs of matrices are hard-when not impossible-to compute and to approximate. Math. Control Signals Syst. 10 (1997) 31-40
38. Zuk O., Kanter I., and Domany E. Asymptotics of the entropy rate for a hidden Markov process. J. Stat. Phys. 121 (2005) 343-360