On the entropy of a hidden Markov process

Data Compression Conference Proceedings

Volumn , Issue , 2004, Pages 362-371

  Jacquet, Philippe ^a   Seroussi, Gadiel ^b   Szpankowski, Wojciech ^c  

^a INRIA ROCQUENCOURT   (France)

^b HEWLETT PACKARD LABORATORIES   (United States)

^c PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; BINARY CODES; CHARACTER RECOGNITION; COMMUNICATION CHANNELS (INFORMATION THEORY); COMPUTER SIMULATION; MATRIX ALGEBRA; PROBABILITY;

BINARY ALPHABETS; HIDDEN MARKOV PROCESS (HMP); RANDOM MATRICES; TAYLOR APPROXIMATION;

MARKOV PROCESSES;

EID: 2642552515     PISSN: 10680314     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. A. W. Drake, Observation of a Markov Process Through a Noisy Channel, PhD thesis, Massachusetts Institute of Technology, 1962.
2. A.W. Drake, "Observation of a Markov source through a noisy channel," in Proc. IEEE Symp. Signal Transmission and Processing, Columbia Univ., New York, 1965, pp. 12-18.
3. L. E. Baum and T. Petrie, "Statistical inference for probabilistic functions of finite state Markov chains," Ann. Math. Statist., vol. 37, pp. 1554-1563, 1966.
4. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Trans. Inform. Theory, vol. IT-48, pp. 1518-1569, June 2002.
5. J. Raviv, "Decision making in Markov chains applied to the problem of pattern recognition," IEEE Trans. Inform. Theory, vol. IT-3, pp. 536-551, Oct. 1967.
6. F. Jelinek, L. R. Bahl, and R. L. Mercer, "Design of a linguistic statistical decoder for recognition of continuous speech," IEEE Trans. Inform. Theory, vol. IT-21, pp. 250-256, May 1975.
7. J. D. Ferguson, ed., Application of Hidden Markov Models to Text and Speech, (Prince-ton, NJ), IDA-CRD, 1980.
8. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, pp. 257-286, Feb. 1989.
9. B. G. Leroux, "Maximum-likelihood estimation for hidden Markov models," Stochastic Processes Their Applic., vol. 40, pp. 127-143, 1992.
10. L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate," IEEE Trans. Inform. Theory, vol. IT-20, pp. 284-287, mar 1974.
11. G. A. Churchill, "Stochastic models for heterogeneous DNA sequences," Bull. Math. Biology, vol. 51, no. 1, pp. 79-94, 1989.
12. A. V. Lukashin and M. Borodovsky, "GeneMark.hmm: New solutions for gene finding," Nucleic Acids Res, vol. 26, no. 4, pp. 1107-1115, 1998.
13. R. Khasminskii and 0. Zeitouni, "Asymptotic filtering for finite state Markov chains," Stochastic Processes and their Applications, vol. 63, pp. 1-10, 1996.
14. E. Ordentlich and T. Weissman, "On the optimality of symbol by symbol filtering and denoising." Preprint, 2003.
15. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdú, and M. Weinberger, "Universal discrete denoising: Known channel," Technical Report HPL-2003-29, Hewlett-Packard Laboratories, Feb. 2003. Available at: http://www.hpl.hp.com/techreports/2003/HPL-2003-29.pdf.
16. W. Szpankowski, Average Case Analysis of Algorithms on Sequences. New York: John Wiley & Sons, Inc., 2001.
17. D. Blackwell, "The entropy of functions of finite-state Markov chains," in Trans. First Prague Conf. Information Theory, Statistical Decision Functions, Random Processes, (Prague, Czechoslovakia), pp. 13-20, Pub. House Chechoslovak Acad. Sci., 1957.
18. T. Petrie, "Probabilistic functions of finite state Markov chains," Ann. Math. Statist., vol. 40, no. 1, pp. 97-115, 1969.
19. J. Tsitsiklis and V. Blondel, "The Lyapunov exponent and joint spectral radius of pairs of matrices are hard - when not impossible - to compute and to approximate," Mathematics of Control, Signals, and Systems, vol. 10, pp. 31-40, 1997.
20. H. Furstenberg and H. Kesten, "Products of random matrices," Ann. Math. Statist, pp. 457-469, 1960.
21. V. Oseledec, "A multiplicative ergodic theorem," Trudy Moskov Mat. Obsc, 1968.
22. T. Holliday, A. Goldsmith, and P. Glynn, "On entropy and Lyapunov exponents for finite state channels." Preprint, http://wsl.stanford.edu/ Publications/THolliday/Lyapunov.pdf, 2003.
23. P. Jacquet and W. Szpankowski, "Entropy computations via analytic depoissonization," IEEE Trans. Inform. Theory, vol. IT-45, pp. 1072-1081, 1999.
24. S. Karlin and H. Taylor, A First Course in Stochastic Processes. New York: Academic Press, 1975.
25. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: John Wiley & Sons, Inc., 1991.