Simulation-based computation of information rates for channels with memory

IEEE Transactions on Information Theory

Volumn 52, Issue 8, 2006, Pages 3498-3508

  Arnold, Dieter M ^a,b,c   Loeliger, Hans Andrea ^a   Vontobel, Pascal O ^a,d,e   Kavčić, Aleksandar ^f   Zeng, Wei ^f  

^a ETH ZURICH   (Switzerland)

^b IBM RESEARCH ZURICH   (Switzerland)

^c Siemens Schweiz AG   (Switzerland)

^d University of Illinois at Urbana Champaign   (United States)

^e HEWLETT PACKARD LABORATORIES   (United States)

^f HARVARD UNIVERSITY   (United States)

Author keywords

Bounds; Channel capacity; Finite state models; Hidden Markov models; Information rate; Sum product algorithm; Trellises

Indexed keywords

APPROXIMATION THEORY; BOUNDARY CONDITIONS; CHANNEL CAPACITY; COMPUTATIONAL METHODS; COMPUTER SIMULATION; FINITE AUTOMATA; MARKOV PROCESSES; MATHEMATICAL MODELS;

FINITE-STATE MODELS; HIDDEN-MARKOV MODELS; INFORMATION RATES; SUM-PRODUCT ALGORITHM; TRELLISES;

COMMUNICATION CHANNELS (INFORMATION THEORY);

EID: 33746625203     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.878110     Document Type: Article

References (49)

1. S. Arimoto, "An algorithm for computing the capacity of arbitrary discrete memoryless channels," IEEE Trans. Inf. Theory, vol. IT-18, no. 1, pp. 14-20, Jan. 1972.
2. D. Arnold, A. Kavčić, R. Kötter, H.-A. Loeliger, and P. O. Vontobel, "The binary jitter channel: A new model for magnetic recording," in Proc. 2000 IEEE Int. Symp. Information Theory, Sorrento, Italy, Jun. 2000, p. 433.
3. D. Arnold and H.-A. Loeliger, "On the information rate of binary-input channels with memory," in Proc. 2001 IEEE Int. Conf. Communications, Helsinki, Finland, Jun. 2001, pp. 2692-2695.
4. D. Arnold and H.-A. Loeliger, "On finite-state information rates from channel simulations," in Proc. 2002 IEEE Int. Symp. Information Theory, Lausanne, Switzerland, Jun./Jul. 2002, p. 164.
5. D. M. Arnold, "Computing Information Rates of Finite-State Models with Application to Magnetic Recording," ETH-Diss 14760 (Ph.D dissertation), ETH Zurich, Zurich, Switzerland, 2003.
6. D. Arnold, H.-A. Loeliger, and P. O. Vontobel, "Computation of information rates from finite-state source/channel models," in Proc. 40th Annu. Allerton Conf. Communication, Control, and Computing, Monticello, IL, Oct. 2002, pp. 457-466.
7. D. Arnold, A. Kavčić, H.-A. Loeliger, P. O. Vontobel, and W. Zeng, "Simulation-based computation of information rates: Upper and lower bounds," in Proc. 2003 IEEE Int. Symp. Information Theory, Yokohama, Japan, Jun./Jul. 2003, p. 231.
8. L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate," IEEE Trans. Inf. Theory, vol. IT-20, no. 2, pp. 284-287, Mar. 1974.
9. A. R. Barron, "The strong ergodic theorem for densities: Generalized Shannon-McMillan-Breiman theorem," Ann. Probab., vol. 13, no. 4, pp. 1292-1303, 1985.
10. R. E. Blahut, "Computation of channel capacity and rate-distortion functions," IEEE Trans. Inf. Theory, vol. IT-18, no. 4, pp. 460-473, Jul. 1972.
11. J. Chen and P. H. Siegel, "On the symmetric information rate of two-dimensional finite-state ISI channels," in Proc. 2003 IEEE Information Theory Workshop, Paris, France, Mar./Apr. 2003, pp. 320-323.
12. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
13. J. Dauwels and H.-A. Loeliger, "Computation of information rates by particle methods," in Proc. 2004 IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 178.
14. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1518-1569, Jun. 2002.
15. T. R. M. Fischer, "Some remarks on the role of inaccuracy in Shannon's theory of information transmission," in Proc. 8th Prague Conf. Information Theory, Prague, Czechoslovakia, 1978, pp. 221-226.
16. G. D. Forney, Jr., "Codes on graphs: Normal realizations," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 520-548, Feb. 2001.
17. A. Ganti, A. Lapidoth, and I. E. Telatar, "Mismatched decoding revisited: General alphabets, channels with memory, and the wide-band limit," IEEE Trans. Inf. Theory, vol. 46, no. 7, pp. 2315-2328, Nov. 2000.
18. A. J. Goldsmith and P. P. Varajya, "Capacity, mutual information, and coding for finite-state Markov channels," IEEE Trans. Inf. Theory, vol. 42, no. 3, pp. 868-886, May 1996.
19. C. Heegard and A. Duel-Hallen, "On the capacity of the noisy runlength channel," in Proc. 1998 IEEE Information Theory Workshop, Beijing, China, Jul. 1988.
20. C. Heegard, A. Duel-Hallen, and R. Krishnamoorthy, "On the capacity of the noisy runlength channel," IEEE Trans. Inf. Theory, vol. 37, no. 3, pp. 712-720, May 1991.
21. W. Hirt, "Capacity and Information Rates of Discrete-Time Channels with Memory," ETH-Diss 8671 (Ph.D. dissertation), ETH Zurich, Zurich, Switzerland, 1988.
22. T. Holliday, A. Goldsmith, and P. Glynn, "Entropy and mutual information for Markov channels with general inputs," in Proc. 40th Annu. Allerton Conf. Communication, Control, and Computing, Monticello, IL, Oct. 2002, pp. 824-833.
23. T. Holliday, A. Goldsmith, and P. Glynn, "Capacity of finite-state channels based on Lyapunov exponents of random matrices," IEEE Trans. Inf. Theory, vol. 52, no. 8, pp. 3509-3532, Aug. 2006.
24. A. Kavčić, "On the capacity of Markov sources over noisy channels," in Proc. 2001 IEEE GLOBECOM, San Antonio, TX, Nov. 2001, pp. 2997-3001.
25. B.-H. Juang and L. R. Rabiner, "A probabilistic distance measure for hidden Markov models," AT&T Tech. J., vol. 64, pp. 391-408, Feb. 1985.
26. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
27. B. G. Leroux, "Maximum-likelihood estimation for hidden Markov models," Stochastic Processes Their Applic., vol. 40, pp. 127-143, 1992.
28. H.-A. Loeliger, "An introduction to factor graphs," IEEE Signal Process. Mag., pp. 28-41, Jan. 2004.
29. M. Mushkin and I. Bar-David, "Capacity and coding for the Gilbert-Elliott channel," IEEE Trans. Inf. Theory, vol. 35, no. 6, pp. 1277-1290, Nov. 1989.
30. A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
31. T. Petrie, "Probabilistic functions of finite state Markov chains," Ann. Math. Statist., vol. 40, no. 1, pp. 97-115, 1969.
32. H. D. Pfister, J. B. Soriaga, and P. H. Siegel, "On the achievable information rates of finite-state ISI channels," in Proc. 2001 IEEE Globecom, San Antonio, TX, Nov. 2001, pp. 2992-2996.
33. R. Pighi, R. Raheli, and F. Cappelletti, "Information rates of multidimensional front-ends for digital storage channels with data-dependent transition noise," in Proc. 2005 IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 1343-1347.
34. W. E, Ryan, F. Wang, R. Wood, and Y. Li, "Optimal code rates for the Lorentzian channel: Shannon codes and LDPC codes," IEEE Trans. Magn., vol. 40, no. 6, pp. 3559-3565, Nov. 2004.
35. S. Shamai (Shitz) and Y. Kofman, "On the capacity of binary and Gaussian channels with run-length-limited inputs," IEEE Trans. Commun., vol. 38, no. 5, pp. 584-594, May 1990.
36. S. Shamai (Shitz), L. H. Ozarow, and A. D. Wyner, "Information rates for a discrete-time Gaussian channel with intersymbol interference and stationary inputs," IEEE Trans. Inf. Theory, vol. 37, no. 6, pp. 1527-1539, Nov. 1991.
37. S. Shamai (Shitz) and R. Laroia, "The intersymbol interference channel: Lower bounds on capacity and channel precoding loss, " IEEE Trans. Inf. Theory, vol. 42, no. 5, pp. 1388-1404, Sep. 1996.
38. V. Sharma and S. K. Singh, "Entropy and channel capacity in the regenerative setup with applications to Markov channels," in Proc. 2001 IEEE Int. Symp. Information Theory, Washington, DC, Jun. 2001. p. 283.
39. O. Shental, N. Shental, and S. Shamai (Shitz), "On the achievable information rates of finite-state input two-dimensional channels with memory," in Proc. 2005 IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 2354-2358.
40. J. B. Soriaga, P. H. Siegel, J. K. Wolf, and M. Marrow, "On Achievable rates of multistage decoding on two-dimensional ISI channels," in Proc. 2005 IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 1348-1352.
41. F. Topsøe, "An information theoretical identity and a problem involving capacity," Studia Scientiarum Math. Hung., vol. 2, pp. 291-292, 1967.
42. P. O. Vontobel and D. M. Arnold, "An upper bound on the capacity of channels with memory and constrained input," in Proc. 2001 IEEE Information Theory Workshop, Cairns, Australia, Sep. 2001, pp. 147-149.
43. P. O. Vontobel, A. Kavčić, D. M. Arnold, and H.-A. Loeliger, "A generalization of the Blahut-Arimoto algorithm to finite-state channels," IEEE Trans. Inf. Theory, to be published.
44. W. Xiang and S. S. Pietrobon, "On the capacity and normalization of ISI channels," IEEE Trans. Inf. Theory, vol. 49, no. 9, pp. 2263-2268, Sep. 2003.
45. S. Yang, A. Kavčić, and S. Tatikonda, "Delayed feedback capacity of finite-state machine channels: Upper bounds on the feedforward capacity," in Proc. 2003 IEEE Int. Symp. Information Theory, Yokohama, Japan, Jun./Jul. 2003, p. 290.
46. E. Zehavi and J. K. Wolf, "On runlength codes," IEEE Trans. Inf. Theory, vol. 34, no. 1, pp. 45-54, Jan. 1988.
47. W. Zeng, J. Tokas, R. Motwani, and A. Kavčić, "Bounds on mutual information rates of noisy channels with timing errors," in Proc. 2005 IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 709-713.
48. Z. Zhang, T. M. Duman, and E. M. Kurtas, "Information rates of binary-input intersymbol interference channels with signal-dependent media noise," IEEE Trans. Magn., vol. 39, no. 1, pp. 599-607, Jan. 2003.
49. Z. Zhang, T. M. Duman, and E. M. Kurtas, "Achievable information rates and coding for MIMO systems over ISI channels and frequency-selective fading channels," IEEE Trans. Commun., vol. 52, no. 10, pp. 1698-1710, Oct. 2004.