Extension of the Blahut-Arimoto algorithm for maximizing directed information

IEEE Transactions on Information Theory

Volumn 59, Issue 1, 2013, Pages 204-222

  Naiss, Iddo ^a,b   Permuter, Haim H ^a  

^a BEN GURION UNIVERSITY OF THE NEGEV   (Israel)

^b SAMSUNG Electronics   (South Korea)

Author keywords

Alternating maximization procedure; backward index time maximization; Blahut Arimoto algorithm; causal conditioning; channels with feedback; directed information; finite state channels (FSCs); Ising channel; trapdoor channel

Indexed keywords

ALTERNATING MAXIMIZATION PROCEDURE; BLAHUT-ARIMOTO ALGORITHM; DIRECTED INFORMATION; FINITE-STATE; ISING CHANNEL; TRAPDOOR CHANNEL;

COMPUTER APPLICATIONS; INFORMATION THEORY;

ALGORITHMS;

EID: 84871726451     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2012.2214202     Document Type: Article

References (35)

1. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 623-656, 1948.
2. R. Blahut, "An algorithm for computing the capacity of arbitrary discrete memoryless channels," IEEE Trans. Inf. Theory, vol. 18, no. 1, pp. 14-20, Jan. 1972.
3. S.Arimoto, "Computation of channel capacity and rate-distortion functions," IEEE Trans. Inf. Theory, vol. 18, no. 4, pp. 460-473, Jul. 1972.
4. I. Csiszár and G. Tusnady, "Information geometry and alternatingminimization procedures," Statist. Decis., vol. 1, pp. 205-237, 1984.
5. R. W. Yeung, Information Theory and Network Coding. NewYork: Springer-Verlag, 2008.
6. J.Massey, "Causality, feedback and directed information," in Proc. Int. Symp. Inf. Theory Appl., Waikiki, HI, Nov. 1990, pp. 303-305.
7. G. Kramer, "Directed information for channels with feedback," Ph.D. dissertation, Swiss Federal Institute of Technology, Zurich, Switzerland, 1998.
8. S. Tatikonda and S. Mitter, "The capacity of channels with feedback," IEEE Trans. Inf. Theory, vol. 55, no. 1, pp. 323-349, Jan. 2009.
9. H. H. Permuter, T. Weissman, and A. J. Goldsmith, "Finite state channels with time-invariant deterministic feedback," IEEE Trans. Inf. Theory, vol. 55, no. 2, pp. 644-662, Feb. 2009.
10. Y. H. Kim, "A coding theorem for a class of stationary channels with feedback," IEEE Trans. Inf. Theory, vol. 54, no. 4, pp. 1488-1499,Apr. 2008.
11. G. Matz and P. Duhamel, "Information geometric formulation and interpretation of accelerator Blahut-Arimoto-type algorithms," in Proc. IEEE Inf. Theory Workshop, San Antonio, TX, Oct. 2004, pp. 66-70.
12. M. Rezaeian and A. Grant, "Computation of total capacity for discrete memoryless multiple-access channels," IEEE Trans. Inf. Theory, vol. 50, no. 11, pp. 2779-2784, Nov. 2004.
13. W. Yu, F. Dupuis, and F. Willems, "Arimoto-Blahut algorithms for computing channel capacity and rate-distortion with side information," presented at the Int. Symp. Inf. Theory, 2004.
14. C. Heegard and A. A. El Gamal, "On the capacity of computer memory with defects," IEEE Trans. Inf. Theory, vol. 29, no. 5, pp. 731-739, Sep. 1983.
15. G. Markavian, S. Egorov, and K. Pickavance, "A modified Blahut algorithm for decoding Reed Solomon codes beyond half the minimum distance," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 2052-2056, Dec. 2004.
16. J. Dauwels, "Numercal computation of the capacity of continuous memoryless channels," presented at the 26th Symp. Inf. Theory, 2005.
17. D. Arnold, H.-A. Loeliger, P. O. Vontobel, and A. Kavcic, "Capacity of finite-state machine channels," IEEE Trans. Inf. Theory, vol. 54, no. 5, pp. 1887-1918, May 2008.
18. O. Sumszyk and Y. Steinberg, "Information embedding with reversible stegotext," in ISIT, Novo mesto, Slovenia, Jul. 2009, pp. 2728-2732.
19. U. Niesen, D. Shah, and G. W. Wornell, "Adaptive alternating minimization algorithms," IEEE Trans. Inf. Theory, vol. 55, no. 3, pp. 1423-1429, Mar. 2009.
20. A. Kavcic, S. Yang, and S. Tatikonda, "Feedback capacity of finitestate machine channels," IEEE Trans. Inf. Theory, vol. 51, no. 3, pp. 799-810, Mar. 2005.
21. J. Chen and T. Berger, "The capacity of finite-state Markov channels with feedback," IEEE Trans. Inf. Theory, vol. 51, no. 3, pp. 780-789, Mar. 2005.
22. H. H. Permuter, P. Cuff, B. Van Roy, and T. Weissman, "Capacity of the trapdoor channel with feedback," IEEE Trans. Inf. Theory, vol. 54, no. 7, pp. 3150-3165, Jul. 2008.
23. S. K. Gorantla and T. P. Coleman, "On reversible Markov chains and maximization of directed information," in Proc. Int. Symp. Inf. Theory, Jun. 2010, pp. 216-220.
24. H. Marko, "The bidirectional communication theory-A generalization of information theory," IEEE Trans. Commun., vol. COM-21, no. 12, pp. 1335-1351, Dec. 1973.
25. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New-York: Wiley, 2006.
26. S. Boyd and L. Vandenberghe, Convex Optimization. New York: Cambridge Univ. Press, 2004.
27. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
28. C. E. Shannon, "The zero error capacity of a noisy channel," IEEE Trans. Inf. Theory, vol. IT-2, no. 3, pp. 8-19, Sep. 1956.
29. D. Blackwell, Modern Mathematics for Engineer ser. II, 1961, pp. 183-193.
30. R. Ash, Information Theory. New York: Wiley, 1965.
31. T. Berger and F. Bonomi, "Capacity and zero-error capacity of Ising channels," IEEE Trans. Inf. Theory, vol. 36, no. 1, pp. 173-180, Jan. 1990.
32. T. Weissman and N. Merhav, "On competitive prediction and its relation to rate-distortion theory," IEEE Trans. Inf. Theory, vol. 49, no. 12, pp. 3185-3194, Dec. 2003.
33. R. Venkataramanan and S. S. Pradhan, "Source coding with feed-forward: Rate-distortion theorems and error exponents for a general source," IEEE Trans. Inf. Theory, vol. 53, no. 6, pp. 2154-2179, Jun. 2007.
34. R. Venkataramanan and S. S. Pradhan, "On evaluating the rate-distortion function of sources with feed-forward and the capacity of channels with feedback," CoRR 2007, http://arxiv.org/abs/cs/0702009.
35. H. Permuter and I.Naiss, in Bounds on rate distortion with feed forward for stationary and ergodic sources, Jul.-Aug. 2011, pp. 385-389.