Hash property and coding theorems for sparse matrices and maximum-likelihood coding

IEEE Transactions on Information Theory

Volumn 56, Issue 5, 2010, Pages 2143-2167

  Muramatsu, Jun ^a   Miyake, Shigeki ^b  

^a NTT Corporation   (Japan)

^b NTT CORPORATION   (Japan)

Author keywords

Gel'fand Pinsker problem; Hash property; Linear codes; Maximum likelihood encoding decoding; One helps one problem; Shannon theory; Sparse matrix; Wyner Ziv problem

Indexed keywords

ENCODING/DECODING; LINEAR CODES; SHANNON THEORY; SPARSE MATRICES; SPARSE MATRIX; WYNER-ZIV PROBLEM;

CHANNEL CAPACITY; COMMUNICATION CHANNELS (INFORMATION THEORY); GELS; MATRIX ALGEBRA;

MAXIMUM LIKELIHOOD;

EID: 77951574913     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2043781     Document Type: Article

References (45)

1. S. M. Aji and R. J. McEliece, "The generalized distributive law, " IEEE Trans. Inf. Theory, vol. 46, no. 2, pp. 325-343, Mar. 2000.
2. A. Bennatan and D. Burshtein, "On the application of LDPC codes to arbitrary discrete-memoryless channels, " IEEE Trans. Inf. Theory, vol. 50, no. 3, pp. 417-438, Mar. 2004.
3. T. M. Cover, "A proof of the data compression theorem of Slepian and Wolf for ergodic sources, " IEEE Trans. Inf. Theory, vol. IT-21, no. 2, pp. 226-228, Mar. 1975.
4. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
5. I. Csiszár, "Linear codes for sources and source networks: Error exponents, universal coding, " IEEE Trans. Inf. Theory, vol. IT-28, no. 4, pp. 585-592, Jul. 1982.
6. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
7. J. L. Carter and M. N. Wegman, "Universal classes of hash functions, " J. Comput. Syst. Sci., vol. 18, pp. 143-154, 1979.
8. A. G. Dimakis, M. J. Wainwright, and K. Ramchandran, "Lower bounds on the rate-distortion function of LDGM codes, " in Proc. IEEE Inf. Theory Workshop, Lake Tahoe, NV, Sep. 2-6, 2007, pp. 541-546.
9. P. Elias, "Coding for noisy channels, " IRE Convention Record, Part 4, pp. 37-46, 1955.
10. q" IEEE Trans. Inf. Theory, vol. 51, no. 5, pp. 1871-1879, May 2005.
11. J. Feldman, M. J. Wainwright, and D. R. Karger, "Using linear programming to decode binary linear codes, " IEEE Trans. Inf. Theory, vol. 51, no. 3, pp. 954-972, Mar. 2005.
12. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
13. S. I. Gel'fand and M. S. Pinsker, "Coding for channel with random parameters, " Probl. Pered. Inf. (Probl. Inf. Transm.), vol. 9, no. 1, pp. 19-31, 1983.
14. A. Gupta and S. Verdú, "Nonlinear sparse-graph codes for lossy compression of discrete nonredundant sources, " IEEE Trans. Inf. Theory, vol. 55, no. 5, pp. 1961-1975, May 2009.
15. T. S. Han and K. Kobayashi, "Exponential-type error probabilities for multiterminal hypothesis testing, " IEEE Trans. Inf. Theory, vol. 35, no. 1, pp. 2-14, Jan. 1989.
16. T. S. Han and K. Kobayashi, Mathematics of Information and Coding. Brooklyn, NY: AMS, 2002.
17. H. Koga, "Source coding using families of universal hash functions, " IEEE Trans. Inf. Theory, vol. 53, no. 9, pp. 3226-3233, Sep. 2007.
18. F. R. Kschischang, B. J. Frey, and H. A. Loeliger, "Factor graphs and the sum-product algorithm, " IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
19. S. Kudekar and R. Urbanke, "Lower bounds on the rate-distortion function of individual LDGM codes, " 2007 [Online]. Available: arXiv:0804.1697 [cs. IT]
20. D. J. C. MacKay, "Good error-correcting codes based on very sparse matrices, " IEEE Trans. Inf. Theory, vol. 45, no. 2, pp. 399-431, Mar. 1999.
21. D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge, U. K.: Cambridge Univ. Press, 2003.
22. E. Martinian and M. Wainwright, "Low density codes achieve the ratedistortion bound, " in Proc. IEEE Data Compression Conf, Mar. 28-30, 2006, pp. 153-162.
23. E. Martinian and M. Wainwright, "Low-density constructions can achieve the Wyner-Ziv and Gelfand-Pinsker bounds, " in Proc. IEEE Int. Symp. Inf. Theory, Seattle, WA, Jul. 9-14, 2006, pp. 484-488.
24. Y. Matsunaga and H. Yamamoto, "A coding theorem for lossy data compression by LDPC codes, " IEEE Trans. Inf. Theory, vol. 49, no. 9, pp. 2225-2229, Sep. 2003.
25. G. Miller and D. Burshtein, "Bounds on the maximum-likelihood decoding error probability of low-density parity-check codes, " IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2696-2710, Nov. 2001.
26. q by LDPC code, " in Proc. IEEE Int. Symp. Inf. Theory, Seattle, WA, Jul. 9-14, 2006, pp. 813-816.
27. S. Miyake and J. Muramatsu, "Constructions of a lossy source code using LDPC matrices, " IEICE Trans. Fundam., vol. E91-A, no. 6, pp. 1488-1501, Jun. 2008.
28. S. Miyake and J. Muramatsu, "A construction of channel code, joint source-channel code, and universal code for arbitrary stationary memoryless channels using sparse matrices, " IEICE Trans. Fundam., vol. E92-A, no. 9, pp. 2333-2344, Sep. 2009.
29. J. Muramatsu and S. Miyake, "Lossy source coding algorithm using lossless multi-terminal source codes, " Tech. Rep. IEICE, IT 2006-50, Jan. 2007, pp. 1-6.
30. J. Muramatsu and S. Miyake, "Hash property and fixed-rate universal coding theorems, " IEEE Trans. Inf. Theory, to be published.
31. J. Muramatsu and S. Miyake, "Construction of codes for wiretap channel and secret key agreement from correlated source outputs by using sparse matrices, " 2009 [Online]. Available: arXiv:0903. 4014 [cs. IT]
32. J. Muramatsu, T. Uyematsu, and T. Wadayama, "Low density parity check matrices for coding of correlated sources, " IEEE Trans. Inf. Theory, vol. 51, no. 10, pp. 3645-3653, Oct. 2005.
33. J. Muramatsu, "Secret key agreement from correlated source outputs using low density parity check matrices, " IEICE Trans. Fundam., vol. E89-A, no. 7, pp. 2036-2046, Jul. 2006.
34. T. Murayama, "Thouless-Anderson-Palmer approach for lossy compression, " Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 69, no. 035105 (R), 2004.
35. C. E. Shannon, "A mathematical theory of communication, " Bell Syst. Tech. J., vol. 27, no. 379-423, pp. 623-656, 1948.
36. C. E. Shannon, "Coding theorems for a discrete source with a fidelity criterion, " IRE Nat. Conventional Record, Part 4, pp. 142-163, 1959.
37. T. Uyematsu, Gendai Shannon Riron (in Japanese). Tokyo, Japan: Baifukan, 1998.
38. T. Wadayama, "A lossy compression algorithm for discrete memoryless sources based on LDPC codes, " in Proc. 3rd Asian-Eur. Workshop Inf. Theory, Kamogawa, Japan, Jun. 25-28, 2003, pp. 98-105.
39. T. Wadayama, "A lossy compression algorithm for binary memoryless sources based on LDPC codes, " Tech. Rep. IEICE, IT 2003-50, 2003, pp. 53-56.
40. A. D. Wyner, "A theorem on the entropy of certain binary sequences and applications II, " IEEE Trans. Inf. Theory, vol. IT-19, no. 6, pp. 772-777, Nov. 1973.
41. A. D. Wyner, "Recent results in the Shannon theory, " IEEE Trans. Inf. Theory, vol. IT-20, no. 1, pp. 2-10, Jan. 1974.
42. A. D. Wyner and J. Ziv, "A theorem on the entropy of certain binary sequences and applications I, " IEEE Trans. Inf. Theory, vol. IT-19, no. 6, pp. 769-771, Nov. 1973.
43. A. D. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder, " IEEE Trans. Inf. Theory, vol. IT-22, no. 1, pp. 1-10, Jan. 1976.
44. R. W. Yeung, A First Course in Information Theory. New York: Springer-Verlag, 2006.
45. R. Zamir, S. Shamai (Shitz), and U. Erez, "Nested linear/lattice codes for structured multiterminal binning, " IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1250-1276, Jun. 2002.