Tight bounds for LDPC and LDGM codes under MAP decoding

IEEE Transactions on Information Theory

Volumn 51, Issue 9, 2005, Pages 3221-3246

  Montanari, Andrea ^a  

^a ECOLE NORMALE SUPÉRIEURE   (France)

Author keywords

Conditional entropy; Low density parity check (LDPC) codes; Maximum a posteriori probability (MAP) decoding; Spin glasses; Statistical physics

Indexed keywords

BIT ERROR RATE; CODES (SYMBOLS); COMMUNICATION CHANNELS (INFORMATION THEORY); DECODING; ERROR CORRECTION; HEURISTIC METHODS; OPTIMIZATION; POISSON DISTRIBUTION; STATISTICAL MECHANICS;

BINARY ERASURE CHANNEL (BEC); CONDITIONAL ENTROPY; ISING SYSTEMS; LOW-DENSITY GENERATOR-MATRIX (LDGM); LOW-DENSITY PARITY-CHECK (LDPC); MEAN FIELD SPIN GLASSES;

SIGNAL ENCODING;

EID: 26444602447     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.853320     Document Type: Article

References (42)

1. R. G. Gallager, Low Density Parity-Check Codes. Cambridge, MA: MIT Press, 1963.
2. M. G. Luby, M. Mitzenmacher, A. Shokrollahi, and D. A. Spielman, "Efficient erasure codes," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 569-584, Feb. 2001.
3. T. J. Richardson and R. L. Urbanke, "The capacity of low-density parity-check codes under message-passing decoding," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599-618, Feb. 2001.
4. T. J. Richardson and R. L. Urbanke, "An introduction to the analysis of iterative coding systems," in Codes, Systems, and Graphical Models, B. Marcus and J. Rosenthal, Eds. New York: Springer-Verlag, 2001, pp. 1-37.
5. M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman, "Improved low-density parity-check codes using irregular graphs," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 585-598, Feb. 2001.
6. S.-Y. Chung, G. D. Forney Jr., T. J. Richardson, and R. Urbanke, "On the design of low-density parity-check codes within 0.0045 dB from the Shannon limit," IEEE Comm. Lett., vol. 5, no. 2, pp. 58-60, Feb. 2001.
7. S. Aji, H. Jin, A. Khandekar, R. J. McEliece, and D. J. C. MacKay, "Thresholds for code ensembles based on 'Typical pairs' decoding," in Codes, Systems and Graphical Models, B. Marcus and J. Rothental, Eds. New York: Springer-Verlag, 2001, pp. 195-210.
8. G. Miller and D. Burshtein, "Bounds on the maximum-likelihood decoding error probability of low-density parity-check codes," IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2696-2710, Nov. 2001.
9. D. Burshtein, M. Krivelevich, S. Litsyn, and G. Miller, "Upper bounds on the rate of LDPC codes," IEEE Trans. Inf. Theory, vol. 48, no. 9, pp. 2437-2449, Sep. 2002.
10. S. Shamai (Shitz) and I. Sason, "Variations on the Gallager bounds, connections, and applications," IEEE Trans. Inf. Theory, vol. 48, no. 12, pp. 3029-3051, Dec. 2002.
11. H. Pishro-Nik and F. Fekri, "On decoding of low-density parity-check codes over the binary erasue channel," IEEE Trans. Inf. Theory, vol. 50, no. 3, pp. 439-454, Mar. 2004.
12. M. Mezard, G. Parisi, and M. A. Virasoro, Spin Glass Theory and Beyond, Singapore: World Scientific, 1987.
13. M. Talagrand, Spin Glasses: A Challenge for Mathematicians. New York: Springer-Verlag, 2003.
14. N. Sourlas, "Spin-glass models as error-correcting codes," Nature, vol. 339, pp. 693-694, 1989.
15. N. Sourlas, "Statistical mechanics and error-correcting codes," in Statistical Mechanics of Neural Networks (Lecture Notes in Physics), L. Garrido, Ed. New York: Springer-Verlag, 1990, vol. 368, p. 317.
16. N. Sourlas, "Statistical mechanics and error-correcting codes," in From Statistical Physics to Statistical Inference and Back, P. Grassberger and J.-P. Nadal, Eds. Dordrecht, The Netherlands: Kluwer Academic, 1994, p. 195.
17. Y. Kabashima, T. Murayama, and D. Saad, "Typical performance of gallager-type error-correcting codes," Phys. Rev. Lett., vol. 84, p. 1355, 2000.
18. T. Murayama, Y. Kabashima, D. Saad, and R. Vicente, "Statistical physics of regular low-density parity-check error-correcting codes," Phys. Rev., vol. E 62, p. 1577, 2000.
19. A. Montanari, "The glassy phase of gallager codes," Europ. Phys. J. B, vol. 23, p. 121, 2001.
20. S. Franz, M. Leone, A. Montanari, and F. Ricci-Tersenghi, "Dynamic phase transition for decoding algorithms," Phys. Rev. E, vol. 66, p. 046120, 2002.
21. A. Montanari and N. Sourlas, "The statistical mechanics of turbo codes," Europ. Phys. J. B, vol. 18, pp. 107-119, 2000.
22. A. Montanari, "Turbo codes: The phase transition," Europ. Phys. J. B, vol. 18, pp. 121-136, 2000.
23. F. Guerra, "Broken replica symmetry bounds in the mean field spin glass model," Commun. Math. Phys., vol. 233, p. 1, 2003.
24. F. Guerra and F. L. Toninelli, "The thermodynamic limit in mean field spin glass models," Commun. Math. Phys., vol. 230, pp. 1, 71-79, 2002.
25. S. Franz and M. Leone, "Replica bounds for optimization problems and diluted spin systems," J. Statist. Phys., vol. 111, p. 535, 2003.
26. S. Franz, M. Leone, and F. L. Toninelli, "Replica bounds for diluted nonpoissonian spin systems," J. Phys. A: Math Gen, vol. 36, p. 10967, 2003.
27. M. Luby, "LT-codes," in Proc. 43rd Symp. Foundations of Computer Science (FOCS 2002), Vancouver, BC, Canada, Nov. 2002, p. 271.
28. R. M. Tanner, "A recursive approach to low complexity codes," IEEE Trans. Inf. Theory, vol. IT-27, no. 5, pp. 533-547, Sep. 1981.
29. M. Luby, M. Mitzenmacher, A. Shokrollahi, D. A. Spielman, and V. Stemann, "Practical loss-resilient codes," in Proc. 29th Annu. ACM Symp. Theory of Computing, El Paso, TX, May 1997, pp. 150-159.
30. T. Richardson and R. Urbanke, Modern Coding Theory, unpublished manuscript. Draft available at http://lthcwww.epfl.ch/index.php.
31. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. Feb., pp. 498-519, 2001.
32. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference. San Francisco, CA: Morgan Kaufmann, 1988.
33. T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, "Design of capacity-approaching irregular low-density parity-check codes," IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 619-637, Feb. 2001.
34. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
35. K. Azuma, "Weighted sums of certain dependent random variables," Tohoku Math. J., vol. 19, pp. 357-354, 1967.
36. C. Di, T. Richardson, and R. Urbanke, "Weight distribution of low-density parity-check codes," IEEE Trans. Inf. Theory. Available at http://lthcwww.epfl.ch/index.php, submitted for publication.
37. C. Measson and R. Urbanke, "Further analytic properties of EXIT-like curves and applications," in Proc. IEEE Int. Symp. Information Theory (ISIT-2003), Yokohama, Japan, Jun./Jul. 2003, p. 266.
38. C. Measson, A. Montanari, and C. Measson, "Maxwell's construction: The hidden bridge between maximum-likelihood and iterative decoding," in Proc. IEEE Int. Symp. Information Theory (ISIT-2004), Chicago, IL, Jun./Jul. 2004, p. 225.
39. J. S. Yedidia, W. T. Freeman, and Y. Weiss, "Constructing free energy approximations and generalized belief propagation algorithms," IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2282-2312, Aug. 2005.
40. D. Aldous and J. A. Fill. Reversible Markov Chains and Random Walks on Graphs. [Online]. Available: www.stat.berkeley.edu/users/aldous/book.html
41. C. Measson, A. Montanari, T. Richardson, and R. Urbanke. Life above threshold: From list decoding to area theorem and MSE. presented at 2004 IEEE Information Theory Workshop, San Antonio, TX. [Online]. Available: http://arxiv.org/abs/cs.IT/0410028
42. I. Sason and R. Urbake, "Parity-check density versus performance of binary linear block codes over memoryless symmetric channels," IEEE Trans. Inf. Theory, vol. 49, no. 7, pp. 1611-1635, Jul. 2003.