Relations between random coding exponents and the statistical physics of random codes

IEEE Transactions on Information Theory

Volumn 55, Issue 1, 2009, Pages 83-92

  Merhav, Neri ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Error exponents; Free energy; Partition function; Phase transitions; Random coding; Random energy model (REM)

Indexed keywords

DECODING; FREE ENERGY; FUNCTIONS; PARAMAGNETIC MATERIALS; PHASE DIAGRAMS; PHYSICS; PROBABILITY; PROBABILITY DENSITY FUNCTION; PROGRAMMING THEORY; RANDOM PROCESSES; SEMICONDUCTOR MATERIALS;

CHANNEL STATISTICS; CODE WORDS; DISCRETE MEMORYLESS CHANNELS; ERROR EXPONENTS; MINIMUM ENERGIES; PARTITION FUNCTION; PROBABILITY OF ERRORS; RANDOM CODES; RANDOM CODING; RANDOM CODING EXPONENTS; RANDOM ENERGY MODEL (REM); STATISTICAL PHYSICS; THREE PHASIS;

PHASE TRANSITIONS;

EID: 58249140401     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.2008133     Document Type: Article

References (44)

1. I. Kanter and D. Saad, "Error-correcting codes that nearly saturate Shannon's bound," Phys. Rev. Lett., vol. 83, no. 13, pp. 2660-2663, Sep. 1999.
2. Y. Kabashima, N. Sazuka, K. Nakamura, and D. Saad, "Tighter decoding reliability bound for Gallager's error-correcting code," Phys. Rev. E, vol. 64, pp. 046113-1-046113-4, 2001.
3. N. Sourlas, "Spin-glass models as error - Correcting codes," Nature vol. 339, pp. 693-695, Jun. 1989.
4. N. Sourlas, "Spin glasses, error-correcting codes and finite-temperature decoding," Europhys. Lett., vol. 25, pp. 159-164, 1994.
5. Y. Kabashima and D. Saad, "Statistical mechanics of error correcting codes," Europhys. Lett., vol. 45, no. 1, pp. 97-103, 1999.
6. D. Guo and S. Verdú, "Multiuser detection and statistical physics," Aug. 2002, Preprint.
7. O. Shental, I. Kanter, and A. J. Weiss, "Capacity of complexity-constrained noise-free CDMA," 2005, Preprint.
8. T. Tanaka, "A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors," IEEE Trans. Inf. Theory, vol. 48, no. 11, pp. 2888-2910, Nov. 2002.
9. T. Tanaka, "Statistical mechanics of CDMA multiuser demodulation," Europhys. Lett., vol. 54, no. 4, pp. 540-546, 2001.
10. A. Procacci and B. Scoppola, "Statistical mechanics approach to coding theory," J. Statist. Phys., vol. 96, no. 3/4, pp. 907-912, 1999.
11. T. Hosaka and Y. Kabashima, "Statistical mechanical approach to error exponents of lossy data compression," J. Phys. Soc. Japan, vol. 74, no. 1, pp. 488-497, Jan. 2005.
12. Y. Kabashima, K. Nakamura, and J. van Mourik, "Statistical mechanics of typical set decoding," Phys. Rev. E, vol. 66, p. 036125, 2002.
13. N. Sarshar and X. Wu, "Statistical mechanics of optimal networked source coding," Preprint.
14. T. Mutayama, "Statistical mechanics of the data compression theorem," J. Phys. A: Math. Gen., vol. 35, pp. L95-L100, 2002.
15. I. Rojdestvenski and M. C. Cottman, "Mapping of statistical physics to information theory with application to biological systems," J. Theor. Biol., pp. 43-54, 2000.
16. O. Shental and I. Kanter, "Shannon capacity of infinite-range spin-glasses," 2006, Preprint.
17. T. Mora and O. Rivoire, "Statistical mechanics of error exponents for error-correcting codes," Jun. 2006, arXiv:cOnd-mat/0606696.
18. Y. Kabashima and T. Hosaka, "Statistical mechanics for source coding with a fidelity criterion," Progr. Theor. Phys. Suppl., no. 157, pp. 197-204, 2005.
19. N. Sourlas, "Statistical Mechanics and Error Correcting Codes," arXiv:cOnd-mat/9811406v1 [cond-mat.stat-mech], Nov. 1998.
20. M. Mézard and A. Montanari, Information, Physics and Computation (draft), Feb. 27, 2006 [Online]. Available: http://www.stanford.edu/montanar/BOOK/book.html
21. A. Barg and G. D. Forney Jr., "Random codes: Minimum distances and error exponents," IEEE Trans. Inf. Theory, vol. 48, no. 9, pp. 2568-2573, Sep. 2002.
22. D. McAllester, "A statistical mechanics approach to large deviations theorems," [Online]. Available: http://citeseer.ist.psu.edu/ 443261.html, Preprint, 2007.
23. Y. Oono, "Large deviation and statistical physics," Progr. Theor. Phys. Suppl., no. 99, pp. 165-205, 1989.
24. P. T. Landsberg and V. Vedral, "Distributions and channel capacities in generalized statistical mechanics," Phys. Lett. A, vol. 247, pp. 211-217, Oct. 1998.
25. A. Montanari, "Two lectures on iterative decoding and statistical mechanics," arXiv:cOnd-mat/0512296, Dec. 14, 2005.
26. A. Montanari and R. Urbanke, "Modern coding theory: The statistical mechanics and computer science point of view," 2007, Preprint.
27. A. Montanari, "The glassy phase of Gallager codes," arXiv:cOnd-mat/ 0104079v1, Apr. 2001.
28. P. Ruján, "Finite temperature error-correcting codes," Phys. Rev. Lett., vol. 70, no. 19, pp. 2968-2971, May 1993.
29. S. Franz, M. Leone, A. Montanari, and F. Ricci-Tersenghi, "The dynamic phase transition for decoding algorithms," May 2, 2002, arXiv:cOnd-mat/ 0205051v1, also in Phys. Rev. E, vol. 66, 2002.
30. K. Tadaki, "A statistical mechanical interpretation of instantaneous codes," in Proc. IEEE Int. Symp. Information Theory (ISIT 2007 , Nice, France, Jun. 2007, pp. 1906-1910.
31. Y. Iba, "The Nishimori line and Bayesian statistics," J. Phys. A: Math. Gen., vol. 32, pp. 3875-3888, 1999.
32. B. Derrida, "Random-energy model: An exactly solvable model for disordered systems," Phys. Rev. B, vol. 24, no. 5, pp. 2613-2626, Sep. 1981.
33. J. Ziv, "Universal decoding for finite-state channels," IEEE Trans. Inf. Theory, vol. IT-31, no. 4, pp. 435-460, Jul. 1985.
34. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
35. S. Arimoto, "On the converse to the coding theorem for discrete memoryless channels," IEEE Trans. Inf. Theory, vol. IT-19, no. 3, pp. 357-359, May 1973.
36. G. Dueck and J. Körner, "Reliability function of a discrete memoryless channel at rates above capacity," IEEE Trans. Inf. Theory vol. IT-25, no. 1, pp. 82-85, Jan. 1979.
37. A. J. Viterbi and J. K. Omura, Principles of Digital Communication and Coding. New York: McGraw-Hill, 1979.
38. R. G. Gallager, "The random coding bound is tight for the average code," IEEE Trans. Inf. Theory, vol. IT-19, no. 2, pp. 244-246, Mar. 1973.
39. N. Merhav, "Error exponents of erasure/list decoding revisited via moments of distance enumerators," IEEE Trans. Inf. Theory, vol. 54, no. 10, pp. 4439-4447, Oct. 2008.
40. R. Etkin, N. Merhav, and E. Ordentlich, "Error exponents of optimum decoding for the interference channel," in Proc. IEEE Int. Symp. Information Theory (ISIT 2008), Toronto, ON, Canada, Jul. 2008, pp. 1778-1782, Full version submitted to IEEE Trans. Inf. Theory, October 2008..
41. Y. Kaspi and N. Merhav, "Error exponents of optimum decoding for degraded broadcast channels and degraded message sets," in Proc. IEEE Int. Symp. Information Theory (ISIT 2008), Toronto, ON, Canada, Jul. 2008, pp. 1518-1522.
42. A. Somekh-Baruch and N. Merhav, "Exact exponents for erasure/list decoding," in preparation.
43. N. Merhav, "The random energy model in a magnetic field and joint source-channel coding," Physica A: Statistical Mechanics and Its Applications, vol. 387, no. 22, pp. 5662-5674, Sep. 2008, doi:10.1016/ j.physa.2008.05.040.
44. N. Merhav, "The generalized random energy model and its application to the statistical physics of ensembles of hierarchical codes," IEEE Trans. Inf. Theory, to be published.