The generalized random energy model and its application to the statistical physics of ensembles of hierarchical codes

IEEE Transactions on Information Theory

Volumn 55, Issue 3, 2009, Pages 1250-1268

  Merhav, Neri ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Error exponents; Generalized random energy model (GREM); Phase transitions; Random coding; Spin glasses

Indexed keywords

DECODING; GLASS; PHYSICS; RANDOM VARIABLES; SPIN DYNAMICS; STATISTICAL MECHANICS; STRUCTURAL DESIGN;

CODE DESIGNS; DERRIDA; DESIGN PARAMETERS; ENERGY LEVELS; ERROR EXPONENTS; GENERALIZED RANDOM ENERGY MODEL (GREM); HIERARCHICAL STRUCTURES; RANDOM CODING; SPIN CONFIGURATIONS; STATISTICAL PHYSICS;

PHASE TRANSITIONS;

EID: 62749109684     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.2011445     Document Type: Article

References (49)

1. T. Hosaka and Y. Kabashima, "Statistical mechanical approach to error exponents of lossy data compression," J. Phys. Soc. Japan, vol. 74, no. 1, pp. 488-497, Jan. 2005.
2. T. Murayama, "Statistical mechanics of the data compression theorem," J. Phys. A: Math. Gen., vol. 35, pp. L95-L100, 2002.
3. Y. Kabashima and T. Hosaka, "Statistical mechanics for source coding with a fidelity criterion," Progr. Theor. Phys., pp. 197-204, 2005, Supplement no. 157.
4. K. Tadaki, "A statistical mechanical interpretation of instantaneous codes," in Proc. IEEE Int. Symp. Information Theory (ISIT 2007 , Nice, France, Jun. 2007, pp. 1906-1910.
5. M. Mézard and A. Montanari, Information, Physics and Computation Nov. 9, 2007 [Online]. Available: http://www.stanford.edu/~montanar/ BOOK/book.html, draft.
6. I. Kanter and D. Saad, "Error-correcting codes that nearly saturate Shannon's bound," Phys. Rev. Lett., vol. 83, no. 13, pp. 2660-2663, Sep. 1999.
7. A. Procacci and B. Scoppola, "Statistical mechanics approach to coding theory," J. Statist. Phys., vol. 96, no. 3/4, pp. 907-912, 1999.
8. N. Sourlas, "Spin-glass models as error-correcting codes," Nature vol. 339, pp. 693-695, Jun. 1989.
9. N. Sourlas, "Spin glasses, error-correcting codes and finite-temperature decoding," Europhys. Lett., vol. 25, pp. 159-164, 1994.
10. Y. Kabashima, N. Sazuka, K. Nakamura, and D. Saad, "Tighter decoding reliability bound for Gallager's error-correcting code," Phys. Rev. E., vol. 64, pp. 046113-1-046113-4, 2001.
11. Y. Kabashima and D. Saad, "Statistical mechanics of error correcting codes," Europhys. Lett., vol. 45, no. 1, pp. 97-103, 1999.
12. O. Shental and I. Kanter, "Shannon Capacity of Infinite-Range Spin-Glasses," Bar-Ilan Univ., Ramat-Gan, Israel, 2005, Tech. Rep..
13. T. Mora and O. Rivoire, "Statistical Mechanics of Error Exponents for Error-Correcting Codes," June 2006, arXiv:cOnd-mat/0606696.
14. A. Montanari, "Two Lectures on Iterative Decoding and Statistical Mechanics" Dec. 14, 2005, arXiv:cOnd-mat/0512296.
15. S. Franz, M. Leone, A. Montanari, and F. Ricci-Tersenghi, "The Dynamic Phase Transition for Decoding Algorithms," May 2, 2002, arXiv:cOnd-mat/020505.
16. A. Montanari and R. Urbanke, "Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View," 2007, preprint.
17. A. Montanari, "The Glassy Phase of Gallager Codes," Apr. 4, 2001, arXiv:cOnd-mat/0104079v1.
18. P. Ruján, "Finite temperature error-correcting codes," Phys. Rev. Lett., vol. 70, no. 19, pp. 2968-2971, May 1993.
19. N. Sourlas, "Statistical mechanics and error correcting codes," in From Statistical Physics to Statistical Inference and Back, P. Grassberger and J. P. Nadal, Eds. Dordrecht, The Netherlands: Kluwer Academic, 1994, pp. 195-204.
20. A. Montanari, "Tight bounds for LDPC and LDGM codes under MAP decoding," IEEE Trans. Inf. Theory, vol. 51, no. 9, pp. 3221-3246, Sep. 2005.
21. T. C. Dorlas and J. R. Wedagedera, "Phase diagram of the random energy model with higher order ferromagnetic term and error correcting codes due to Sourlas," Phys. Rev. Lett., vol. 83, no. 21, pp. 4441-4444, Nov. 1999.
22. B. Derrida, "Random-energy model: Limit of a family of disordered models," Phys. Rev. Lett., vol. 45, no. 2, pp. 79-82, Jul. 1980.
23. B. Derrida, "The random energy model," Phys. Repts. (Review Section of Physics Letters), vol. 67, no. 1, pp. 29-35, 1980.
24. B. Derrida, "Random-energy model: An exactly solvable model for disordered systems," Phys. Rev. B., vol. 24, no. 5, pp. 2613-2626, Sep. 1981.
25. M. Campellone, S. Franz, and G. Parisi, "Crossover behavior of a one-dimensional random energy model," Phys. Rev. E., vol. 5, no. 5, pp. 5455-5460, Nov. 1998.
26. N. B. Jana, "Exponential Random Energy Model," Feb. 28, 2006, arXiv:mAth/0602670v1 math.PR.
27. N. Merhav, "Relations between random coding exponents and the statistical physics of random codes," IEEE Trans. Inf. Theory vol. 55, no. 1, pp. 83-92, Jan. 2009.
28. B. Derrida, "A generalization of the random energy model which includes correlations between energies," J. Phys.-Lettres, vol. 46, p. L-401-107, May 1985.
29. B. Derrida and E. Gardner, "Solution of the generalised random energy model," J. Phys. C: Solid-State Phys., vol. 19, pp. 2253-2274, 1986.
30. B. Derrida and E. Gardner, "Magnetic properties and the function q x) of the generalised random-energy model," J. Phys. C: Solid State Phys., no. 19, pp. 5783-5798, 1986.
31. E. Gardner and B. Derrida, "The probability distribution of the partition function of the random energy model," J. Phys. A: Math. Gen., vol. 22, pp. 1975-1981, 1989.
32. T. C. Dorlas and W. M. B. Dukes, "On the Generalised Random Energy Model" 2001, preprint.
33. D. B. Saaskian, "Diluted Generalized Random Energy Model," Nov. 22, 1997, arXiv:cOnd-mat/979238v2 [cond-mat.dis-nn].
34. A. Bovier and I. Kurkova, "Derrida's generalised random energy models 1: Models with finitely many hierarchies," Ann. I. Poincaré-PR 40 pp. 439-480, 2004.
35. A. Bovier and I. Kurkova, "Derrida's Generalised Random Energy Models 2: Gibbs Measures and Probability Cascades," 2002, preprint.
36. S. F. Edwards and P. W. Anderson, "Theory of spin glasses," J. Phys. F., vol. 5, pp. 965-974, 1975.
37. D. Sherrington and S. Kirkpatrick, "Solvable model of a spin glass," Phys. Rev. Lett., vol. 35, pp. 1792-1796, 1975.
38. T. M. Cover and E. Ordentlich, "Universal portfolios with side information," IEEE Trans. Inf. Theory, vol. 42, no. 2, pp. 348-363, Mar. 1996.
39. W. H. R. Equitz and T. M. Cover, "Successive refinement of information," IEEE Trans. Inf. Theory, vol. 37, no. 2, pp. 269-274, Mar. 1991.
40. V. N. Koshelev, "Hierarchical coding of discrete sources," Probl. Pered. Inform., vol. 16, pp. 186-203, 1980, English translation: Probl. Inf. Transm., vol. 16, no. 3, pp. 31-49, 1980..
41. B. Rimoldi, "Successive refinement of information: Characterization of achievable rates," IEEE Trans. Inf. Theory, vol. 40, no. 1, pp. 253-259, Jan. 1994.
42. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. Hoboken, NJ: Wiley, 2006.
43. A. D. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder," IEEE Trans. Inf. Theory, vol. IT-22, no. 1, pp. 1-10, Jan. 1976.
44. S. I. Gel'fand and M. S. Pinsker, "Coding for channel with random parameters," Probl. Inf. and Contr., vol. 9, no. 1, pp. 19-31, 1980.
45. A. D. Wyner, "The wire-tap channel," Bell Syst. Tech. J., vol. 54, no. 8, pp. 1355-1387, Oct. 1975.
46. N. Merhav, "Error exponents of erasure/list decoding revisited via moments of distance enumerators," IEEE Trans. Inf. Theory, vol. 54, no. 10, pp. 4439-4447, Oct. 2008.
47. A. J. Viterbi and J. K. Omura, Principle of Digital Communication and Coding. New York: McGraw-Hill, 1979.
48. R. G. Gallager, "The random coding bound is tight for the average code," IEEE Trans. Inf. Theory, vol. IT-19, no. 2, pp. 244-246, Mar. 1973.
49. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.