Statistical physics of signal estimation in gaussian noise: Theory and examples of phase transitions

IEEE Transactions on Information Theory

Volumn 56, Issue 3, 2010, Pages 1400-1416

  Merhav, Neri ^a   Guo, Dongning ^b   Shamai, Shlomo ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b Northwestern University   (United States)

Author keywords

Denoising, de Bruijns identity; Gaussian channel; Minimum mean square error (MMSE) estimation; Phase transitions; Random energy model; Spin glasses; Statistical mechanics

Indexed keywords

DE-NOISING; GAUSSIAN CHANNELS; MINIMUM MEAN SQUARE ERROR (MMSE) ESTIMATION; MINIMUM MEAN-SQUARE-ERROR ESTIMATIONS; RANDOM ENERGY MODEL; RANDOM ENERGY MODELS;

BLOCK CODES; ESTIMATION; FISHER INFORMATION MATRIX; GAUSSIAN NOISE (ELECTRONIC); GLASS; MAN MACHINE SYSTEMS; MEAN SQUARE ERROR; PHYSICS; SIGNAL ANALYSIS; SPIN DYNAMICS; STATISTICAL MECHANICS; STATISTICS;

PHASE TRANSITIONS;

EID: 77949531339     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2039047     Document Type: Article

References (30)

1. R. J. Baxter, Exactly Solved Models in Statistical Mechanics. New York: Academic, 1982.
2. A. Bianchi, A. Bovier, and D. Ioffe, "Sharp asymptotics for metastability in the random field Curie-Weiss model," Electr. J. Prob., vol.14, pp. 1541-1603, 2008.
3. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. Hoboken, NJ: Wiley, 2006.
4. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications. New York: Springer, 1998.
5. B. Derrida, "Random-energy model: Limit of a family of disordered models," Phys. Rev. Lett., vol.45, no.2, pp. 79-82, Jul. 1980.
6. B. Derrida, "The random energy model," Phys. Rep.(Rev. Section of Phys. Lett.), vol.67, no.1, pp. 29-35, 1980.
7. B. Derrida, "Random-energy model: An exactly solvable model for disordered systems," Phys. Rev. B, vol.24, no.5, pp. 2613-2626, Sep. 1981.
8. B. Derrida, "A generalization of the random energy model which includes correlations between energies," J. de Phys.-Lett., vol.46, May 1985, L-401-1107
9. B. Derrida and E. Gardner, "Solution of the generalized random energy model," J. Phys. C: Solid State Phys., vol.19, pp. 2253-2274, 1986.
10. B. Derrida and E. Gardner, "Magnetic properties and the function of the generalized random-energy model," J. Phys.C: Solid State Phys., no.19, pp. 5783-5798, 1986.
11. R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, ser. A series of comprehensive studies in mathematics. New York: Springer-Verlag, 1985, vol.271.
12. D. Guo, S. Shamai, and S. Verdú, "Mutual information and minimum mean-square error in Gaussian channels," IEEE Trans. Inf. Theory, vol.51, no.4, pp. 1261-1282, Apr. 2005.
13. D. Guo and T. Tanaka, , M. Honig, Ed., "Generic multiuser detection and statistical physics," in Advances in Multiuser Detection. Hoboken, NJ: Wiley-IEEE, 2009.
14. D. Guo and S. Verdú, "Randomly spread CDMA: Asymptotics via statistical physics," IEEE Trans. Inf. Theory, vol.51, no.6, pp. 1982-2010, Jun. 2005.
15. J. Honerkamp, Statistical Physics: An Advanced Approach With Applications. New York: Springer-Verlag, 2002.
16. E. T. Jaynes, "Information theory and statistical mechanics," Phys. Rev. A, vol.106, pp. 620-630, May 1957.
17. E. T. Jaynes, "Information theory and statistical mechanics-II," Phys. Rev. A, vol.108, pp. 171-190, Oct. 1957.
18. N. Merhav, "An identity of Chernoff bounds with an interpretation in statistical physics and applications in information theory," IEEE Trans. Inf. Theory, vol.54, no.8, pp. 3710-3721, Aug. 2008.
19. N. Merhav, "Relations between random coding exponents and the statistical physics of random codes," IEEE Trans. Inf. Theory, vol.55, no.1, pp. 83-92, Jan. 2009.
20. N. Merhav, "The generalized random energy model and its application to the statistical physics of ensembles of hierarchical codes," IEEE Trans. Inf. Theory, vol.55, no.3, pp. 1250-1268, Mar. 2009.
21. N. Merhav, D. Guo, and S. Shamai (Shitz), "Statistical physics of signal estimation in Gaussian Noise: Theory and examples of phase transitions," Dec. 29, 2008 [Online]. Available: http://arxiv.org/PS-cache/ arxiv/pdf/0812/0812.4889v1.pdf
22. M. Mézard and A. Montanari, Information, Physics and Computation. Oxford, U.K.: Oxford Univ. Press, 2009.
23. K. R. Narayanan and A. R. Srinivasa, "On the thermodynamic temperature of a general distribution," Nov. 10, 2007 [Online]. Available: http://arxiv.org/PS-cache/arxiv/pdf/0711/0711.1460v2.pdf
24. J. W. Negele and H. Orland, Quantum Many-Particles Systems. Reading, MA: Addison-Wesley, 1988, Frontier in Physics Lecture Notes.
25. H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction, ser. Number 111 in International Series of Monographs on Physics. Oxford , U.K.: Oxford Univ. Press, 2001.
26. D. P. Palomar and S. Verdú, "Representation of mutual information via input estimates," IEEE Trans. Inf. Theory, vol.53, no.2, pp. 453-470, Feb. 2008.
27. M. Peleg, A. Sanderovich, and S. Shamai (Shitz), "On extrinsic information of good codes operating over Gaussian channels," Eur. Trans. Telecommun., vol.18, no.2, pp. 133-139, 2007.
28. O. Shental and I. Kanter, "Shannon meets Carnot: Generalized second thermodynamic law," Euro. Phys. Lett., vol.85, pp. 10006(p1)-10006(p5), Jan. 2009.
29. N. Sourlas, "Spin-glass models as error-correcting codes," Nature, vol.339, pp. 693-695, Jun. 1989.
30. N. Sourlas, "Spin glasses, error-correcting codes and finite-temperature decoding," Europhys. Lett., vol.25, pp. 159-164, 1994.